CN115908738A - Method and system for fusion and reconstruction of spatial structure association of independent visual target - Google Patents
Method and system for fusion and reconstruction of spatial structure association of independent visual target Download PDFInfo
- Publication number
- CN115908738A CN115908738A CN202211707921.2A CN202211707921A CN115908738A CN 115908738 A CN115908738 A CN 115908738A CN 202211707921 A CN202211707921 A CN 202211707921A CN 115908738 A CN115908738 A CN 115908738A
- Authority
- CN
- China
- Prior art keywords
- image
- image blocks
- reconstruction
- model
- target scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a method and a system for the spatial structure correlation fusion reconstruction of an independent vision target s Dividing n original divided image blocks to obtain n associated divided image blocks, performing associated division based on a graph model to obtain n associated divided image blocks, fusing the n original divided image blocks and the n associated divided image blocks, converting the n original divided image blocks into a world coordinate system to obtain n white-mode image blocks, and performing multi-view geographic reconstruction to obtain a texture map; and then adding the coordinates to a white model of the target scene to obtain a reconstructed three-dimensional model of the target scene. Aiming at the problem that a clear image with clear edges is difficult to obtain in the current image reconstruction method, the invention uses a mode based on scene image segmentation to obtainThe method comprises the steps of obtaining a noise-free clean image and a basic model of a target object, fusing the noise-free clean image and the basic model to obtain a complete independent three-dimensional model and a corresponding map, and having the advantages of low requirement on the number of images and good quality of a reconstructed model.
Description
Technical Field
The invention relates to a three-dimensional reconstruction technology in the field of computer vision, in particular to a method and a system for the spatial structure association fusion reconstruction of an independent vision target.
Background
Three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of a three-dimensional object, is the basis for processing, operating and analyzing the properties of the three-dimensional object in a computer environment, and is also a key technology for establishing virtual reality expressing an objective world in a computer. The existing three-dimensional reconstruction technical scheme is mainly divided into two types: (1) A multi-view based three-dimensional dense reconstruction scheme for techniques to recover dense structures of a scene from multi-view perspectives with some degree of overlap. And (2) performing three-dimensional sparse reconstruction based on the image sequence. The idea of three-dimensional sparse reconstruction based on image sequences is to estimate the camera parameters using the camera motion trajectory. The camera shoots a plurality of images at different viewpoints, and the position information and the motion trail of the camera are calculated by utilizing the images, so that three-dimensional point cloud is generated under a space coordinate system, and the space structure of an object is restored. The three-dimensional dense reconstruction scheme based on multiple views has high requirements on the tidiness of images, independent image data of a specific target are difficult to acquire under actual working conditions, and a clean and noise-free three-dimensional model is difficult to reconstruct; the three-dimensional sparse reconstruction based on the image sequence has high requirements on the number of images of image data, and the three-dimensional sparse reconstruction cannot be carried out because a sufficient number of images are difficult to acquire under actual working conditions.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the invention provides a method and a system for independent visual target space structure association fusion reconstruction aiming at the problem that clear-edged clean images are difficult to obtain in the current image reconstruction method.
In order to solve the technical problems, the invention adopts the technical scheme that:
a method for spatial structure association fusion reconstruction of an independent visual target comprises the following steps:
s101, extracting a significant image I from an image I of a target scene s ；
S102, for the significant image I s Carrying out image segmentation to obtain n original segmentation image blocks;
s103, performing associated segmentation based on a graph model on the n original segmented image blocks to obtain n associated segmented image blocks;
s104, fusing the n original divided image blocks obtained by image division and the n associated divided image blocks obtained by associated division based on the image model to finally obtain n divided image blocks;
s105, converting the n divided image blocks into a world coordinate system to obtain n white-mode image blocks;
s106, performing multi-view geographic reconstruction on each white mode image block in the n white mode image blocks to obtain a texture map of the target scene;
and S107, attaching the texture mapping of the target scene to a white mold of the target scene according to the coordinates to obtain a reconstructed three-dimensional model of the target scene, wherein the white mold of the target scene is a three-dimensional model established based on the three-dimensional point cloud data of the target scene.
Optionally, in step S101, a salient image I is extracted from the image I of the target scene s Refers to the extraction of the display from the image I of the target scene by using a DoG filterLiterary image I s 。
Optionally, the functional expression of the DoG filter is:
in the above equation, doG (x, y) is the result of filtering the location (x, y) in the image I of the target scene, e is a natural constant, σ 1 And σ 2 Two different variances, G (x, y, σ), respectively 1 ) And G (x, y, σ) 2 ) Respectively using σ 1 And σ 2 The result of the gaussian filtering of the location (x, y) in image I of the target scene.
Optionally, step S103 includes:
s201, according to n original divided image blocks I s1 ,I s2 ,...,I sn Constructing a graph model, wherein the vertex in the graph model is an image block, the edge in the graph model is an incidence relation whether the image blocks are connected, and the value of the incidence relation whether the image blocks are connected is 0 or 1;
s202, aiming at the image blocks in the image model, calculating the feature similarity between any image block pair to obtain n associated features I formed by the feature similarity between all the image block pairs s1 ,I s2 ,...,I sn According to n associated characteristics I s1 ,I s2 ,...,I sn A significant image I s Divided into n associated image blocks I h1 ,I h2 ,...,I hn 。
Optionally, the step S202 of calculating the feature similarity between any pair of image blocks refers to: regarding an image block as a vector, calculating a euclidean distance between two vectors of an arbitrary image block pair as a feature similarity between the image block pair.
Optionally, the step S104 of fusing the n original divided image blocks obtained by image division and the n associated divided image blocks obtained by associated division based on the image model means performing pixel addition to finally obtain the n divided image blocks I 1 ,I 2 ,...,I n 。
OptionallyStep S105 includes: for dividing n image blocks I 1 ,I 2 ,...,I n The coordinates (u, v, 1) of each pixel point (u, v) are subjected to space conversion under a pixel coordinate system-camera coordinate system-world coordinate system by using the internal reference phi of an imaging camera of the image I of the target scene to obtain the coordinates (x, y, z) of the pixel point under the world coordinate system, and finally the segmented image block P converted into the world coordinate system is obtained 1 ,P 2 ,...,P n 。
Optionally, after performing multi-view geographic reconstruction on each white model image block of the n white model image blocks in step S106, the corresponding white model image block is further subjected to deformation optimization restoration based on the original divided image block.
In addition, the invention also provides a system for the spatial structure association fusion reconstruction of the independent visual target, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the spatial structure association fusion reconstruction method of the independent visual target.
Furthermore, the present invention also provides a computer-readable storage medium having stored thereon a computer program for being programmed or configured by a microprocessor to perform the method for independent visual target spatial structure associated fusion reconstruction.
Compared with the prior art, the invention mainly has the following advantages: the method comprises extracting a saliency image I from an image I of a target scene s Dividing to obtain n original divided image blocks, performing associated division based on a graph model to obtain n associated divided image blocks, fusing and converting the n original divided image blocks and the n original divided image blocks into a world coordinate system to obtain n white-mode image blocks, and performing multi-view geographic reconstruction to obtain a texture map; and then adding the coordinates to a white model of the target scene to obtain a reconstructed three-dimensional model of the target scene. Aiming at the problem that a clear image with clear edges is difficult to obtain in the current image reconstruction method, the invention obtains a noise-free clean image and a basic model of a target object by using a scene image segmentation-based mode, and obtains a complete independent three-dimensional model and a corresponding chartlet by fusion, and has the advantages of low requirement on the number of images and high quality of a reconstructed modelThe advantage of good quality.
Drawings
FIG. 1 is a schematic diagram of a basic flow of a method according to an embodiment of the present invention.
Fig. 2 is an example of an image I of a target scene input in an embodiment of the present invention.
Fig. 3 is an example of a white mode of an input target scene in an embodiment of the present invention.
Fig. 4 is an example of a reconstructed three-dimensional model of a target scene obtained in an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the method for fusion and reconstruction of spatial structure association of independent visual objects of this embodiment includes:
s101, extracting a significant image I from an image I of a target scene s ；
S102, for the significant image I s Carrying out image segmentation to obtain n original segmented image blocks;
s103, performing associated segmentation based on a graph model on the n original segmented image blocks to obtain n associated segmented image blocks;
s104, fusing n original divided image blocks obtained by image division and n associated divided image blocks obtained by association division based on a graph model to finally obtain n divided image blocks;
s105, converting the n divided image blocks into a world coordinate system to obtain n white-mode image blocks;
s106, performing multi-view geographic reconstruction on each white mode image block in the n white mode image blocks to obtain a texture map of the target scene;
and S107, attaching the texture mapping of the target scene to a white mold of the target scene according to the coordinates to obtain a reconstructed three-dimensional model of the target scene, wherein the white mold of the target scene is a three-dimensional model established based on the three-dimensional point cloud data of the target scene. For example, in this embodiment, an input image I of a target scene is shown in fig. 2, a white mold of the input target scene is shown in fig. 3, and a reconstructed three-dimensional model of the target scene is finally obtained as shown in fig. 4.
In this embodiment, the slave target in step S101Image I of a scene from an extracted saliency image I s Refers to the extraction of a significant image I from an image I of a target scene by using a DoG filter s . As an optional implementation manner, the functional expression of the DoG filter in this embodiment is:
in the above equation, doG (x, y) is the result of filtering the location (x, y) in the image I of the target scene, e is a natural constant, σ 1 And σ 2 Two different variances, G (x, y, σ), respectively 1 ) And G (x, y, σ) 2 ) Respectively using σ 1 And σ 2 The result of the gaussian filtering of the location (x, y) in the image I of the target scene.
In this embodiment, step S103 includes:
s201, according to n original divided image blocks I s1 ,I s2 ,...,I sn Constructing a graph model, wherein the vertex in the graph model is an image block, the edge in the graph model is an incidence relation whether the image blocks are connected, and the value of the incidence relation whether the image blocks are connected is 0 or 1;
s202, aiming at the image blocks in the image model, calculating the feature similarity between any image block pair to obtain n associated features I formed by the feature similarity between all the image block pairs s1 ,I s2 ,...,I sn According to n associated characteristics I s1 ,I s2 ,...,I sn A significant image I s Divided into n associated image blocks I h1 ,I h2 ,...,I hn 。
In this embodiment, the step of calculating the feature similarity between any pair of image blocks in step S202 refers to: regarding an image block as a vector (e.g., a vector of 1 × 4096 in this embodiment), an euclidean distance between two vectors of an arbitrary image block pair is calculated as a feature similarity between the image block pair. In addition, other distance algorithms may be employed to calculate feature similarity between any pair of image blocks.
In this embodiment, the step S104 is to perform the image processingThe integration of the n original divided image blocks obtained by division and the n associated divided image blocks obtained by associated division based on the image model means that pixel addition is performed to finally obtain the n divided image blocks I 1 ,I 2 ,...,I n 。
In this embodiment, step S105 includes: for dividing n image blocks I 1 ,I 2 ,...,I n The coordinates (u, v, 1) of each pixel (u, v) are spatially converted under a pixel coordinate system-camera coordinate system-world coordinate system using an imaging camera internal reference phi of the image I of the target scene to obtain coordinates (x, y, z) of the pixel under the world coordinate system, and a relationship between the coordinates (u, v, 1) and the coordinates of the pixel under the world coordinate system can be expressed as:
(u,v,1)＝φ(x,y,z)，
in the above formula, phi is the imaging camera internal reference of the image I of the target scene, and finally the divided image block P converted into the world coordinate system is obtained 1 ,P 2 ,...,P n 。
Step S106, performing Multi-View geography (Multi View Geometry) reconstruction on each white mold image block in the n white mold image blocks to obtain an existing method, which may specifically refer to a website: http:// colmap.
In addition, considering that there may be distortion after performing Multi-View geographic (Multi View Geometry) reconstruction on each of the n white mode image blocks in step S106, as a preferred embodiment, after performing Multi-View geographic reconstruction on each of the n white mode image blocks in step S106, the embodiment further performs distortion optimization and restoration on the corresponding white mode image block based on the original segmented image block. It should be noted that, performing the deformation optimization and restoration is also an existing image modeling processing technology, for example, the method adopted in this embodiment may be referred to as: centin M, pezzotti N, signoroni A. Poisson-drive seamless completion of triangular screens [ J ]. Computer air-defined geometrical Design,2015, 35.
In this embodiment, the texture map of the target scene obtained in step S106 is in the png format, and the texture map of the png format of the target scene is attached to the white mold of the target scene according to the coordinates in step S107, so that the reconstructed three-dimensional model of the target scene can be obtained. The white model of the target scene is a three-dimensional model established based on three-dimensional point cloud data of the target scene, and a method for establishing the white model of the target scene is also an existing method, for example, see documents: kazhdan M, bolitho M, hoppe H. Poisson surface retrieval [ C ]// Proceedings of the four Europatics symposium on Geometry processing.2006,7.
In summary, for the problem that it is difficult to obtain a clean image with clear edges in the current image reconstruction method, the method for fusion reconstruction of spatial structure association of independent visual targets of this embodiment obtains a noise-free clean image and a basic model of a target object by using a scene image segmentation-based method, and performs fusion to obtain a complete independent three-dimensional model and a corresponding map, and has the advantages of low requirement on the number of images and good quality of a reconstructed model.
In addition, the embodiment also provides a system for the spatial structure dependent fusion reconstruction of the independent visual target, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the spatial structure dependent fusion reconstruction method of the independent visual target.
Furthermore, the present embodiment also provides a computer-readable storage medium, in which a computer program is stored, the computer program being programmed or configured by a microprocessor to perform the method for spatial structure-dependent fusion reconstruction of independent visual objects.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the scope of the present invention is not limited to the above embodiments, and all technical solutions that belong to the idea of the present invention belong to the scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.
Claims (10)
1. A method for spatial structure association fusion reconstruction of an independent visual target is characterized by comprising the following steps:
s101, extracting a significant image I from an image I of a target scene s ；
S102, for the significant image I s Carrying out image segmentation to obtain n original segmentation image blocks;
s103, performing associated segmentation based on a graph model on the n original segmented image blocks to obtain n associated segmented image blocks;
s104, fusing the n original divided image blocks obtained by image division and the n associated divided image blocks obtained by associated division based on the image model to finally obtain n divided image blocks;
s105, converting the n divided image blocks into a world coordinate system to obtain n white-mode image blocks;
s106, performing multi-view geographic reconstruction on each white mode image block in the n white mode image blocks to obtain a texture map of the target scene;
and S107, attaching the texture mapping of the target scene to a white mold of the target scene according to the coordinates to obtain a reconstructed three-dimensional model of the target scene, wherein the white mold of the target scene is a three-dimensional model established based on the three-dimensional point cloud data of the target scene.
2. The method for spatial structure dependent fusion reconstruction of independent vision object according to claim 1, wherein the step S101 extracts a salient image I from the image I of the object scene s Refers to the extraction of a significant image I from an image I of a target scene by using a DoG filter s 。
3. The method for spatial structure-dependent fusion reconstruction of independent visual objects according to claim 2, wherein the functional expression of the DoG filter is:
in the above equation, doG (x, y) is the result of filtering the location (x, y) in the image I of the target scene, e is a natural constant, σ 1 And σ 2 Two different variances, G (x, y, σ), respectively 1 ) And G (x, y, σ) 2 ) Respectively using σ 1 And σ 2 The result of the gaussian filtering of the location (x, y) in the image I of the target scene.
4. The method for spatial structure dependent fusion reconstruction of independent vision object according to claim 1, wherein step S103 comprises:
s201, according to n original divided image blocks I s1 ，I s2 ，...，I sn Constructing a graph model, wherein the vertex in the graph model is an association relation of whether image blocks are connected, and the edge in the graph model is an association relation of whether the image blocks are connected, and the value of the association relation is 0 or 1;
s202, aiming at the image blocks in the image model, calculating the feature similarity between any image block pair to obtain n associated features I formed by the feature similarity between all the image block pairs s1 ，I s2 ，...，I sn According to n associated characteristics I s1 ，I s2 ，...，I sn A significant image I s Divided into n associated image blocks I h1 ，I h2 ，...，I hn 。
5. The method for spatial structure correlation fusion reconstruction of independent visual object according to claim 4, wherein the step S202 of calculating the feature similarity between any pair of image blocks is: regarding an image block as a vector, calculating a euclidean distance between two vectors of an arbitrary image block pair as a feature similarity between the image block pair.
6. The method for spatial structure correlation fusion reconstruction of independent vision object as claimed in claim 1, wherein the step S104 of fusing both the n original segmented image blocks obtained by image segmentation and the n correlated segmented image blocks obtained by correlation segmentation based on the image model means performing pixel addition to finally obtain n segmented image blocks I 1 ，I 2 ，...，I n 。
7. The independent view of claim 1The method for fusion reconstruction of spatial structure association of visual target, wherein step S105 comprises: for dividing n image blocks I 1 ，I 2 ，...，I n The coordinates (u, v, 1) of each pixel point (u, v) are subjected to space conversion under a pixel coordinate system-camera coordinate system-world coordinate system by using the internal reference phi of an imaging camera of the image I of the target scene to obtain the coordinates (x, y, z) of the pixel point under the world coordinate system, and finally the segmented image block P converted into the world coordinate system is obtained 1 ，P 2 ，...，P n 。
8. The method for spatial structure association fusion reconstruction of an independent vision object according to claim 1, wherein in step S106, after performing multi-view geographic reconstruction on each of the n white model image blocks, the method further performs deformation optimization restoration on the corresponding white model image block based on the original segmented image block.
9. A system for independent visual target spatial structure associative fusion reconstruction, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the method for independent visual target spatial structure associative fusion reconstruction according to any one of claims 1 to 8.
10. A computer-readable storage medium, in which a computer program is stored, which is adapted to be programmed or configured by a microprocessor to perform a method for independent visual target spatial structure associative fusion reconstruction according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211707921.2A CN115908738A (en) | 2022-12-28 | 2022-12-28 | Method and system for fusion and reconstruction of spatial structure association of independent visual target |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211707921.2A CN115908738A (en) | 2022-12-28 | 2022-12-28 | Method and system for fusion and reconstruction of spatial structure association of independent visual target |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115908738A true CN115908738A (en) | 2023-04-04 |
Family
ID=86496959
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211707921.2A Pending CN115908738A (en) | 2022-12-28 | 2022-12-28 | Method and system for fusion and reconstruction of spatial structure association of independent visual target |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115908738A (en) |
-
2022
- 2022-12-28 CN CN202211707921.2A patent/CN115908738A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107845134B (en) | Three-dimensional reconstruction method of single object based on color depth camera | |
| KR100914845B1 (en) | Method and apparatus for 3d reconstructing of object by using multi-view image information | |
| EP2595116A1 (en) | Method for generating depth maps for converting moving 2d images to 3d | |
| JP2018534698A (en) | Method and system for large scale determination of RGBD camera attitude | |
| KR20180054487A (en) | Method and device for processing dvs events | |
| CN107767339B (en) | Binocular stereo image splicing method | |
| CN111882668A (en) | Multi-view three-dimensional object reconstruction method and system | |
| CN108124489B (en) | Information processing method, apparatus, cloud processing device and computer program product | |
| JP2008191072A (en) | Three-dimensional shape restoration method, three-dimensional shape restoring device, three-dimensional shape restoration program implemented with the method, and recording medium with the program stored | |
| CN110580720A (en) | camera pose estimation method based on panorama | |
| CN112734914A (en) | Image stereo reconstruction method and device for augmented reality vision | |
| Morgenstern et al. | Progressive non-rigid registration of temporal mesh sequences | |
| CN116012432A (en) | Stereoscopic panoramic image generation method and device and computer equipment | |
| Pan et al. | Depth map completion by jointly exploiting blurry color images and sparse depth maps | |
| Wang et al. | TerrainFusion: Real-time digital surface model reconstruction based on monocular SLAM | |
| Tseng et al. | Depth image super-resolution via multi-frame registration and deep learning | |
| Musialski et al. | Interactive Multi-View Facade Image Editing. | |
| CN111243062A (en) | Manufacturing method for converting planar mural into three-dimensional high-definition digital mural | |
| Mahmoud et al. | Fast 3d structure from motion with missing points from registration of partial reconstructions | |
| CN115908738A (en) | Method and system for fusion and reconstruction of spatial structure association of independent visual target | |
| CN116342831A (en) | Three-dimensional scene reconstruction method, three-dimensional scene reconstruction device, computer equipment and storage medium | |
| Klose et al. | Reconstructing Shape and Motion from Asynchronous Cameras. | |
| CN113034345B (en) | Face recognition method and system based on SFM reconstruction | |
| Sun et al. | Weakly-supervised reconstruction of 3D objects with large shape variation from single in-the-wild images | |
| JP6910622B2 (en) | Image processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |