WO2018209791A1

WO2018209791A1 - Method and system for reconstructing shape of 3d object

Info

Publication number: WO2018209791A1
Application number: PCT/CN2017/093512
Authority: WO
Inventors: 陈宝权; 丹尼尔科恩⋅奥尔; 菲尔⋅艾伯曼; 奥伦⋅卡齐尔; 周强; 罗泽刚; 安德雷⋅沙夫; 陈⋅格列佛
Original assignee: 山东大学
Priority date: 2017-05-19
Filing date: 2017-07-19
Publication date: 2018-11-22
Also published as: CN107228640B; CN107228640A

Abstract

A method and system for reconstructing the shape of a 3D object. The method comprises: immersing a 3D object into a liquid from multiple angles, and simultaneously tracking and recording changes in liquid level to construct several non-uniform cross subject slices; performing resampling measurement on the non-uniform cross subject slices to obtain uniform slices and liquid level information corresponding thereto; using a Gaussian kernel to smoothen the liquid level information corresponding to the uniform slices; constructing a sparse linear equation according to the smoothened liquid level information to approximate an immersion process of the 3D object in the liquid during immersion; solving the sparse linear equation system to reconstruct a voxel of the 3D object; converting the reconstructed 3D object voxel into a grid; and smoothening the grid to finally reconstruct the 3D object.

Description

Method and system for reconstructing 3D object shape

Technical field

The invention belongs to the field of 3D object shape reconstruction, and in particular relates to a 3D object shape reconstruction method and system.

Background technique

Current methods for acquiring and reconstructing 3D object shapes mainly focus on optical scanning. However, when the shape of the 3D object includes a height that is inaccessible to the line of sight of the scanner or an occlusion, the 3D object cannot be accurately acquired and reconstructed by optical scanning.

Therefore, based on conventional (optical) scanners, complex shapes cannot be properly acquired or reconstructed. In addition, some objects are made of glossy or transparent materials, which is another challenge that ordinary optics cannot handle.

Summary of the invention

In order to solve the deficiencies of the prior art, the present invention provides a method for reconstructing a shape of a 3D object, which adopts a liquid as a technical means of a sensor, and is more suitable for a 3D object having an occluded portion and a glass material than a conventional optical scanner. Good scanning results.

The method for reconstructing a 3D object shape of the present invention includes:

Step a: immersing the 3D object from the plurality of angles into the liquid, and simultaneously tracking the change of the liquid level, and constructing a plurality of non-uniform intersecting object slices;

Step b: performing resampling measurement on the non-uniform cross object slice to obtain a unified slice and corresponding water level information;

Step c: smoothing the water level information corresponding to the unified slice by using a Gaussian kernel;

Step d: construct a sparse linear equation system according to the smoothed water level information to approximate the immersion process of the 3D object in the liquid during the immersion process;

Step e: solving the above-mentioned sparse linear equations to reconstruct voxels of the 3D object;

Step f: converting the reconstructed 3D object voxel into a mesh;

Step g: Smooth the mesh and finally reconstruct the 3D object.

Further, in the step a, the 3D object is immersed into the liquid from a plurality of angles using a jig.

The invention controls the uniform descending of the 3D object by the clamp, measures the weight change multiple times during the descending process, and finally obtains the volume of the plurality of volume slices during the descending process.

Further, before step d, the method further comprises: performing an experimental test on the fixture to obtain water level information corresponding to the unified slice of the smoothed fixture, and finally obtaining smoothed water level information of only the 3D object.

This can eliminate the influence of the clamp steps a to c, and ultimately improve the accuracy of the reconstructed 3D object.

Further, in step e, the LSMR is used to find a sparse linear equation system.

In order to utilize sparsity, the present invention uses LSMR based on the Golub-Kahan double diagonalization process. LSMR iteratively Find a solution to the sparse problem while taking advantage of the sparsity of matrix B.

Further, in step f, the reconstructed 3D object voxels are converted into a mesh using a method of isosurface construction.

Further, in the process of transforming the reconstructed 3D object voxel into a mesh by using the isosurface construction method, the Marching Cubes algorithm is used to process the 3D object voxels one by one, and the cube intersecting the isosurface is classified, and the interpolation is used. The intersection of the isosurface and the cube edge is finally obtained.

Further, the specific process of obtaining the grid includes:

1 Construct an index table of 256 intersecting relations according to the symmetric relationship between the isosurface and the voxel;

2 extract the 8 vertices of the cube, form a voxel and number the 8 vertices;

3 determining whether the vertex is in-plane or out-of-plane according to a comparison of each vertex and a threshold;

4 The 01 strings composed of the 8 vertices form an 8-bit index value;

5 Use the index value to find the corresponding relationship in the index table above, and find the point with each side of the cube;

6 using the intersection to form a triangular patch or a polygon patch;

7 traverse all voxels of the 3D image and repeat 2 to 6.

The invention also provides a 3D object shape reconstruction system.

A 3D object shape reconstruction system of the present invention includes:

A non-uniform cross-object slicing building module is used for simultaneously recording a liquid level change in a process of immersing a 3D object from a plurality of angles into a liquid, and constructing a plurality of non-uniform cross-object slices;

The unified slice and its corresponding water level information acquisition module are configured to resample the non-uniform cross object slice to obtain a unified slice and corresponding water level information;

a water level information smoothing module, configured to smooth water level information corresponding to the unified slice by using a Gaussian kernel;

An immersion process approximation module for constructing a sparse linear equation system based on the smoothed water level information to approximate an immersion process of the 3D object in the liquid during the immersion process;

Reconstructing a voxel module of the 3D object for solving the above-described sparse linear equations to reconstruct a voxel of the 3D object;

a mesh transformation module for converting a reconstructed 3D object voxel into a mesh;

A 3D object reconstruction module for smoothing the mesh and finally reconstructing the 3D object.

Further, in the non-uniform cross object slice building module, the 3D object is immersed into the liquid from a plurality of angles by using a jig.

Further, the system further comprises a fixture water level information eliminating module, which is used for experimental testing of the fixture, and obtains the water level information corresponding to the unified slice of the smoothed fixture, and finally obtains the smoothed water level information of only the 3D object.

Compared with the prior art, the beneficial effects of the present invention are:

(1) The present invention adopts a liquid as a technical means of a sensor, and obtains a better scanning effect than a conventional optical scanner for a three-dimensional object having an occluded portion and a glass material.

(2) Compared with traditional scanning methods, such as CT scanning, structured light scanning, the economy is cheap.

(3) The present invention controls the uniform drop of the 3D object by the jig, measures the weight change multiple times during the descent, and finally obtains the volume of the plurality of volume slices during the descent.

DRAWINGS

The accompanying drawings, which are incorporated in the claims of the claims

1 is a flow chart of a method for reconstructing a shape of a 3D object of the present invention;

2 is a schematic structural view of a 3D object shape reconstruction system of the present invention;

Figure 3 (a) is a reconstruction effect diagram of the shape of the 3D object of the present invention with a sampling number of 100 dips;

Figure 3 (b) is a reconstruction effect diagram of the shape of the 3D object of the present invention with a sampling number of 325 dips;

Figure 3 (c) is a reconstruction effect diagram of the shape of the 3D object of the present invention with a sampling number of 550 dips;

Figure 3 (d) is a reconstruction effect diagram of the shape of the 3D object of the present invention with a sampling number of 775 dips;

Fig. 3(e) is a reconstruction effect diagram of the shape of the 3D object of the present invention with a sampling number of 1000 dips.

detailed description

It should be noted that the following detailed description is illustrative and is intended to provide a further description of the application. All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise indicated.

It is to be noted that the terminology used herein is for the purpose of describing particular embodiments, and is not intended to limit the exemplary embodiments. As used herein, the singular " " " " " " There are features, steps, operations, devices, components, and/or combinations thereof.

1 is a flow chart of a method of reconstructing a 3D object shape of the present invention.

As shown in FIG. 1, the method for reconstructing a 3D object shape of the present invention includes:

Step a: The 3D object is immersed into the liquid from a plurality of angles, and at the same time, the liquid level change is recorded, and a number of non-uniform cross-object slices are constructed.

Specifically, in the step a, the 3D object is immersed into the liquid from a plurality of angles using a jig.

In a specific implementation process, the invention uses a mechanical arm as a clamp to control the uniform velocity of the object, and the change of the liquid level volume uses a weight sensor, and the sensor converts the volume change into a weight change under the liquid container, and then passes the weight. The transformation pushes out the volume change. The weight change is measured several times during the descent process, and finally, the volume of many volume slices can be obtained during the descent.

Step b: Perform resampling measurement on the non-uniform cross object slice to obtain a unified slice and corresponding water level information.

Since the object drops by d units and the water rises by k units in the i-th step during the descent of the 3D object, the volume measurement corresponds to the volume of the object in the given direction of vertical d+k unit length. Where i, d, and k are all positive integers.

The information obtained above is non-uniform cross-object slice data, and the above data needs to be pre-processed, so the non-uniform processing is performed by a continuous equation and re-sampled into a unified form, and the width of the newly sampled volume slices is the same.

Step c: Smoothing the water level information corresponding to the unified slice by using a Gaussian kernel.

This can reduce high frequency noise and ultimately improve the accuracy of the reconstructed 3D object.

Step d: Construct a sparse linear equation system based on the smoothed water level information to approximate the immersion process of the 3D object in the liquid during the immersion process.

Specifically, before step d, the method further comprises: performing an experimental test on the fixture to obtain water level information corresponding to the unified slice of the smoothed fixture, and finally obtaining smoothed water level information of only the 3D object.

The present invention simulates the impregnation process by acting on a rotation and summation matrix that vectorizes voxel objects (imagine a small lattice). The rotation matrix represents the direction of the object (since the invention requires immersion of the object to be measured from different directions), and the summation matrix S represents the height of the water.

Thus, for each direction (there is an immersion experiment in this direction and the measured data is obtained), the present invention can write a set of linear equations in the form of S*(R_theta)*object=measure.

In the present invention, it is an explicit writing of the equation: {S}*R=v.

Where S and R are matrices that act locally on a row or adjacent elements, and multiplication yields a sparse matrix ({S}*R).

Step e: Solving the above-mentioned sparse linear equations to reconstruct the voxels of the 3D object.

Specifically, in step e, the LSMR is used to find a sparse linear system of equations.

In order to utilize sparsity, the present invention uses LSMR based on the Golub-Kahan double diagonalization process. The LSMR iteratively finds a solution to the sparse problem while taking advantage of the sparsity of the matrix B.

Step f: Convert the reconstructed 3D object voxels into a mesh.

Specifically, in step f, the reconstructed 3D object voxels are converted to a mesh using a method of isosurface construction.

In the process of transforming the reconstructed 3D object voxels into mesh using the isosurface construction method, the MarchingCubes algorithm is used to process the 3D object voxels one by one, the cubes intersecting the isosurfaces are classified, and the isosurfaces are calculated by interpolation. The intersection with the edge of the cube ends up with the grid.

Specifically, the specific process of obtaining the grid includes:

2 extract the 8 vertices of the cube, form a voxel and number the 8 vertices;

4 The 01 strings composed of the 8 vertices form an 8-bit index value;

6 using the intersection to form a triangular patch or a polygon patch;

7 traverse all voxels of the 3D image and repeat 2 to 6.

Step g: Smooth the mesh and finally reconstruct the 3D object.

3(a) is a reconstruction effect diagram of the shape of the 3D object of the present invention with a sampling number of 100 dips;

From Fig. 3(a) to Fig. 3(e), it can be seen that the shape of the reconstructed D object becomes more and more accurate as the number of sampling times increases.

The invention adopts the liquid as the technical means of the sensor, and obtains a better scanning effect than the conventional optical scanner for the three-dimensional object having the occlusion portion and the glass material.

Therefore, the reconstruction method of the 3D object shape of the present invention is based on the ancient Archimedes principle, the Archimedes principle: the volume in which the liquid is displaced is equal to the volume in which the object is immersed in the water surface.

By immersing the object along the axis into the liquid, the liquid volume displacement can be measured and converted into a series of thin volume sections along the shape of the dip axis.

By repeatedly immersing the object in water in various angular directions, different volume displacements are produced and converted into so-called "immersion transformations". Collect samples at different angles. This in turn generates enough data to restore the geometry of the input shape.

Since the present invention is based on the use of a volume sample generated by a liquid interaction object, it is possible to acquire a portion of the occlusion and the line of sight that is inaccessible in a relatively simple manner.

The immersion transformation is reversible so that the three-dimensional shape of the object can be reconstructed therefrom. The inverse transformation needs to solve an undetermined problem. The matrices involved are large and sparse, almost orthogonal. Therefore, they have non-zero parts and structural characteristics that can be used to accelerate numerical calculations. Given a given set of samples for a given object, a pre-computed factorial decomposition matrix is used to calculate an approximate linear time immersion transformation of the number of samples, and a stable numerical solution to the problem is obtained.

When the scale of the problem is small to medium, the problem is solved by (implicitly) calculating the pseudo-inverse, resulting in a solution of the smallest norm. For very large problems, LSMR is applied, they do not need to be decomposed, and the residual norm is minimized.

A key advantage of the proposed method is the use of a liquid as a sensor. Unlike optical sensors, liquids have no line of sight requirements, they penetrate into the cavity and hidden parts of the object being measured, bypassing all visibility and optical limitations of conventional scanning devices.

2 is a schematic structural view of a reconstruction system of a 3D object shape of the present invention.

As shown in FIG. 2, a 3D object shape reconstruction system of the present invention includes:

Wherein, in the non-uniform cross object slice building module, the 3D object is immersed into the liquid from a plurality of angles by using a jig.

The system further comprises a fixture water level information elimination module, which is used for experimental testing of the fixture, and obtains the water level information corresponding to the unified slice of the smoothed fixture, and finally obtains the smoothed water level information of only the 3D object.

The above description of the specific embodiments of the present invention has been described with reference to the accompanying drawings, but it is not intended to limit the scope of the present invention. Those skilled in the art should understand that the skilled in the art does not require the creative work on the basis of the technical solutions of the present invention. Various modifications or variations that can be made are still within the scope of the invention.

Claims

A method for reconstructing a shape of a 3D object, comprising:

Step a: immersing the 3D object from the plurality of angles into the liquid, and simultaneously tracking the change of the liquid level, and constructing a plurality of non-uniform intersecting object slices;

Step b: performing resampling measurement on the non-uniform cross object slice to obtain a unified slice and corresponding water level information;

Step c: smoothing the water level information corresponding to the unified slice by using a Gaussian kernel;

Step d: construct a sparse linear equation system according to the smoothed water level information to approximate the immersion process of the 3D object in the liquid during the immersion process;

Step e: solving the above-mentioned sparse linear equations to reconstruct voxels of the 3D object;

Step f: converting the reconstructed 3D object voxel into a mesh;

Step g: Smooth the mesh and finally reconstruct the 3D object.
A method of reconstructing a shape of a 3D object according to claim 1, wherein in said step a, the 3D object is immersed into the liquid from a plurality of angles by means of a jig.
The method for reconstructing a 3D object shape according to claim 2, further comprising: performing an experimental test on the fixture to obtain water level information corresponding to the unified slice of the smoothed fixture, and finally The smoothed water level information of only 3D objects is obtained.
A method of reconstructing a 3D object shape according to claim 1, wherein in step e, the LSMR is used to find a sparse linear equation system.
The method for reconstructing a 3D object shape according to claim 1, wherein in step f, the reconstructed 3D object voxel is converted into a mesh by using an isosurface construction method.
The method for reconstructing a 3D object shape according to claim 5, wherein in the process of converting the reconstructed 3D object voxel into a mesh by using the isosurface construction method, the Marching Cubes algorithm is used to process one by one. The 3D object voxel classifies the cube intersecting the isosurface, and uses interpolation to calculate the intersection of the isosurface and the cube edge, and finally obtains the mesh.
A method for reconstructing a shape of a 3D object according to claim 6, wherein the specific process of obtaining the mesh comprises:

1 Construct an index table of 256 intersecting relations according to the symmetric relationship between the isosurface and the voxel;

2 extract the 8 vertices of the cube, form a voxel and number the 8 vertices;

3 determining whether the vertex is in-plane or out-of-plane according to a comparison of each vertex and a threshold;

4 The 01 strings composed of the 8 vertices form an 8-bit index value;

5 Use the index value to find the corresponding relationship in the index table above, and find the point with each side of the cube;

6 using the intersection to form a triangular patch or a polygon patch;

7 traverse all voxels of the 3D image and repeat 2 to 6.
A 3D object shape reconstruction system, comprising:

A non-uniform cross-object slicing building module is used for simultaneously recording a liquid level change in a process of immersing a 3D object from a plurality of angles into a liquid, and constructing a plurality of non-uniform cross-object slices;

The unified slice and its corresponding water level information acquisition module are configured to resample the non-uniform cross object slice to obtain a unified slice and corresponding water level information;

a water level information smoothing module, configured to smooth water level information corresponding to the unified slice by using a Gaussian kernel;

An immersion process approximation module for constructing a sparse linear equation system based on the smoothed water level information to approximate an immersion process of the 3D object in the liquid during the immersion process;

Reconstructing a voxel module of the 3D object for solving the above-described sparse linear equations to reconstruct a voxel of the 3D object;

a mesh transformation module for converting a reconstructed 3D object voxel into a mesh;

A 3D object reconstruction module for smoothing the mesh and finally reconstructing the 3D object.
A 3D object shape reconstruction system according to claim 8, wherein in the non-uniform cross object slice building module, the 3D object is immersed into the liquid from a plurality of angles by means of a jig.
A 3D object shape reconstruction system according to claim 9, wherein the system further comprises a fixture water level information eliminating module for performing an experimental test on the fixture to obtain a unified slice corresponding to the smoothed fixture. The water level information finally results in smoothed water level information for only 3D objects.