CN101615304A - A method for generating robust visual shells - Google Patents

Abstract

A method for generating robust visual shells, belonging to the field of 3D modeling. The method comprises the following steps: acquiring an image set of an object at the same time at least two different viewpoints; performing parallel processing on each image in the image set to obtain and save contour information; using a line segment weighted intersection method according to the contour information Construct the initial line segment set model of the object; modify the initial line segment set model by using the linear filter of the line segment set center to obtain the result line segment set model; use the line segment set polygon detection to reconstruct the surface of the object to obtain the visible shell of the object. Calculate the visual shell of the object through the method of weighted line segment intersection, line segment set center linear filtering and line segment set polygon detection, which overcomes the shortcomings of traditional line segment intersection and surface boundary detection, while ensuring the accuracy of the algorithm results and enhancing the algorithm robustness.

Description

A method for generating robust visual shells

【Technical field】

The invention relates to a method for three-dimensional modeling, in particular to a method for generating a robust visible shell.

【Background technique】

Rapid acquisition of 3D object modeling technology has a wide range of applications in the fields of computer vision, virtual reality, artificial intelligence, machinery manufacturing, digital entertainment, and cultural relics protection. At present, the 3D model acquisition technology mainly includes three methods: using modeling software to construct a 3D model, obtaining a 3D model through instruments and equipment, and using image sequences to reconstruct a 3D model of a scene. The first method uses modeling software such as 3DMax and AutoCAD to construct a 3D model, which can accurately construct virtual objects and render strange effects, but the corresponding software is complex to use and the modeling cycle is long; the second method uses 3D scanning equipment to obtain 3D information , the modeling accuracy is high, the cycle is short, but the equipment is more expensive; the third method is based on the image sequence modeling, the user only needs to use an ordinary digital camera to shoot around the target object, and reconstruct the model through the photo image information, the data acquisition equipment is simple, efficient. Therefore, 3D reconstruction based on image sequence has become an important research object of computer vision because of its simple operation, low-cost equipment and high efficiency.

Two-dimensional image sequence modeling refers to the technology of reconstructing the original model of the object from the silhouette information of a group of two-dimensional photos of the object. The more shooting angles of the two-dimensional photos, the closer the reconstruction result is to the real object. Since the visible shell obtained by the intersection of the camera viewpoint and the cone generated by the silhouette contains the object, only an object model close to the real object can be obtained. At present, algorithms based on visual shell technology are mainly divided into two categories: voxel clipping methods and surface reconstruction methods.

Voxel cropping method, Martin and Aggarwal first proposed a voxel method to generate a visual shell from photos taken by cameras at different angles. Later, Chien proposed a method of using octree structure to represent the visual shell under the condition of parallel shooting. The data structure of octree is used to store the voxel information of spatial clipping. This kind of spatial expression can efficiently deal with the division of space. And store information, improve the operation efficiency. On the basis of Chien, R.Szeliski proposed an improved octree algorithm for shooting at any angle. But generally speaking, the method based on voxel cropping has the following disadvantages: (1) the accuracy of the reconstructed visible shell is low; (2) the time complexity and space complexity are high, even when the octree is introduced (3) The surface is indirectly generated by independent voxel elements, which lack close connection with each other and will generate redundant information in most cases. With the development of multi-processor parallel technology, voxel clipping methods can be well improved.

The surface reconstruction method can solve the third shortcoming of the voxel method. Matsuik proposed a polygon intersection method. This algorithm uses the cone formed by the camera's shooting viewpoint and the silhouette contour line on the picture to directly intersect to generate a visible shell. Later, Franco used the method of epipolar geometry and boundary detection to directly reconstruct the surface of the object, which improved the efficiency of the algorithm. The work included in this method includes polyhedron intersecting operations, polygon intersecting operations, and edge detection operations. Cut the surface of the object according to the side of each cone, and directly record the surface information of the object. Although this method solves the coherence between surfaces, as the number of photographs increases, the reconstructed surface of the object will be very complicated. And the stability of most surface reconstruction methods will be affected by the boundary points, often resulting in incomplete reconstruction of the surface, especially when the reconstructed object has a complex topology. In addition to this, different constraints need to be considered when modeling by surface reconstruction methods, such as smoothness constraints, occluded and textured edges, partial occlusion, reliability.

【Content of invention】

In view of this, it is necessary to provide a method for generating visual shells with high reconstruction accuracy and strong robustness.

A method for generating a robust visual shell, comprising the steps of: acquiring a collection of images of an object at the same moment under at least two different viewpoints; performing parallel processing on each image in the collection of images, obtaining contour information and saving ; Utilize the line segment weighted intersection method to construct the initial line segment set model of the object according to the profile information; use the line segment set center linear filter to correct the initial line segment set model to obtain the result line segment set model; utilize the line segment set polygon detection to reconstruct the surface of the object, Get the visual shell of the object.

Calculate the visual shell of the object through the method of weighted line segment intersection, linear filter of line segment set center and line segment set polygon detection method, which overcomes the inaccurate shortcomings of traditional line segment intersection and surface boundary detection, and at the same time ensures the accuracy of the calculation results of the method and enhances the robustness.

Preferably, the viewpoints are uniformly distributed on a horizontal circle centered on the object, and a digital camera is used to capture images of the object under each viewpoint.

Preferably, the contour information of the image is obtained by using an optimal ladder edge detection algorithm of a canny edge detection operator.

Preferably, the method for constructing the initial line segment set model includes the following steps:

For the contour of the object under each viewpoint, it is discretized into a set of points, and the points in the collection are all located on the contour of the object to form a point contour, and each viewpoint and the corresponding point contour form a viewpoint cone;

One bus line of the viewpoint cone intersects with other viewpoint cones to generate a bus line segment set, and the line segment weighted intersection method is used to process the bus line segment set to obtain the result set under one bus line of the viewpoint cone;

Intersect all generatrix lines in each viewpoint cone with the same number of corresponding point outline midpoints and other viewpoint cones, and the result set after the same processing as the above steps is the initial line segment set model of the object.

Preferably, the line segment weighted intersection method includes the following steps:

Represent all line segments in the set of bus line segments with a start point and an end point with one-dimensional parameters;

It is defined that the first line segment whose starting point is the minimum starting point of the bus line segment set and the maximum end point is the end point belongs to the weighted line segment set. The bus line segment set includes multiple line segment groups. Each line segment group is obtained by the intersection of the bus line and a viewpoint cone. The line segment group The number is the same as the number of other view frustums;

set a temporary set to empty;

checks whether each segment in a segment group overlaps all segments in the weighted segment set, and if so, inserts the overlapping result into the temporary set and increments the weighting counter associated with the overlapping result;

updating the weighted line segment set according to the temporary set, and emptying the temporary set;

Process each line segment group one by one until all the line segments in the bus line segment set are processed;

Select the line segment with the largest weighted counter from the final weighted line segment set to form the result set.

Preferably, the step of updating the weighted line segment set includes:

If the line segment in the weighted line segment set overlaps with the line segment in the temporary set, remove the overlapping part of the line segment to form a new line segment, otherwise keep it;

Merge the collection of line segments processed above with the temporary collection.

Preferably, surface reconstruction includes the following steps:

Classify the line segments in the initial line segment set model according to the normal vector;

In the same type of line segment, a set of planes is obtained according to the coplanarity of the line segments, and each plane is distinguished according to the intercept;

Adjacent planes intersect to obtain boundaries;

Adjacent boundaries are intersected to obtain vertices, which form polygons for surface reconstruction;

Repeat for all segments in the initial segment set model.

【Description of drawings】

Fig. 1 is a schematic diagram of viewpoint cone intersection;

Fig. 2 is a flowchart of generating a visual shell;

Fig. 3 is a flowchart of constructing an initial line segment set model of an object.

【Detailed ways】

In this embodiment, the method for generating a visual shell includes the following steps:

S1: Obtain a collection of images of an object at the same moment from at least two different viewpoints. This step is used to collect the original information of the object to be modeled. The more viewpoints collected, the greater the amount of information and the higher the reconstruction accuracy. In this embodiment, 12 viewpoints are collected. An ordinary digital camera can be used as the image acquisition tool of each viewpoint. The viewpoints are evenly distributed on a horizontal circle centered on the object. At the same moment, the object is photographed from 12 different viewpoints to obtain an image collection.

S2: Parallel processing is performed on each image in the image collection to obtain contour information and save it. In this step, the edge detection algorithm is used to process the contour information of the image. The edge of the image refers to the part of the image where the brightness of the local area changes significantly. The edge part of the image concentrates most of the information of the image. Determining and extracting the edge of the image is very important for the recognition and understanding of the entire image scene. In this embodiment, the optimal ladder edge detection algorithm of the canny edge detection operator is used to obtain the contour information of the image. Specifically include:

S202: Smooth the image with a Gaussian filter.

S204: Using the finite difference of the first-order partial derivative to calculate the magnitude and direction of the gradient;

S206: performing non-maximum suppression on the gradient amplitude;

S208: Detect and connect edges with a double-threshold algorithm. The contour information of the image is obtained according to the above method and saved for further processing.

S3: Construct the initial line segment set model of the object by using the line segment weighted intersection method according to the contour information. The detailed steps are as follows:

S302: Obtain a contour, and discretize the contour into a set of points to form a point contour. Each viewpoint and corresponding point outline form a viewpoint cone.

S304: A generatrix of the viewpoint cone intersects with other viewpoint cones to obtain a generatrix line segment set. Intersecting a generatrix with one of the other viewpoint cones may result in a point, or a line segment or multiple non-overlapping line segments. As shown in FIG. 1 , it is a schematic diagram in which a generatrix l of a viewpoint cone with viewpoint A as the vertex intersects with a certain viewpoint cone among other viewpoint cones. The intersection of the generatrix l and the viewpoint cone produces two line segments l ₁ , l ₂ , forming a line segment group S ₁ ={l ₁ , l ₂ }. The bus line l intersects with all other viewpoint cones to generate the bus line segment set {S _i } _i=1 ^N under the bus line l. In this embodiment, N=11, that is, the bus line l intersects with other 11 viewpoint cones , so the bus line segment set is {S ₁ , S ₂ , ..., S ₁₁ }, including 11 line segment groups.

S306: Using the line segment weighted intersection method to process the bus line segment set to obtain a result set under one bus line of the viewpoint cone. In the bus line segment set {S ₁ , S ₂ , ..., S ₁₁ }, between S ₁ and S ₁₁ , use the line segment weighted intersection method to process the bus line segment set to obtain the result under the bus line l Set, the core idea of the weighted intersection method of line segments is to represent the part with the most overlap among all line segments of multiple groups on the same straight line. There may be only one line segment in the result set, or there may be many line segments.

S308: Add the result set to the initial line segment set.

S310: Check whether the point directed from the view point is the last point processed on the outline of the point where it is located. If yes, proceed to the next step, if not, return to S304.

S312: Check whether the processed contour is the last contour. If yes, it means that the initial line segment set has been added, if not, return to S302.

S314: Obtain an initial line segment set model.

In the above step of S306, the example algorithm is as follows:

All line segments in the bus line segment set {S _i } _i=1 ^N are represented by a start point and an end point with one-dimensional parameters. Since all the line segments in the bus line segment set are located on the bus line, they have the same direction in space, so a line segment can be represented by only a start point and an end point with one-dimensional parameters. in $S_i={\∪}_{k\∈\{\mathrm{1,2},...,m_i\}}[a_{\mathrm{ik}},b_{\mathrm{ik}}],$ a _ik ≤ b _ik ≤ a _ik+1 . m _i represents the number of line segments in S _i , and a _ik ≤ b _ik ≤ _{a ik+1} represents the orderly arrangement of the values from the start point to the end point. As in this embodiment, S ₁ ={l ₁ , l ₂ }=∪ _{k∈{1, 2}} [a _1k , b _1k ].

S602: Find the minimum value a ₀ in a _ik and the maximum value b ₀ in b _ik . The first line segment is represented by [a ₀ , b ₀ ], and the first line segment belongs to the weighted line segment set S _OLSS , that is, S _OLSS ={[a ₀ , b ₀ ]}. The element counter M=1, records the number of elements in the weighted line segment set S _OLSS .

S604: Select a line segment group, such as S ₁ .

S606: Take a line segment in the line segment group, such as [a ₁₁ , b ₁₁ ]. Set the temporary set S as an empty set, namely

S608: Check whether the line segment taken in S606 overlaps with all the line segments in the weighted line segment set S _OLSS . The overlapping means that the starting point and/or the ending point of one of the two line segments is on the other line segment. If it overlaps, then insert the overlapping result into the temporary set S, add 1 to the weighted counter associated with the overlapping result, and insert the overlapping line segment in the weighted line segment set S _OLSS into the temporary set S after removing the overlapping result. Such as [a ₁₁ , b ₁₁ ]=[2,4], [a ₀ , b ₀ ]=[1,9], then the overlapping result is [2,4], and the weighted counter associated with [2,4] adds 1.

S610: Updating the weighted line segment set S _OLSS . Delete overlapping line segments in the weighted line segment set S _OLSS in S608 , and merge the temporary set S with the weighted line segment set S _OLSS to obtain a new weighted line segment set S _OLSS . According to the number M of line segments in the new weighted line segment set, arrange the line segments [a ₀ , b ₀ ], [a ₁ , b ₁ ], ..., [a _M , b _{M ]} in ascending order of starting point ]. As in the example of S608, the processing result is S _OLSS ={[1, 2], [2, 4], [4, 9]}, where the weighted counter value of [2, 4] is 1, [1, 2] and The value of the weighted counter for [4, 9] is 0.

S612: Check whether it is the last line segment in the line segment group. If not, go to step S606; if yes, go to the next step.

S614: Check whether it is the last line segment group. If not, go to step S604; if yes, go to the next step.

S616: Select the largest line segment of the weighted counter from the result S _OLSS of the above processing to form a result set.

The image point outline includes closely adjacent points, and the set of rays from the viewpoint to points on the point outline is also closely adjacent. The method of intersecting weighted line segments can effectively calculate the result set obtained by the intersection of a ray and other cones. The set of result sets calculated for all rays of the point profile should be regularly arranged closely, and by weighted line segments The intersection method to calculate the approximate intersection of the line segment set in a ray will be affected by noise and produce deviations, so it is necessary to correct it.

S4: Correct the initial line segment set model to obtain the result line segment set model. The correction method is a centerline filtering method, and its object is the set of processed result sets of all bus line segment sets under a viewpoint.

If {S _i } _i=1 ^N (N is the number of line segments, S _i is the i-th line segment) is a set of line segments, and S _i =[a _i , b _i ], where a _i ≤ _bi , this Central linear filter {S′ _i } _i=1 ^N of set of line segments:

l _i ＝b _i -a _i ; indicates the length of S _i

$l_i^{\′}={\mathrm{\Σ}}_{j=-N}^N{h}_j*l_{i+j},$ (in ${\mathrm{\Σ}}_{j=-N}^N{h}_j=1,$ N ∈ {3, 5, 7}); adjusted length.

a' _i = ( _bi + a _i )/2-l' _i /2; re-determined starting point.

b' _i = ( _bi + a _i )/2+l' _i /2; re-determined end point.

If used directly $a_i^{\′}={\mathrm{\Σ}}_{j=-N}^N{h}_j*a_{i+j},$ and $b_i^{\′}={\mathrm{\Σ}}_{j=-N}^N{h}_j*b_{i+j}$ If filtering is performed, the overall drift of the smoothed line segment set will appear. The line segment set of the visible shell model obtained by the weighted line segment intersection algorithm is the maximum approximation result of the visible shell of the object. For the abnormal line segment information, the irregular line segment set can be regularized by the central linear filtering method. Make the overall trend of the line segment set obvious, which is convenient for the next step of processing.

S5: Using the resulting line segment set model to reconstruct the surface of the object. If a light cone plane is visible to multiple viewpoints, the light cone plane will intersect the viewpoint cones under multiple visible viewpoints, and obviously only their intersection will fall inside the final polyhedral visible shell. The polygon obtained by intersecting the light cone plane with all other photo cones is the visible shell information of the object. In this embodiment, the above principles are adopted for surface reconstruction, and the specific steps include:

S502: Classify the line segments in the initial line segment set model according to the normal vectors, that is, the line segments with similar normal vectors are classified into one category, and a threshold can be set in the application as a criterion for judging the similar normal vectors.

S504: For all line segments in the same class, calculate the plane group generated by the set of line segments. All line segments with the same normal vector are coplanar to form a plane in this plane group, and the planes are distinguished according to the intercept of the plane.

S506: Search for each plane adjacent to it, and intersect the adjacent planes to obtain a boundary.

S508: Adjacent boundaries intersect to obtain vertices, and the vertices are connected in sequence to obtain polygons for surface reconstruction.

S510: Processing the next set of similar line segments until all the line segments are processed, then the visible shell of the object can be obtained.

The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it should not be understood as limiting the patent scope of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (7)

1, a kind of method that generates the visual shell of robust is characterized in that, comprises the steps:

Obtain the image collection of object synchronization under at least two different points of view;

Every width of cloth image in the described image collection is carried out parallel processing, obtain profile information and preservation;

Utilize the line segment weighting to ask the friendship method to make up the initial segment aggregation model of object according to described profile information;

Utilize the filtration of line segment aggregate central linear that the initial segment aggregation model is revised and obtain line segment aggregate model as a result;

Utilize the line segment aggregate polygon to detect object is carried out resurfacing, obtain the visual shell of object.

2, the method for the visual shell of generation robust as claimed in claim 1 is characterized in that, it is on the horizontal circumference at center that described viewpoint is uniformly distributed in the object, adopts digital camera shot object image under each viewpoint.

3, the method for the visual shell of generation robust as claimed in claim 1 is characterized in that, adopts the profile information of the optimum ladder edge detection algorithm acquisition image of canny edge detection operator.

4, the method for the visual shell of generation robust as claimed in claim 1 is characterized in that, the method that makes up the initial segment aggregation model comprises the steps:

To the profile of the object under each viewpoint, with its discrete set for point, the point in the described set all is positioned on the profile of object, constitutes dot profile, and each viewpoint and respective point profile constitute a viewpoint cone;

A bus of viewpoint cone and other viewpoint cones intersect generation busbar line segment collection, utilize the line segment weighting to ask the friendship method busbar line segment collection to be handled the result set that obtains under bus of this viewpoint cone;

All identical with the quantity of corresponding dot profile mid point in each viewpoint cone buses and other viewpoint cone are the initial segment aggregation model of object through the set of the result set after the processing identical with above-mentioned steps after crossing.

5, the method for the visual shell of generation robust as claimed in claim 4 is characterized in that, described line segment weighting asks the friendship method to comprise the steps:

All line segments that busbar line segment is concentrated are represented with starting point and terminal point with one dimension parameter;

It is starting point that definition is concentrated minimum starting point with busbar line segment, first line segment that maximum terminal point is a terminal point belongs to the weighting line-segment sets, the busbar line segment collection comprises a plurality of sets of line segments, and each sets of line segments is by bus and crossing the obtaining of viewpoint cone, and the quantity of sets of line segments is identical with the quantity of other viewpoint cones;

Set an interim collection for empty;

Check whether each bar line segment and all line segments in the weighting line-segment sets in the sets of line segments are overlapping, if overlapping, overlapping result is inserted interim collection, and weighted count device that will be related with this overlapping result add one;

According to described interim collection the weighting line-segment sets is upgraded, and emptied interim collection;

Handle each sets of line segments one by one, concentrate all line segments all processed intact up to busbar line segment;

The line segment of choosing weighted count device maximum from final weighting line-segment sets constitutes result set.

6, the method for the visual shell of generation robust as claimed in claim 5 is characterized in that, the step of upgrading the weighting line-segment sets comprises:

If it is overlapping that line segment in the weighting line-segment sets and the interim line segment of concentrating have, then the lap of this line segment is removed and formed new line segment, otherwise reservation;

The line segment aggregate of above-mentioned processing is gathered also with interim.

7, the method for the visual shell of generation robust as claimed in claim 1 is characterized in that, resurfacing comprises the steps:

Line segment in the initial segment aggregation model is classified according to normal vector;

Situation according to the line segment coplane in the same class line segment obtains one group of plane, and each plane is distinguished according to the difference of intercept;

The crossing border that obtains of adjacent planar;

Adjacent boundary intersects to the limit, is configured for the polygon of resurfacing by the summit;

All line segments in the re-treatment initial segment aggregation model.

Images