JP2021039500A - Method of generating three-dimensional shape data of construct - Google Patents

Abstract

To generate three-dimensional shape data of a construct constituted of a plurality plane regions most of which are regularly arranged, in a shorter time compared to conventional techniques.SOLUTION: Three dominant axes are specified from point group data representing a three-dimensional shape of a construct, which is generated by SfM, and the point group data is transformed to a coordinate system with the three specified axes, and meshes of which normal vectors are along an xy plane, among meshes shown by the point group data subjected to the coordinate transform are projected on the xy plane as boundary meshes. The xy plane is divided into pixels, and pixels including the boundary meshes are defined as boundary pixels, and segments fitting the boundary pixels are specified as edges, and adjacent edges are connected. With respect to each region of regions into which pixels other than the boundary pixels are clustered, a plane region where an average value of z coordinate values of pixels within the region is taken as a z coordinate value is specified. In addition, a plane region generated by sweeping each edge of the edges between z coordinate values of two plane regions adjacent to the edge is specified.SELECTED DRAWING: Figure 8

Description

The present invention relates to a technique for generating three-dimensional shape data of a structure.

From the 3D scan image data representing the 3D shape of the structure generated by the depth sensor such as a 3D laser scanner, it is possible to generate the 3D mesh model data representing the 3D shape of the structure by collecting a large number of meshes. It has been. The 3D mesh model data is easy to handle because it represents the 3D shape of an object with a smaller amount of data than the 3D scanned image data.

As a method of generating 3D mesh model data from 3D scanned image data, for example, there are the following methods.

First, local quadratic polymorphic curve fitting is iteratively performed on the three-dimensional scanned image data to calculate the principal curvature of the mesh vertices, and initial segmentation is performed based on the calculated principal curvature.

In the initial segmentation, the principal curvature of each mesh vertex is analyzed to generate a seed region consisting of a small set of connected vertices presumed to be on the curved surface instead of the original boundary between the curved surfaces. Then, using region glowing that uses both a quadric polynomial surface and a quadric surface, the surface fitting to the area and the addition of neighboring vertices to the seed area are performed iteratively, and the partial area on the mesh vertex and the analysis that approximates it are performed. Extract the curved surface.

Subsequently, region merging is used to integrate the initial regions generated by the initial segmentation so that the number of regions is as small as possible within the threshold specified by the user.

For example, Patent Document 1 is a document that discloses a technique for generating 3D mesh model data from 3D scanned image data.

JP-A-2007-241996

According to the method described above, it is possible to generate three-dimensional mesh model data representing the three-dimensional shape of the structure regardless of the shape of the structure. However, in the case of the above-mentioned method, it is necessary to search all the quadratic polynomial curved surfaces, which requires a large amount of calculation time.

By the way, some structures have a shape that approximates a set of plane regions that are orthogonal to any of the three axes that are orthogonal to each other. For example, most of the shapes of the superstructure of a bridge viewed from below are in a coordinate system in which the vertical direction is the z-axis, the axial direction of the bridge is the x-axis, and the width direction of the bridge is the y-axis. In space, it is composed of a plurality of plane regions substantially parallel to any one of the xy plane, the xz plane, and the yz plane.

In view of the above circumstances, the present invention is a means for generating three-dimensional shape data representing a three-dimensional shape of a structure composed of a plurality of regularly arranged plane regions in a short time as compared with the prior art. I will provide a.

In the present invention, the coordinate values of the points are set for each of a plurality of points on the surface of the structure which are arranged in the space of the coordinate system having the x-axis, the y-axis and the z-axis orthogonal to each other and have irregularities in the z-axis direction. A step of acquiring point group data which is a collection of point data to be shown, a step of extracting a point in a plane region parallel to the z-axis among a plurality of points indicated by the point group data as a boundary point, and the step of extracting the point in the xy plane. A step of specifying a plurality of regions surrounded by boundary points, and a plurality of point data included in the point group data in which the x-coordinate value and the y-coordinate value are within the region for each of the plurality of regions. A step of specifying a plane region parallel to the xy plane whose z coordinate value is a representative value of the z coordinate value of the point data, and two plane regions adjacent to each other among the plurality of specified plane regions parallel to the xy plane. The three-dimensional shape of the structure is obtained by providing data representing a step of specifying a plane region parallel to the z-axis of a rectangle having two outer edges and a plane region parallel to the xy plane and a plane region parallel to the z-axis. As a first aspect, a method for generating three-dimensional shape data of a structure including a step of generating as three-dimensional shape data representing the above is provided.

According to the method for generating the three-dimensional shape data of the structure according to the first aspect, the three-dimensional shape data representing the three-dimensional shape of the structure composed of a plurality of regularly arranged plane regions is used as the conventional technique. It can be generated in a short time in comparison.

Further, the present invention is a method for generating three-dimensional shape data of a structure according to the first aspect, when the coordinate system is a second coordinate system and the point cloud data is a second coordinate system point cloud data. The step of acquiring the point cloud data is a collection of point data indicating the coordinate values of the points with respect to each of the plurality of points on the surface of the structure in the first coordinate system different from the second coordinate system. A step of acquiring the 1-coordinate system point cloud data, a step of specifying the dominant three axes of the three-dimensional shape represented by the first coordinate system point cloud data as the three axes of the second coordinate system, and the first coordinate. As the second aspect, we propose a configuration including a step of converting the coordinate value of the point cloud indicated by the system point cloud data into the coordinate value of the second coordinate system to generate the second coordinate system point cloud data. ..

According to the method of generating the three-dimensional shape data of the structure according to the second aspect, the three-dimensional shape of the structure is represented by the coordinate values of the coordinate system of the three axes different from the dominant three axes of the three-dimensional shape of the structure. Using the point cloud data, it is possible to generate three-dimensional shape data representing the three-dimensional shape of the structure in a plurality of plane regions.

Further, the present invention is a method for generating three-dimensional shape data of a structure according to a second aspect, and the generation step comprises a plane region parallel to the xy plane and a plane region parallel to the z-axis. The step of converting the coordinate value of the second coordinate system to be represented by the coordinate value of the first coordinate system, and the converted coordinate value of the first coordinate system make the plane region parallel to the xy plane parallel to the z-axis. As a third aspect, we propose a configuration having a step of generating three-dimensional shape data representing a plane region.

According to the method for generating the three-dimensional shape data of the structure according to the third aspect, the three-dimensional shape data representing the three-dimensional shape of the structure can be generated by the coordinate values of the coordinate system of the point cloud data to be used.

Further, the present invention is a method for generating three-dimensional shape data of a structure according to any one of the first to third aspects, and a step of specifying the plurality of regions is a line passing through a region including the boundary point. A configuration is proposed as a fourth aspect, which includes a step of specifying a straight line or a curve having a predetermined characteristic, and a step of specifying a plurality of regions having the specified straight line or a curve of a predetermined characteristic as an outer edge. To do.

According to the method for generating the three-dimensional shape data of the structure according to the fourth aspect, the shape of the boundary line of the plane region constituting the three-dimensional shape of the structure is composed of a straight line or a curve having a predetermined characteristic such as a circle or an ellipse. If so, it is possible to generate three-dimensional shape data representing the three-dimensional shape of the structure with high accuracy.

The present invention also relates to each of a plurality of points on the surface of a structure that is arranged on a computer in a space of a coordinate system having x-axis, y-axis, and z-axis orthogonal to each other and has irregularities in the z-axis direction. A process of acquiring point group data, which is a collection of point data indicating the coordinate values of points, and a process of extracting a point in a plane region parallel to the z-axis from a plurality of points indicated by the point group data as a boundary point. , The process of specifying a plurality of regions surrounded by the boundary points on the xy plane, and for each of the plurality of regions, the x-coordinate value and the y-coordinate value of the plurality of point data included in the point group data are the regions. The process of specifying a plane region parallel to the xy plane whose z coordinate value is the representative value of the z coordinate value of the plurality of point data in the data, and the plane region parallel to the specified xy plane are adjacent to each other. The process of specifying the plane region parallel to the z-axis of the rectangle having the outer edges of the two plane regions as two sides, and the data representing the plane region parallel to the xy plane and the plane region parallel to the z-axis are obtained. As a fifth aspect, a program for executing a process of generating as three-dimensional shape data representing a three-dimensional shape of a structure is provided.

According to the program according to the fifth aspect, three-dimensional shape data representing the three-dimensional shape of a structure composed of a plurality of plane regions, most of which are regularly arranged, is obtained by a computer in a short time as compared with the prior art. Can be generated.

According to the present invention, it is possible to generate three-dimensional shape data representing a three-dimensional shape of a structure composed of a plurality of plane regions, most of which are regularly arranged, in a short time as compared with the prior art.

The figure which showed the processing flow of the method which concerns on one Embodiment. The figure which showed the processing flow of the method which concerns on one Embodiment. The figure which illustrated the image represented by the 2nd coordinate system point cloud data generated by the method which concerns on one Embodiment. The figure which illustrated the image which represents the boundary pixel specified by the method which concerns on one Embodiment. The figure which illustrated the image which shows the edge specified in the method which concerns on one Embodiment. The figure for demonstrating the process which connects two edges in the method which concerns on one Embodiment. The figure for demonstrating the process which connects two edges in the method which concerns on one Embodiment. The figure which illustrated the image which represents the plane area parallel to the xy plane specified by the method which concerns on one Embodiment. The figure which illustrated the 3D shape represented by the point cloud data and 3D simplified model data generated by the method which concerns on 1 Embodiment.

[Embodiment]
Hereinafter, a method for generating three-dimensional shape data of a structure according to an embodiment of the present invention (hereinafter, referred to as “method M”) will be described. In the method M, the three-dimensional shape data representing the three-dimensional shape of the structure by the point cloud (hereinafter referred to as “point cloud data”) is used to represent the three-dimensional shape of the structure by the data group indicating the vertices, edges, loops, and regions. This is a method of generating original shape data (hereinafter referred to as "three-dimensional simplified model data").

However, a structure capable of generating effective three-dimensional simplified model data by the method M is a structure in which there are three dominant axes orthogonal to each other. A structure having three dominant axes orthogonal to each other means a structure in which most of the surface of the structure is composed of a plurality of plane regions having any one of the three axes orthogonal to each other as a normal. To do.

1A and 1B (hereinafter, these are referred to as "FIG. 1") are diagrams showing the processing flow of the method M. The processing shown in FIG. 1 is performed by the computer according to the program according to the present embodiment. Hereinafter, the processing flow of the method M will be described by taking as an example the case where the target for generating the three-dimensional simplified model data is the structure S and the superstructure of the bridge seen from below is the structure S.

First, image data representing a two-dimensional image obtained by photographing the structure S from a plurality of viewpoints is acquired (step S01). Subsequently, the point cloud data of the structure is generated from the plurality of image data acquired in step S01 by SfM (Structure from Motion) (step S02). In the present embodiment, in step S02, point cloud data representing a mesh group having a normal vector is generated. Each vertex and normal vector of the mesh represented by the point cloud data generated in step S02 is represented using coordinate values in a certain coordinate system (hereinafter, referred to as "first coordinate system"). Hereinafter, the point cloud data generated in step S02 will be referred to as "first coordinate system point cloud data".

Subsequently, the dominant three axes of the three-dimensional shape represented by the first coordinate system point cloud data are specified by the Gaussian mapping of the vertex normal vector and the RANSAC method (step S03).

Hereinafter, the coordinate system in which the three axes specified in step S03 are the x-axis, the y-axis, and the z-axis is referred to as a "second coordinate system". The z-axis of the second coordinate system is the vertical axis, the x-axis is the axial direction of the structure S (traveling direction of the vehicle, etc.), and the y-axis is the width direction of the structure S (width direction of the vehicle, etc.). Is the axis of.

Subsequently, point cloud data (hereinafter referred to as "second coordinate system point cloud data") obtained by converting the coordinate values of the point cloud indicated by the first coordinate system point cloud data into the coordinate values of the second coordinate system is generated (step). S04) (an example of a step of acquiring the point cloud data described in the claim range). FIG. 2 is a diagram illustrating an image represented by the second coordinate system point cloud data.

Subsequently, from the plurality of meshes indicated by the second coordinate system point cloud data, a mesh in which the angle formed by the normal vector with the xy plane is within a predetermined threshold (substantially parallel) is extracted (step S05). ) (An example of the extraction steps described in the billing scope). The mesh extracted in step S05 is a mesh in a plane region parallel to the z-axis on the surface of the structure S. Hereinafter, the mesh extracted in step S05 is referred to as a “boundary mesh”, and the vertices of the boundary mesh are referred to as “boundary vertices” (an example of the boundary points described in the claims).

Subsequently, the vertices of the mesh indicated by the second coordinate system point cloud data are projected onto the xy plane (step S06). When projecting a point on the xy plane, it means changing the z-coordinate value of the point to a zero value.

Subsequently, the xy plane is divided into pixels, which are unit regions of a predetermined size and a predetermined shape (step S07). In this embodiment, as an example, one pixel is a square of 10 cm square.

Subsequently, the pixels including the boundary vertices are extracted (step S08). Hereinafter, the pixels extracted in step S08 are referred to as "boundary pixels". FIG. 3 is a diagram illustrating an image representing boundary pixels. In FIG. 3, the boundary pixels are shown in white.

Subsequently, segmentation is performed on the boundary pixels with a group of pixels linearly distributed by the region growth method as one block (step S09). Subsequently, a straight line (line segment) that fits the segmented pixel group is specified (step S10). The straight line specified in step S10 is a straight line that approximates a line passing through a segmented pixel group (a region including a boundary vertex). Hereinafter, the line segment specified in step S10 will be referred to as an "edge" (an example of the outer edge described in the claims). Both ends of the edge are the vertices in the 3D simplified model. FIG. 4 is a diagram illustrating an image representing an edge.

Subsequently, connection processing between discontinuous edges is performed (step S11). Specifically, the connection processing between edges is performed for the following two patterns.

(Pattern 1) As shown in FIG. 5A, when the distance between the end points of two adjacent edges is within a predetermined threshold value, each of the two edges is extended to identify the intersection, and the intersection is specified. Let the intersection be a new vertex common to the two edges (Fig. 5 (b)).

(Pattern 2) As shown in FIG. 6A, when the distance between one end point of two adjacent edges and the other edge is within a predetermined threshold value, one edge is extended to specify an intersection. Then, the intersection is used as a new vertex of the extended edge, and the other edge is divided into two edges at the intersection (FIG. 6 (b)).

Subsequently, a region in which non-boundary pixels are continuous is clustered (step S12). Subsequently, for each of the clustered regions, a plane region parallel to the xy plane having the average value of the z-coordinate values of the pixels in the region as the z-coordinate value of the pixel is specified (step S13) (described in the claim range). An example of a step of identifying a plane region parallel to the xy plane of. FIG. 7 is a diagram illustrating an image showing a plane region parallel to the xy plane.

Subsequently, from the plane regions specified in step S13, two plane regions adjacent to each edge are searched, and the adjacency relationship between the plane regions is specified (step S14). Further, for each of the plane regions specified in step S13, the edge group surrounding the plane region is specified as a loop (step S15).

Subsequently, the plane region generated by sweeping each edge in the z-axis direction between the z-coordinate values of the two plane regions adjacent to the edge is specified as a plane region parallel to the z-axis (step S16). (An example of a step of specifying a plane region parallel to the z-axis described in the claim range). The plane region specified in step S16 is a rectangular plane region having two edges as two adjacent plane regions. The process of step S16 creates new edges and loops.

By the above processing, a plane region parallel to the xy plane, a plane region parallel to the z-axis, a loop surrounding those plane regions, an edge constituting each loop, and each edge representing the shape of the structure S The vertices at both ends are identified.

Subsequently, the coordinate values indicating the plane region, loop, edge, and vertex specified as described above are converted into the coordinate values of the first coordinate system (step S17). Then, the data indicating the plane region, the loop, the edge, and the apex according to the coordinate values after the conversion in step S17 are converted into the three-dimensional simplified model data (the three-dimensional shape data described in the claim range) representing the three-dimensional shape of the structure S. Generate as (an example) (step S18) (an example of the generation step described in the scope of claim). FIG. 8 is a diagram illustrating the three-dimensional shape represented by the first coordinate system point cloud data and the three-dimensional shape represented by the three-dimensional simplified model data generated in step S18. FIG. 8 (a) is an example of the three-dimensional shape represented by the first coordinate system point cloud data, and FIG. 8 (b) is an example of the three-dimensional shape represented by the three-dimensional simplified model data, FIG. 8 (c). Is an example of a diagram in which these three-dimensional shapes are superimposed. The three-dimensional simplified model data generated by the method M has a significantly reduced number of elements compared to the first coordinate system point cloud data, but from FIG. 8, the three-dimensional shape represented by the first coordinate system point cloud data can be obtained. It can be seen that they are very similar.

The above is the description of the processing of the method M. According to the above-mentioned method M, three-dimensional simplified model data representing the structure S with a small amount of data is generated in a short time as compared with the case of the conventional technique.

[Modification example]
The above-described embodiment is a specific example of the present invention, and can be variously modified within the scope of the technical idea of the present invention. Examples of these modifications are shown below. In addition, two or more modified examples shown below may be combined as appropriate.

(1) The method for generating the first coordinate system point cloud data in steps S01 and S02 is not limited to SfM. For example, the first coordinate system point cloud data may be generated by a depth sensor such as a stereo camera or a laser scanner. In that case, the computer may acquire the first coordinate system point cloud data generated by the external device, and perform the processing after step S03 using the first coordinate system point cloud data.

(2) The format of the first coordinate system point cloud data is not limited to that representing a mesh group having a normal vector. For example, first coordinate system point cloud data representing a polygon model may be used.

(3) In step S03, the method for specifying the dominant three axes of the three-dimensional shape represented by the first coordinate system point cloud data is not limited to the method using the Gaussian mapping of the vertex normal vector and the RANSAC method.

(4) The size and shape of the pixel are not limited to those described above.

(5) In the above-described embodiment, the edge is a straight line, but the edge may be a curved line having a predetermined characteristic such as an arc or an elliptical arc. In this case, in step S10, a straight line or a curve having a predetermined characteristic that fits the segmented pixel group is specified.

(6) In the above-described embodiment, the average value of the z-coordinate values of the pixels in the clustered region in step S12 is used as the z-coordinate value of the plane region parallel to the xy plane specified in step S13. Here, instead of the average value, other statistical values such as an intermediate value and a mode value may be used.

(7) In the above-described embodiment, among the meshes indicated by the second coordinate system point cloud data, those whose normal vector is substantially parallel to the xy plane are extracted, and the vertices (boundary points) of those meshes are included. A line passing through the region is specified as a boundary line of a plane region parallel to the xy plane. The method of specifying the boundary line of the plane region parallel to the xy plane is not limited to this. For example, the points indicated by the second coordinate system point cloud data may be projected onto the xy plane, and the boundary line may be specified based on the distribution of the projected point cloud on the xy plane. That is, a line passing through a region where the density of the point cloud is equal to or higher than a predetermined threshold value on the xy plane may be specified as a boundary line.

Claims (5)

Point data indicating the coordinate values of a plurality of points on the surface of a structure having irregularities in the z-axis direction, which are arranged in the space of a coordinate system having x-axis, y-axis, and z-axis orthogonal to each other. Steps to acquire point cloud data that is a collection,
A step of extracting a point in a plane region parallel to the z-axis from a plurality of points indicated by the point cloud data as a boundary point, and
A step of identifying a plurality of regions surrounded by the boundary points in the xy plane, and
For each of the plurality of regions, the representative value of the z-coordinate value of the plurality of point data whose x-coordinate value and y-coordinate value are within the region among the plurality of point data included in the point cloud data is defined as the z-coordinate value. Steps to identify the plane area parallel to the xy plane to be
A step of specifying a plane region parallel to the z-axis of a rectangle having two outer edges of two plane regions adjacent to each other among a plurality of specified plane regions parallel to the xy plane.
Generation of three-dimensional shape data of a structure including a step of generating data representing a plane region parallel to the xy plane and a plane region parallel to the z-axis as three-dimensional shape data representing the three-dimensional shape of the structure. Method. When the coordinate system is the second coordinate system and the point cloud data is the second coordinate system point cloud data,
The step of acquiring the point cloud data is
A step of acquiring first coordinate system point cloud data which is a collection of point data indicating the coordinate values of the points with respect to each of a plurality of points on the surface of the structure in the first coordinate system different from the second coordinate system. ,
A step of specifying the dominant three axes of the three-dimensional shape represented by the first coordinate system point cloud data as the three axes of the second coordinate system, and
The structure according to claim 1, further comprising a step of converting the coordinate value of the point cloud indicated by the first coordinate system point cloud data into the coordinate value of the second coordinate system to generate the second coordinate system point cloud data. How to generate 3D shape data. The step to generate is
A step of converting the coordinate values of the second coordinate system representing the plane region parallel to the xy plane and the plane region parallel to the z-axis into the coordinate values of the first coordinate system.
The structure according to claim 2, further comprising a step of generating three-dimensional shape data representing a plane region parallel to the xy plane and a plane region parallel to the z-axis based on the converted coordinate values of the first coordinate system. How to generate 3D shape data. The step of identifying the plurality of regions is
A step of specifying a straight line that approximates a line passing through a region including the boundary point or a curve having a predetermined characteristic, and
The method for generating three-dimensional shape data of a structure according to any one of claims 1 to 3, further comprising a step of specifying a plurality of regions having the specified straight line or a curve having a predetermined characteristic as an outer edge. On the computer
Point data indicating the coordinate values of a plurality of points on the surface of a structure having irregularities in the z-axis direction, which are arranged in the space of a coordinate system having x-axis, y-axis, and z-axis orthogonal to each other. The process of acquiring point cloud data, which is a collection, and
A process of extracting a point in a plane region parallel to the z-axis from a plurality of points indicated by the point cloud data as a boundary point, and
The process of identifying a plurality of regions surrounded by the boundary points on the xy plane, and
For each of the plurality of regions, the representative value of the z-coordinate value of the plurality of point data whose x-coordinate value and y-coordinate value are within the region among the plurality of point data included in the point cloud data is defined as the z-coordinate value. The process of specifying the plane area parallel to the xy plane to be performed, and
A process of specifying a plane region parallel to the z-axis of a rectangle having two sides as outer edges of two plane regions adjacent to each other among a plurality of specified plane regions parallel to the xy plane.
A program for executing a process of generating data representing a plane region parallel to the xy plane and a plane region parallel to the z-axis as three-dimensional shape data representing the three-dimensional shape of the structure.