WO2022097279A1

WO2022097279A1 - Merge region detection method, merge region detection device, and program

Info

Publication number: WO2022097279A1
Application number: PCT/JP2020/041566
Authority: WO
Inventors: 真由子渡邊; 隆一谷田; 英明木全
Original assignee: 日本電信電話株式会社
Priority date: 2020-11-06
Filing date: 2020-11-06
Publication date: 2022-05-12
Also published as: JP7453586B2; JPWO2022097279A1

Abstract

This merge region detection method has an acquisition step and a boundary determination step. The acquisition step involves acquiring tree structure data in which first point cloud data, which includes information about the positions in a prescribed space of each point in part of the space, is represented by information about the presence of points in each region obtained by dividing the prescribed space into regions of a tree structure formed from a plurality of layers. The boundary determination step involves determining, in units of regions and on the basis of the uppermost layer in which a point is present in the layers of the tree structure data, a merge boundary that is a boundary used when merging the first point cloud data in the prescribed space and second point cloud data that includes information about each point in the space for at least part of the prescribed space.

Description

Merge area detection method, merge area detection device and program

The present invention relates to a merge area detection method, a merge area detection device, and a program.

It is predicted that the chances of acquiring point cloud data in space with devices such as fixed LiDAR (Light Detection and Ringing), in-vehicle LiDAR, drone-mounted LiDAR, and MMS (Mobile Mapping System) will increase in the future. With such an increase in opportunities, the amount and number of point cloud data to be stored will also increase. Point cloud data is often encoded when the data is stored. Therefore, for example, G-PCC (Geometry based Point Cloud Compression) is used for encoding the point cloud data (see, for example, Non-Patent Document 1).

Point cloud data is a set of points acquired for a certain subspace in a continuous space in reality. Therefore, the point cloud data has boundaries in space. When using point cloud data, the data is not always handled only by the acquired unit. For example, there is a case where a plurality of continuous point cloud data are merged as a space. When performing this merge, it is necessary to decode the encoded point cloud data once. Here, it is assumed that there is no coordinate deviation between the data to be merged.

As an automatic merging method of point cloud data, a method of adding all the point clouds shown by the two point cloud data can be considered without worrying about the coordinates of the point cloud in the space. However, in general, point cloud data is not acquired by neatly dividing it into adjacent subspaces. That is, both point clouds may exist in the same region. In the following, the area where the point cloud of such two point cloud data exists is described as "the area where the point clouds overlap".

The two point cloud data to be merged may be acquired at different dates and times. Furthermore, the status of the structures in the region where the point clouds overlap may change while the point cloud data is acquired. Examples are when a new building is built, when there is no car parked last time, and so on. When all the point clouds are added together even though the condition of the structure has changed in this way, it is considered difficult to correctly grasp the state of the space.

Of the two point cloud data to be merged, which is the priority point cloud data (hereinafter referred to as "updated data") and which is not the priority point cloud data (hereinafter referred to as "old data"). In some cases, it has been decided. In that case, it is considered that the point cloud data that correctly represents the state of the space can be created by merging the area where the point clouds overlap, leaving only the updated data without using the old data.

On the other hand, the point cloud data is basically measured using the reflected wave from the laser at a certain position. Therefore, the range of the acquired point cloud is different. In order to divide these areas neatly, it is generally necessary to use manual or detailed point cloud analysis methods. Since these are performed while checking the structure represented by the point cloud, there is a possibility that the merge can be performed accurately. However, it is expected that human cost and time cost will increase because the processing is time-consuming.

In view of the above circumstances, it is an object of the present invention to provide a merge area detection method, a merge area detection device, and a program that can determine an area when merging point cloud data having overlaps in a target space. ..

In one aspect of the present invention, the first point group data including information on the position of each point in a part of a predetermined space is divided into the predetermined space into a region of a tree structure composed of a plurality of layers. The first point group in the predetermined space based on the acquisition step of acquiring the tree structure data represented by the information on the presence or absence of points in each area and the highest hierarchy in which points exist in the tree structure data hierarchy. With the boundary determination step that determines the merge boundary, which is the boundary when merging the data and the second point group data including the information of each point in at least a part of the space in the predetermined space, in the unit of the region. It is a merge area detection method having.

In one aspect of the present invention, the first point group data including information on the position of each point in a part of a predetermined space is divided into the predetermined space into a region of a tree structure composed of a plurality of layers. The first point group in the predetermined space based on the acquisition unit that acquires the tree structure data represented by the information on the presence or absence of points in each area and the highest hierarchy in which points exist in the tree structure data hierarchy. With a boundary determination unit that determines the merge boundary, which is the boundary when merging the data and the second point group data including the information of each point in at least a part of the space in the predetermined space, in the unit of the region. , A merged area detector.

One aspect of the present invention is a program for causing a computer to execute the above-mentioned merge area detection method.

According to the present invention, it is possible to determine an area for merging point cloud data having overlaps in the target space.

It is a block diagram which shows the structure of the merge area detection apparatus by 1st Embodiment. It is a figure which shows the parent block and the child block of the space area division used in the same embodiment. It is a figure which shows the division of the spatial area containing the point cloud data used in the same embodiment. It is a figure which shows the example of the octath tree used in the same embodiment. It is a flow chart which shows the operation of the merge area detection apparatus in the same embodiment. It is a flow chart which shows the operation of the merge area detection apparatus in the same embodiment. It is a figure for demonstrating the operation of the merge area detection apparatus in the same embodiment. It is a figure for demonstrating the operation of the merge area detection apparatus in the same embodiment. It is a figure for demonstrating the operation of the merge area detection apparatus in the same embodiment. It is a figure for demonstrating the operation of the merge area detection apparatus in the same embodiment. It is a flow chart which shows the operation of the merge area detection apparatus in the same embodiment. It is a figure for demonstrating the operation of the merge area detection apparatus in the same embodiment. It is a block diagram which shows the structure of the merge area detection apparatus by 2nd Embodiment. It is a figure for demonstrating the conversion from the octath tree data to the quadtree data in the same embodiment. It is a figure for demonstrating the generation of the tree structure data using the coordinates of the point projected on the two-dimensional plane in the same embodiment. It is a flow chart which shows the operation of the merge area detection apparatus in the same embodiment. It is a figure which shows the hardware configuration of the merge area detection apparatus in 1st and 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, an area for merging the point cloud data of the old data and the point cloud data of the updated data is defined without using manual or detailed analysis method. This area is an area for replacing the old data with the updated data, or an area for matching the old data with the updated data. In the present embodiment, it is possible to determine the area to be merged at high speed by using the tree structure data in which the point cloud data is represented by a tree structure such as an ocree.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration of the merge area detection device 1 according to the first embodiment. The merge area detection device 1 includes a storage unit 11, an input unit 12, a division unit 13, a conversion unit 14, a replacement unit 15, a boundary determination unit 16, and a merge unit 17.

The storage unit 11 stores the point cloud data of the old data and the point cloud data of the updated data. The point cloud data includes data of a set of coordinate values in which a point exists in a spatial region. The coordinate values of the points included in the old data and the coordinate values of the points included in the updated data are values in the same coordinate space. The spatial area of the old data and the spatial area of the updated data partially overlap.

The input unit 12 inputs data. The input may be the reception of data from another device or the reading of data from another device or recording medium. Further, the input may include data input by an existing input device such as a keyboard or a touch panel. The input unit 12 inputs old data and updated data, and writes these input data in the storage unit 11.

The division unit 13 divides the spatial area of the old data and the spatial area of the updated data into division areas, respectively. The division area is a unit for converting point cloud data into tree structure data. The tree structure data is data in which the divided space including the points in the spatial region obtained by obtaining the set of points indicated by the point cloud data is expressed by an octree structure. A common division area is used for the division of the old data and the division of the update data.

The conversion unit 14 converts the point cloud indicated by the old data into tree structure data for each divided area. The conversion unit 14 adds header information indicating the size and position of the divided region to the converted tree structure data. Similarly, the conversion unit 14 converts the point cloud indicated by the update data into tree structure data for each division area, and adds header information indicating the size and position of the division area to the converted tree structure data. Thereby, the old data and the updated data can be represented by the tree structure data for each common divided area regardless of the spatial area of the old data and the spatial area of the updated data.

The replacement unit 15 replaces the area of the old data corresponding to the real space represented by the update data with the update data with respect to the old data. Alternatively, the replacement unit 15 replaces the divided area in which the updated data does not exist with the divided area of the old data with respect to the updated data.

The boundary determination unit 16 determines the merge boundary for the divided area including the space where the point clouds of the old data and the updated data overlap. A merge boundary is a boundary that replaces old data with updated data or a boundary that adds old data to updated data. The merging unit 17 merges the point cloud data of the old data and the point cloud data of the updated data in the divided region according to the merging boundary determined by the boundary determining unit 16.

The tree structure data of the point cloud data will be described with reference to FIGS. 2 to 4. In the present embodiment, the coordinate value of each point included in the point cloud data is represented by the value of each component in the xyz coordinate, that is, the x coordinate value, the y coordinate value, and the z coordinate value.

FIG. 2 is a diagram showing a parent block and a child block used for spatial region division when generating tree structure data. By dividing the parent block B, which is the space of the cube, into two equal parts in each of the three directions (x-axis, y-axis, z-axis) orthogonal to each other, the child blocks B-0 to eight cubes from the parent block B B-7 is generated.

FIG. 3 is a diagram showing division of a spatial area for generating tree structure data. Block B0 is a cube with 2 ⁿ sides on each side. The block B0 corresponds to a division area used when the division unit 13 divides the spatial area of the old data and the spatial area of the updated data. To simplify the calculation process, the coordinate values of the point cloud data are converted into the coordinate values of the relative coordinate system. Specifically, the coordinates of the point cloud data are translated so that the minimum value of each component of x, y, and z becomes 0. As a result, the coordinates of one vertex of block B0 become (0,0,0). Each side of x = 2 ⁿ , y = 2 ⁿ , and z = 2 ⁿ of block B0 does not include a point. n takes a positive integer. n may be determined based on the size of the point cloud to be encoded.

When the spatial area division shown in FIG. 2 is performed with the block B0 as the parent block, eight child blocks B1-0 to B1-7 are generated. Next, when the spatial area division shown in FIG. 2 is performed using the block B1-i (i is an integer of 0 or more and 7 or less) including one or more points among the blocks B1-0 to B1-7 as the parent block. Blocks B2-i-0 to B2-i-7, which are eight child blocks, are generated. Blocks B1-i that do not contain points are not divided.

Of the blocks B2-i-0 to B2-i-7, the spatial region division shown in FIG. 2 is performed by using the block B2-i-j (j is an integer of 0 or more and 7 or less) containing one or more points as a parent block. Will be done. As a result, blocks B3-i-j-0 to B3-i-j-7, which are eight child blocks, are generated. Further, the process of dividing the space area with the child block containing one or more points as the parent block is repeated a predetermined time. The child block generated by the m-th division (m is an integer of 1 or more) is described as the m-th layer block.

FIG. 4 is a diagram showing an example of an ocree. The point cloud data is represented by an octree corresponding to the divided three-dimensional space as shown in FIG. The uppermost node N0 corresponds to the block B0. Node N0 is connected to eight nodes N1-0 to N1-7 in the first layer. Node N1-i corresponds to block B1-i. A node corresponding to a block containing a point is described as a node containing a point, and a node corresponding to a block not including a point is described as a node not including a point.

The node N1-i including the point is connected to the eight nodes N2-i-0 to N2-i-7 on the second layer. Node N2-i-j corresponds to block B2-i-j. Since the block B1-i not including the point is not divided, the node N1-i corresponding to the block B1-i is not connected to the node of the second layer. In FIG. 4, the nodes N1-0 to N1-6 not including the points are not connected to the nodes in the second layer. Further, the node N1-7 including the point is connected to the node N2-7-0 to N2-7-7 in the second layer. Nodes N2-7-0 and N2-7-2 to 7 that do not include points are not connected to the node in the third layer. The node N2-7-2 including the point is connected to the eight nodes N3-7-1-0 to N3-7-1-7 in the third layer.

From the point cloud data, a tree-structured node corresponding to the spatial area divided as described above is generated. For the node corresponding to the block containing the point, a block value indicating whether or not each of the child blocks having the block as the parent block contains the point is given. That is, the node corresponding to the block containing the point is given a block value indicating whether or not each of the nodes one level below the node contains the point. This block value is expressed by the equation (1). x _k is a code indicating whether or not a point is included in the kth child block (k is an integer of 0 or more and 7 or less) among the eight child blocks. "1" means that the point is included, and "0" means that the point is not included.

For example, if only blocks B1-2 and B1-7 of the child blocks of block B0 contain points, the block value of the node corresponding to block B0 is f (0,0,1,0,0,0,0). , 1) = 33. Further, when all eight child blocks of a certain block include points, the block value is expressed as f (1,1,1,1,1,1,1,1) = 255. In this way, each node is given a block value in which the position of the point is represented by a value from 0 to 255 by the equation (1).

FIG. 5 is a flow chart showing the processing of the merge area detection device 1. FIG. 5 shows a process when the update data is merged with the old data. The input unit 12 of the merge area detection device 1 inputs data A and data B (step S110). The storage unit 11 stores the input data A and data B. Data A and data B are point cloud data. The data A and the data B may be data having an ocree-structured structure for each divided region. The division unit 13 sets the data A as the old data and the data B as the update data according to the instruction input by the input unit 12.

When the data A and the data B are not the data having an ocree-structured structure for each division area, the division unit 13 divides the old data and the update data into the division areas, respectively (step S120). The conversion unit 14 converts the data A into tree-structured data having an octa-tree structure for each divided region. Similarly, the conversion unit 14 converts the data B into tree-structured data having an octa-tree structure for each divided region. The conversion unit 14 adds header information indicating the size and position of the divided area to the tree structure data of each divided area. The position of the divided region is represented by, for example, the coordinate values in the coordinate system before conversion to the relative coordinate system.

The replacement unit 15 replaces the divided area of the old data corresponding to the real space represented by the updated data with the divided area of the updated data (step S130). The boundary determination unit 16 sets the divided area in which the old data exists and partially or completely overlaps with the updated data as the processing target divided area. The boundary determination unit 16 determines the merge boundary for each processing target division area. The merge unit 17 merges the update data with the old data according to the merge boundary determined by the boundary determination unit 16 (step S140).

FIG. 6 is a flow chart showing the merge boundary determination process of the merge area detection device 1. FIG. 6 shows the details of the process of step S140 of FIG. The boundary determination unit 16 performs the processing of FIG. 6 for each of the processing target division regions.

First, the boundary determination unit 16 determines whether or not the update data exists in all the division areas adjacent to the processing target division area based on the tree structure data of each division area of the update data (step S210). The boundary determination unit 16 determines whether or not the update data exists in all the division areas adjacent to the division area to be processed, instead of the tree structure data, the size of the division area added to the tree structure data of the division area. The judgment may be made based on the header information indicating the position or the like. When the boundary determination unit 16 determines that the update data exists in all the division areas adjacent to the processing target division area (step S210: YES), the boundary determination unit 16 determines that all the processing target division areas are replaced with the update data. Notify the replacement unit 15 of the information of the processing target division area. The replacement unit 15 replaces all the processing target division areas of the update data with the update data of the processing target division area (step S220).

When the boundary determination unit 16 determines that there is a division area without update data in the division area adjacent to the processing target division area (step S210: NO), the boundary determination unit 16 performs the processing of step S230. The boundary determination unit 16 acquires information on the hierarchy set by the user (step S230). The hierarchy set by the user is the hierarchy to be the target of the process of determining the update target area. This hierarchy corresponds to the hierarchy of the tree structure shown in FIG. The boundary determination unit 16 may read the information of the hierarchy from the storage unit 11, or may acquire the information of the hierarchy input by the input unit 12. It is assumed that the hierarchy set by the user is from hierarchy N1 (N1 is an integer of 1 or more) to hierarchy N2 (N2 is an integer of N1 or more).

The boundary determination unit 16 sets the layer n to be processed as the layer N1 (step S240). The boundary determination unit 16 identifies the boundary between the processing target division area and the adjacent division area where there is no update data, based on the header information given to the tree structure data of each division area of the update data. The boundary determination unit 16 sets the specified boundary as the merge boundary of the initial value (step S250).

The boundary determination unit 16 refers to the tree structure data of the processing target division area indicated by the update data, and determines whether or not each block Bn of the layer n includes the update data area. When the block Bn is set to have a point in the tree structure data of the update data, the boundary determination unit 16 determines that the block Bn includes the area of the update data. On the other hand, when the block Bn is set not to have a point in the tree structure data of the update data, the boundary determination unit 16 determines that the block Bn does not include the area of the update data. It should be noted that it is not possible to distinguish between the area where the point cloud is not measured and the area where the point cloud is measured but the point is not detected in the area where there is no update data. The boundary determination unit 16 assigns "1" to the block Bn including the area of the update data, and assigns "0" to the block Bn not including the area of the update data.

The boundary determination unit 16 detects the first block Bn to which 1 is assigned to each of the columns of the block Bn in the processing target division area starting from the initial value of the merge boundary. For each column, the boundary determination unit 16 moves the merge boundary of the column to the boundary between the detected block Bn and the block Bn adjacent to the merge boundary side of the same column as the detected block Bn (step S260). ). When the merge boundary becomes discontinuous between adjacent columns due to this movement, the boundary determination unit 16 connects the discontinuous merge boundaries along the boundary of the block Bn.

The boundary determination unit 16 determines that there is the next layer when the layer n of the current processing target does not reach the lower limit N2 of the layer (step S270: YES). The boundary determination unit 16 updates the layer n of the processing target to the layer (n + 1) immediately below the layer n of the current processing target (step S280). The boundary determination unit 16 repeats the process from step S260.

As described above, the boundary determination unit 16 repeats the processes of steps S260 to S280. Then, when the boundary determination unit 16 determines that the layer n to be processed has reached the lower limit N2 of the layer (step S270: NO), the boundary determination unit 16 notifies the merge unit 17 of the processing target division area and the merge boundary. The merging unit 17 replaces the blocks in the processing target divided area of the old data that are closer to the adjacent divided area in which the updated data exists than the determined merge boundary with the updated data of the block (step S290). .. The replacement may be performed by a point cloud or by replacing the target location in the tree structure data.

The process of FIG. 6 will be described with reference to FIGS. 7 to 10. In FIGS. 7 to 10, for the sake of simplicity, the divided area and each block are represented by a plane.

FIG. 7 is a diagram showing a processing target divided area and an adjacent divided area. The processing target division area D1 and the division areas D2 to D5 adjacent to the processing target division area D1 correspond to the block B0 in FIG. 3, respectively. FIG. 7A shows a case where update data exists in all the division areas adjacent to the processing target division area D1. In this case, the boundary determination unit 16 determines that the entire processing target division area D1 of the old data is replaced with the processing target division area D1 of the update data (step S210: YES, step S220).

FIG. 7B shows a case where there is a division area without update data in the division area adjacent to the processing target division area D1. In FIG. 7B, there is update data in the adjacent divided regions D4 and D5. However, there is no update data in the adjacent divided regions D2 and D3. In this case, the boundary determination unit 16 performs the processing after step S230 with respect to the processing target division area D1.

8 to 10 are diagrams for explaining the determination of the merge boundary in the processing target division area D1. The case where the layer N1 acquired in step S230 of FIG. 6 is 1 and the layer N2 is 3 will be described. The boundary determination unit 16 sets the layer n to be processed to N1 (= 1) (step S240 in FIG. 6).

FIG. 8 shows the processing when the layer n is 1. The area E represents an area where update data is obtained. In the processing target divided area D1, the area not included in the update data area E is an area in which the old data is present but the update data is not obtained.

Of the division areas adjacent to the processing target division area D1, there is no update data in the division area D2 and the division area D3. Therefore, the boundary determination unit 16 sets the boundary between the processing target division area D1 and the division areas D2 and D3 as the initial value merge boundary R (step S250 in FIG. 6).

Next, the boundary determination unit 16 refers to the tree structure data of the update data, and determines whether or not each of the blocks B1-i of the layer 1 in the processing target division area D1 includes the point of the update data. Note that FIG. 8 shows the four blocks B1-0 to B1-3 of the layer 1.

The boundary determination unit 16 assigns 1 to the block B1-i containing the points of the update data and 0 to the blocks B1-i not including the points of the update data. The boundary determination unit 16 detects the first block B1-i to which 1 is assigned to each column of each block B1-i starting with the initial value of the merge boundary R. That is, the boundary determination unit 16 detects the block closest to the current (initial value) merge boundary R among the blocks B1-i having a value of 1 in each column.

The boundary determination unit 16 detects the block B1-3 close to the current merge boundary R because the values of both the blocks B1-3 and B1-2 constituting the vertical column L1 are 1. Since the detected block B1-3 includes the merge boundary R, the boundary determination unit 16 does not move the merge boundary R for the column L1 (step S260 in FIG. 6).

The boundary determination unit 16 detects block B1-1 close to the current merge boundary R because the values of both blocks B1-1 and B1-0 constituting the vertical column L2 are 1. Since the detected block B1-1 includes the merge boundary R, the boundary determination unit 16 does not move the merge boundary R for the column L2 (step S260 in FIG. 6).

The boundary determination unit 16 detects the block B1-3 close to the current merge boundary R because the values of both the blocks B1-3 and B1-1 constituting the horizontal column L3 are 1. Since the detected block B1-3 includes the merge boundary R, the boundary determination unit 16 does not move the merge boundary R for the column L3 (step S260 in FIG. 6).

The boundary determination unit 16 detects block B1-2 close to the current merge boundary R because the values of blocks B1-2 and B1-0 constituting the horizontal column L4 are both 1. Since the detected block B1-2 includes the merge boundary R, the boundary determination unit 16 does not move the merge boundary R for the column L4 (step S260 in FIG. 6).

In the boundary determination unit 16, since the layer n to be processed has not reached the lower limit 2 of the layer (step S270: YES), the layer n to be processed is set to the layer 2 one level higher than the current 1 (FIG. 6). Step S280).

FIG. 9 shows the processing when the layer n is 2. The boundary determination unit 16 refers to the tree structure data of the update data, and determines whether or not each of the blocks B2-i-j of the layer 2 in the processing target division area D1 includes the point of the update data. The boundary determination unit 16 may limit the determination target to the child blocks of the blocks B1-1, B1-2, and B1-3 including the boundary in the layer 1 which is one layer above.

The boundary determination unit 16 assigns 1 to the block B2-i-j containing the points of the update data and 0 to the blocks B2-i-j not including the points of the update data. The boundary determination unit 16 detects the first block B2-i-j to which 1 is assigned to each of the columns of the blocks B2-i-j starting with the initial value of the merge boundary R. That is, the boundary determination unit 16 detects the block closest to the current merge boundary R among the blocks B2-i-j having a value of 1 in each column.

The boundary determination unit 16 is the block B2-2-3 having a value of 1 among the blocks B2-3-3, B2-3-2, B2-2-3, and B2-2-2 constituting the vertical column L11. Is detected. The boundary determination unit 16 moves the merge boundary R to the boundary between the block B2-2-3 and the block B2-3-2 adjacent to the merge boundary R side of the block B2-2-3 in the column L11 ( Step S260 in FIG. 6).

Similarly, the boundary determination unit 16 detects the block B2-3-0 closest to the current merge boundary R among the blocks having a value of 1 in the vertical column L12. The boundary determination unit 16 moves the merge boundary R to the boundary between the detected block B2-3-0 and the block B2-3-1 adjacent to the current merge boundary R side of the block B2-3-0 (). Step S260 in FIG. 6).

Similarly, the boundary determination unit 16 detects the block B2-1-3 closest to the current merge boundary R among the blocks having a value of 1 in the vertical column L13 headed by the block B2-1-3. Further, the boundary determination unit 16 detects the block B2-1-1 closest to the current merge boundary R among the blocks having a value of 1 in the vertical column L14 in which the block B2-1-1 is the head. Since the detected blocks B2-1-3 and B2-1-1 include the current merge boundary R, the boundary determination unit 16 does not move the merge boundary R for the columns L13 and L14 (step S260 in FIG. 6). ).

The boundary determination unit 16 detects the block B2-1-3 closest to the current merge boundary R among the blocks having a value of 1 in the column L15 next to the block B2-3-3 at the head. The boundary determination unit 16 moves the merge boundary R to the boundary between the detected block B2-1-3 and the block B2-3-1 adjacent to the current merge boundary R side of the block B2-1-3 (). Step S260 in FIG. 6). The boundary determination unit 16 performs the same processing as described above for the horizontal rows L16 to L18.

By the above processing, the merge boundary R of the horizontal column L17 remains at the initial value, but the merge boundary R of the horizontal column L18 is between the block B2-2-2 and the block B2-2-0. .. Since these merge boundaries R are discontinuous, the boundary determination unit 16 connects the merge boundary R of column L17 and the merge boundary R of column L18.

In the boundary determination unit 16, since the layer n to be processed has not reached the lower limit 3 of the layer (step S270: YES), the layer n to be processed is set to the layer 3 one level higher than the present (step S280 in FIG. 6). ).

FIG. 10 shows the processing when the layer n is 3. The boundary determination unit 16 refers to the tree structure data of the update data and determines whether or not each of the blocks of the layer 3 in the processing target division area D1 includes a point of the update data. The boundary determination unit 16 determines that the target of the determination is the blocks B2-1-4, B2-2-1, B2-2-3, B2-2-4, B2 including the boundary in the layer 2 which is one layer above. It may be limited to child blocks of 3-1, B2-3-2, and B2-3-3.

The boundary determination unit 16 assigns 1 to the block of the third layer that includes the point of the update data, and assigns 0 to the block that does not include the point of the update data. The boundary determination unit 16 performs the same processing as in FIG. 9 using the value assigned to the block in the third layer, and moves the merge area R (step S260 in FIG. 6). The boundary determination unit 16 determines that the layer n to be processed has reached the lower limit 3 of the layer (step S270: NO in FIG. 6).

The merge unit 17 replaces the block on the update data side of the determined merge boundary among the blocks in the processing target division area of the old data with that block of the update data (step S290 in FIG. 6).

The boundary determination unit 16 may perform the process shown in FIG. 11 instead of the process shown in FIG. FIG. 11 is a flow chart showing a merge boundary determination process of the merge area detection device 1. In FIG. 11, the same processing as in FIG. 6 is designated by the same reference numerals, and the description thereof will be omitted. The process shown in FIG. 11 differs from the process shown in FIG. 6 in that the process of step S310 is performed before the process of step S290. In step S310, the boundary determination unit 16 moves the merge boundary by one block in the direction of the area where the update data is located.

FIG. 12 is a diagram showing an example in which the boundary determination unit 16 processes the merge boundary R determined as shown in FIG. 10 in step S310 of FIG. As shown in FIG. 12, the boundary determination unit 16 moves the merge boundary R shown in FIG. 10 in the direction of the update data E by one block.

The process shown in FIG. 6 (described as process A) and the process shown in FIG. 11 (described as process B) are the difference between making the merge boundary outside or slightly inside the edge of the update data. In the case of process A, it is possible to merge with all the updated data left. On the other hand, the update data may exist only in a part of the block adjacent to the inside of the boundary, and the point cloud related to this block may be sparse. This is because, after merging in step S290, the part outside the region E in the shaded block in FIG. 10 does not include a point. In the process B, the old data is used for the merge boundary, and only the area inside the block where the update data is surely present is filled with the update data. This may prevent the boundaries from becoming sparse.

In the above, the merge area detection device 1 extracts the target area to be updated by the update data from the old data. Then, when the merge area detection device 1 merges the old data and the update data, the merge area detection device 1 detects an area of the old data area that does not include the update data, and leaves the old data in the detected area to update the data. Add together. However, the merge area detection device 1 may extract an area that is not updated from the old data. In that case, in step S120, the replacement unit 15 specifies the divided area in which the updated data does not exist among the divided areas of the old data. The replacement unit 15 adds the specified divided area to the update data, and replaces the added divided area with the old data of the divided area. Then, in step S220, the replacement unit 15 leaves the processing target division area as the update data. In step S290, the merging unit 17 replaces the block on the old data side of the determined merge boundary among the blocks in the processing target divided area of the update data with the block at the same position of the old data.

(Second embodiment)
In the first embodiment, the area for merging the old data and the updated data is determined three-dimensionally. In the second embodiment, points are projected in the z-axis (axis perpendicular to the ground) direction, and the area to be merged is determined two-dimensionally. As a result, the processing of the merge area detection device can be reduced in weight and simplified. The second embodiment will be described with a focus on the differences from the first embodiment.

FIG. 13 is a block diagram showing the configuration of the merge area detection device 2 according to the second embodiment. In FIG. 13, the same parts as those of the merge area detection device 1 according to the first embodiment shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted. The difference between the merge area detection device 2 and the merge area detection device 1 shown in FIG. 1 is that the merge area detection device 2 includes a conversion unit 24 and a boundary determination unit 26 in place of the conversion unit 14 and the boundary determination unit 16.

The conversion unit 24 performs the same processing as the conversion unit 14. Further, the conversion unit 24 projects the divided area and each point of the update data included in the divided area onto a two-dimensional plane. The conversion unit 24 generates tree structure data in which the point cloud data projected on the divided region is represented by the quadtree data for each of the two-dimensional divided regions. The boundary determination unit 26 determines a merge boundary using the update data converted into a two-dimensional plane for the divided region where the old data and the update data overlap.

An example of calculating the tree structure data of the points projected on the two-dimensional plane will be described with reference to FIGS. 14 and 15. These figures show projections in the z-axis direction.

FIG. 14 is a diagram for explaining conversion from octatree data to quadtree data. Nodes N-0 to N-7 correspond to blocks B-0 to B-7, which are child blocks of block B shown in FIG. 2, respectively. In FIG. 14, there are points only in blocks B-2 and B-3. Therefore, the node N-2 is connected to the nodes B-2-0 to B-2-7 corresponding to each of the blocks B-2-0 to B-2-7, which are child blocks of the node N-2. .. Similarly, the node N-3 is connected to the nodes B-3-0 to B-3-7 corresponding to each of the blocks B-3-0 to B-3-7, which are child blocks of the node N-3. The node. Each node is given a "1" if there is a point, and a "0" if there is no point.

The conversion unit 24 calculates the logical sum of the node values corresponding to each of the overlapping blocks in the z-axis direction. As shown in FIG. 2, block B-0 and block B-1, block B-2 and block B-3, block B-4 and block B-5, block B-6 and block B-7, respectively, z. It overlaps in the axial direction. The conversion unit 24 uses the block value of the plane that projects the block B-0 and the block B-1 overlapping in the z-axis direction as the logical sum of the value of the node N-1 and the value of the node N-2. .. Similarly, the conversion unit 24 sets the block value of the plane that projects the blocks B-2 and the blocks B-3 that overlap in the z-axis direction as the logic of the value of the node N-2 and the value of the node N-3. It is a sum. At this time, the conversion unit 24 calculates the logical sum of the value of the node N-2-j corresponding to the child block at the same position and the value of the node N-3-j. The conversion unit 24 performs a process of calculating the block value of the plane that projects the blocks overlapping in the z-axis direction with respect to the obtained logical sum.

FIG. 15 is a diagram for explaining an example of generating tree structure data using the coordinates of points projected on a two-dimensional plane. As shown in FIG. 15A, the conversion unit 24 removes the z-axis coordinate value from the three-dimensional coordinate value of each point included in the point cloud data and converts it into the coordinates of the xy plane. FIG. 15B is a diagram showing a divided region in which a three-dimensional divided region overlapping in the z-axis direction is projected onto the xy plane. The conversion unit 24 generates four child blocks by dividing the divided region into two equal parts in each of the two directions (x-axis and y-axis) orthogonal to each other. The conversion unit 24 uses the generated child block as a parent block, and repeats the process of generating four child blocks from the parent block. The transforming unit 24 uses the coordinates of the projected points to generate a tree-structured node corresponding to each block of the layered plane. The conversion unit 24 assigns a block value indicating whether or not each of the child blocks having the block as a parent block contains a point to the node corresponding to the block containing the point, and creates the quadtree data. ..

The merge area detection device 2 performs the process shown in FIG. In step S140 of FIG. 5, the merge area detection device 2 performs the process shown in FIG. 16 instead of the process shown in FIG.

FIG. 16 is a flow chart showing the merge boundary determination process of the merge area detection device 2. In FIG. 16, the same processing as in FIG. 6 is designated by the same reference numerals, and the description thereof will be omitted. The process shown in FIG. 16 is different from the process shown in FIG. 6 in that the merge area detection device 2 performs the process of step S410 before the process of step S230. In step S410, the conversion unit 24 projects the point cloud data of the update data in the z-axis direction to form a point cloud on a two-dimensional plane. The conversion unit 24 generates quadtree data of point cloud data on a plane that projects the division area to be processed. Then, the boundary determination unit 26 of the merge area detection device 2 performs the processing of step S250, step S260, and step S280 using the generated quadtree data.

In step S290, the merging unit 17 replaces the block of the divided area to be processed of the old data, which is closer to the adjacent divided area where the updated data exists than the determined merge boundary, with the updated data of the block. The boundary determination unit 26 determines the merge boundary in the plane. Therefore, the merge unit 17 replaces using the merge boundary determined by the boundary determination unit 26 regardless of the position of the z-axis of the processing target division region.

The merge area detection device 2 may perform the processes of steps S210 to S270 of FIG. 16 once for a plurality of processing coping areas having the same projection on the xy plane. Further, the merge area detection device 2 may perform the process of step S410 after the process of step S230. Further, the merge area detection device 2 may perform the process of step S310 in FIG. 11 before the process of step S290.

By determining the area two-dimensionally, in addition to weight reduction and simplification of processing, there are the following merits. That is, it is assumed that when updating the 3D data on the map, the situation where the building was built on the vacant lot is acquired when there is a shield such as a car between the building and Lidar. .. In this case, the existence information of the building regarding the space where the ground of the vacant lot existed cannot be obtained, and there is a possibility that the ground of the vacant lot is below and the building is only above. However, when a point is projected in the z-axis direction, the lower ground can be deleted only by the information that the upper side of the building exists, and the state can be updated only with new data.

If you just want to replace the old data with the updated data, you can handle it by merging a part of the hierarchical old data tree structure into the updated data tree structure. However, when the range to be replaced with the updated data is unknown, a technique for determining the area to be replaced is required. According to the above-described embodiment, the area to be replaced can be determined.

Further, in the above-described embodiment, it is premised that the inside of the area of the old data and the updated data is all continuous data, that is, the data of the convex shape area. This is based on the assumption that since the point cloud data is acquired for a continuous space, the data in a distant region is generally not acquired. For example, it is assumed that the point cloud data is acquired in a donut shape. In that case, it can be dealt with by the division into the division area in step S120.

Note that step S130 in FIG. 5 is a process when there is no boundary between the old data and the updated data. In addition, the explanation so far has been made when there is duplication of the point cloud between the old data and the updated data. A configuration for determining whether the area included in the update data includes an area that does not overlap with the old data is further added, and the merge

area detection devices

1 and 2 do not look at the area of the update data that does not overlap with the old data in detail. May be processed. For example, if the

boundary determination units

16 and 26 can confirm that all the old data in the area (on the update data side) inside the determined merge boundary is 0, the

boundary determination units

16 and 26 terminate the determination process of the lower hierarchy. May be good.

In the above, the point cloud data is divided into division areas of a common size, and each division area is converted into octal tree data and merged. On the other hand, in PCC (Point Cloud Compression), which is being formulated in MPEG (Moving Picture Experts Group) standardization, especially in G-PCC for space and stationary objects, point cloud data is converted to octal tree representation and then arithmetic code. Be made. Therefore, when one or both of the old data and the updated data is a G-PCC stream which is compressed data, the ocree processing is not necessary when the merging processing is performed by a common block. The G-PCC stream may be arithmetically decoded and merged using the ocree data. That is, in the merge process using the once compressed G-PCC stream instead of the point cloud data itself, the process can be performed without conversion to an octal tree. In this way, the old data and the updated data of the G-PCC stream can be merged without completely decoding the coordinate data. Further, when MPEG is used, it is possible to use the information in the G-PCC stream as the information corresponding to the header information for each block (coding unit).

Up to this point, the embodiment of merging without decoding using the data of the octaree structure has been explained. From here, a method that enables merging without performing inverse conversion will be described even if stream data obtained by converting data having an octa-tree structure into binarized values or the like is used. As described above, by using the header information, it is possible to specify the area of the old data corresponding to the area of the real space represented by the updated data. If the area can be specified, the specified area can be merged without inverse conversion by replacing the old data with the payload of the stream of update data.

In the above, the parent block of the cube is divided into eight child blocks of the cube, but the parent block and the child block do not have to be cubes. For example, when the parent block is divided into n child blocks, the spatial area containing the points is represented by the n-branch structure instead of the tree-structured data represented by the octa-tree structure as described above. Tree structure data is used.

The functions of the merge

area detection devices

1 and 2 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client in that case. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, and may be further realized by combining the above-mentioned functions with a program already recorded in the computer system.

A hardware configuration example of the merge

area detection devices

1 and 2 will be described. FIG. 11 is a device configuration diagram showing a hardware configuration example of the merge

area detection devices

1 and 2. The merge area detection device 1 includes a processor 71, a storage unit 72, a communication interface 73, and a user interface 74.

The processor 71 is a central processing unit that performs calculations and controls. The processor 71 is, for example, a CPU. The processor 71 reads a program from the storage unit 72 and executes it. Some of the functions of the merge

area detection devices

1 and 2 may be realized by using hardware such as ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). The storage unit 72 further has a work area for the processor 71 to execute various programs and the like. The communication interface 73 is connected so as to be able to communicate with another device. The user interface 74 is an input device such as a keyboard, a pointing device (mouse, tablet, etc.), a button, a touch panel, and a display device such as a display. An artificial operation is input by the user interface 74. For example, the user interface 74 inputs information on the upper limit layer and information on the lower limit layer.

Each of the merge

area detection devices

1 and 2 may be realized by a plurality of computer devices connected to the network. In this case, it may be arbitrary which of these plurality of computer devices realizes each of the functional units of the merge

area detection devices

1 and 2. Further, the same functional unit may be realized by a plurality of computer devices.

According to the above-described embodiment, the merge area detection device has an acquisition unit and a boundary determination unit. The acquisition unit is, for example, the conversion unit 14 of the embodiment. The acquisition unit divides the first point cloud data including the position information of each point in a part of the space in the predetermined space into the area of the tree structure consisting of a plurality of layers, and the point in each area. Acquire the tree structure data represented by the presence / absence information. The predetermined space corresponds to the divided area of the embodiment, and the area corresponds to the block. The demarcation unit determines the first point cloud data in a predetermined space and the information of each point in at least a part of the space in the predetermined space, based on the highest hierarchy in which the points exist in the hierarchy of the tree structure data. The merge boundary, which is the boundary when merging with the second point cloud data including, is determined in the unit of the area.

The boundary determination unit performs initial value setting processing in which the boundary between the adjacent space where the point of the first point cloud data does not exist and the predetermined space among the adjacent spaces which are adjacent to the predetermined space is used as the merge boundary of the initial values. , Judgment target is the process of determining whether or not a point in the first point cloud data exists in the area of the hierarchy to be determined, and moving the merge boundary to the boundary where the area where the point does not exist changes to the area where the point exists. It may be repeated while changing the hierarchy of the above from the upper hierarchy to the lower hierarchy.

The merge area detection device may further include a merge unit. The merge unit merges the points of the first point cloud data and the points of the second point cloud data based on the merge boundary determined by the boundary determination unit.

The merge area detection device may further include a division portion. The division unit divides the space represented by the first point cloud data and the space represented by the second point cloud data into the division area of the tree structure data generation unit. The merge unit uses the point information of the first point cloud data in the divided region where the points of the first point cloud data exist and the second point cloud data does not exist. Further, the merging unit also uses the point information of the first point cloud data in the divided area in which the area of the first point cloud data is included in all the adjacent divided areas. Further, in the divided area in which the area of the first point cloud data is not included in any of the adjacent divided areas, the merge unit is first based on the merge boundary determined by the boundary determination unit with the divided area as a predetermined space. Merge the points in the point cloud data with the points in the second point cloud data. The merging unit is, for example, the replacement unit 15 and the merging unit 17 of the embodiment.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to these embodiments, and includes designs and the like within a range that does not deviate from the gist of the present invention.

1 ... merge area detector, 2 ... merge area detector, 11 ... storage unit, 12 ... input unit, 13 ... division unit, 14 ... conversion unit, 15 ... replacement unit, 16 ... boundary determination unit, 17 ... merge unit, 24 ... Conversion unit, 26 ... Boundary determination unit, 71 ... Processor, 72 ... Storage unit, 73 ... Communication interface, 74 ... User interface

Claims

Whether or not there is a point in each of the first point cloud data including information on the position of each point in a part of the space in the predetermined space, which is obtained by dividing the predetermined space into a tree structure region consisting of a plurality of layers. The acquisition step to acquire the tree structure data represented by information, and
Based on the highest hierarchy in which points exist in the hierarchy of the tree structure data, the first point cloud data in the predetermined space and information on each point in at least a part of the space in the predetermined space are included. A boundary determination step that determines the merge boundary, which is the boundary when merging with the second point cloud data, in the unit of the area.
Merge area detection method with.
The boundary determination step is
An initial value setting step in which the boundary between the adjacent space, which is a space adjacent to the predetermined space, and the boundary between the adjacent space where the point of the first point cloud data does not exist and the predetermined space is the merge boundary of the initial values, and the initial value setting step.
A process of determining whether or not a point of the first point cloud data exists in the area of the hierarchy to be determined, and moving the merge boundary to a boundary changing from the area where the point does not exist to the area where the point exists. Is included in the iterative processing step of repeating while changing the layer to be determined from the upper layer to the lower layer.
The merge area detection method according to claim 1.
Further having a merge step of merging the points of the first point cloud data and the points of the second point cloud data based on the merge boundary determined in the boundary determination step.
The merge area detection method according to claim 1 or 2.
The division area of the generation unit of the tree structure data further has a division step for dividing the space represented by the first point cloud data and the space represented by the second point cloud data.
In the merge step,
The divided region in which the points of the first point cloud data exist and the second point cloud data does not exist, and the divided region in which the region of the first point cloud data is included in all the adjacent divided regions. Uses the point information of the first point cloud data.
In the divided area in which the area of the first point cloud data is not included in any of the adjacent divided areas, the merge boundary is determined and determined by the boundary determination step with the divided area as the predetermined space. The point of the first point cloud data and the point of the second point cloud data are merged based on the merge boundary.
The merge area detection method according to claim 3.
Whether or not there is a point in each of the first point cloud data including information on the position of each point in a part of the space in the predetermined space, which is obtained by dividing the predetermined space into a tree structure region consisting of a plurality of layers. An acquisition unit that acquires tree structure data represented by information,
Based on the highest hierarchy in which points exist in the hierarchy of the tree structure data, the first point cloud data in the predetermined space and information on each point in at least a part of the space in the predetermined space are included. A boundary determination unit that determines the merge boundary, which is the boundary when merging the second point cloud data, in the unit of the area.
A merge area detector that comprises.
On the computer
A program for executing the merge area detection method according to any one of claims 1 to 4.