JP6693193B2 - Information processing device and information synthesis program - Google Patents

Description

  The present invention relates to an information processing device and an information composition program.

  There is known a technique related to registration of a plurality of three-dimensional (dimension) point group data acquired at different measurement points, so-called registration.

  For example, there is a method in which a target having a known shape is installed in the environment and the target is detected from each of the three-dimensional point groups to perform the alignment. In this way, when using the target for alignment, it takes time and effort to install the target, and it is up to the workers in the field to decide at which point the target should be installed. Be done.

  Therefore, there is a demand for a method of automatically performing alignment from the shape of the environment represented by the three-dimensional point cloud data without using the target.

  As an example of such a method, a plurality of characteristic points or features such as lines and surfaces that are easily distinguishable from other points are extracted from a plurality of three-dimensional point groups having different measurement points, and correct by an algorithm such as RANSAC (RANdom Sample Consensus). There is a method of aligning by finding correspondence. However, when it is applied to measurement in a wide space, a large number of points are dealt with, which increases the calculation cost.

  On the other hand, instead of processing the 3D point cloud as it is, by slicing each 3D point cloud in the horizontal direction at a constant height and matching the point cloud contained therein in 2D, Methods have been proposed to streamline the calculation.

  This method requires slicing a plurality of three-dimensional point clouds horizontally at the same height with respect to the environment. It suffices to perform all measurements at the same height, but if you want to measure the environment without blind spots, you may want to change the leg length of the measuring instrument or place a sensor on the desk or table to perform measurement. Come on. Therefore, there is a demand for a method of matching the height standards of the three-dimensional point cloud data in advance.

  As a method of adjusting the height reference of each three-dimensional point cloud data before slicing the three-dimensional point cloud, there is a method of detecting a observable floor surface at many measurement points and using the floor surface as a reference. For example, as the method of detecting the floor surface, there are the following two methods.

  As a first method, there is a method in which all planes formed by point groups whose normals are vertically upward are extracted and the plane having the lowest height is used as a reference plane. As a second method, there is a method in which all the planes formed by the point groups whose normals are vertically upward are extracted and the plane having the largest area is used as the reference plane.

JP 2012-112705 A JP, 10-96607, A

  However, in the above technique, the reference planes to be combined between the 3D point clouds cannot be associated with each other in an environment where occlusion occurs, and thus the accuracy of combining the 3D point cloud data may be reduced.

  In one aspect, an object of the present invention is to provide an information processing device and an information synthesizing program capable of increasing the synthesis accuracy of three-dimensional point cloud data under an environment with occlusion.

  In one aspect, the information processing device irradiates while scanning light at different measurement positions, and uses the reflected light reflected from the object to acquire point cloud information based on the distance from the device to the object. Between the different measurement positions and the extraction unit that extracts point cloud information belonging to a plane area from the point cloud information acquired by the acquisition unit, and between the different measurement positions, between the plane areas extracted by the extraction unit. And a position adjustment unit that performs position adjustment using the point group information corrected according to the offset amount calculated by the calculation unit.

  It is possible to improve the synthesis accuracy of three-dimensional point cloud data in an environment with occlusion.

FIG. 1 is a diagram illustrating a usage example of the portable information processing apparatus according to the first embodiment. FIG. 2A is a diagram showing an example of occlusion. FIG. 2B is a diagram showing an example of occlusion. FIG. 3 is a block diagram of the functional configuration of the portable information processing apparatus according to the first embodiment. FIG. 4 is a diagram showing an example of a method of measuring three-dimensional point cloud data. FIG. 5 is a diagram showing an example of a set of horizontal plane heights. FIG. 6 is a diagram showing a relationship between a set of heights of a horizontal plane and an offset amount. FIG. 7 is a diagram illustrating an example of a set of heights. FIG. 8 is a diagram showing an example of the evaluation value. FIG. 9 is a flowchart illustrating the procedure of the overall process according to the first embodiment. FIG. 10 is a flowchart of a registration process procedure according to the first embodiment. FIG. 11: is a figure which shows the application example regarding the extraction of a horizontal surface. FIG. 12 is a diagram illustrating a hardware configuration example of a computer that executes the information composition program according to the first and second embodiments.

  An information processing apparatus and an information composition program according to the present application will be described below with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Then, the respective embodiments can be appropriately combined within the range in which the processing contents do not contradict each other.

[Example of application scene]
FIG. 1 is a diagram illustrating an example of use of the portable information processing device 10 according to the first embodiment. FIG. 1 illustrates a site 2A to a site 2N as an example of a section in which work is performed. In addition, below, the site 2A to the site 2N may be collectively referred to as "site 2".

  The portable information processing device 10 shown in FIG. 1 is implemented as a portable device in which the worker 3 carries a hand. For example, when the worker 3 performs work on the site 2A to the site 2N, it does not matter if one information processing device 10 is installed for each site 2, and one information processing device 10 is held at each site 2. Can be used by turning. That is, the worker 3 brings the portable information processing device 10 to the next site 2 by hand-carrying or the like every time the work at the site 2 is completed, and places the portable information processing device 10 at an arbitrary position on the next site 2. Support data can be provided.

  In this way, the portable information processing apparatus 10 can measure the surrounding environment all over and digitize it with a simple operation by bringing it to the site 2A to the site 2N. In addition, various information support such as recording the behavior of people in the environment and providing information according to the state of people by measuring people in the environment and linking with digitized environment information Is possible.

  For example, the portable information processing apparatus 10 will be described in detail later with reference to FIG. 2, but a laser range finder capable of scanning a two-dimensional plane and measuring a distance to the environment is rotated by a motor, Realize measurement of three-dimensional point cloud data.

[Occlusion]
When this portable information processing device 10 is carried to the site 2 and measured, a wide range of measurements can be performed with one measurement, but a partition, a table, etc. placed in the site 2 act as a shield or a site 2 Due to the structure of the facility inside, measurement may be blocked by walls and the like from other than the viewpoint facing the entrance. As a result, the above-mentioned shield obstructs the generation of unmeasured locations (occlusion). In this case, the portable information processing device 10 is moved to another place so that the unmeasured place can be measured, and the measurement is performed again. By repeating this, it is possible to obtain a plurality of three-dimensional point cloud data in which the blind spots generated at the measurement points are compensated. On the other hand, since the positional relationship between the plurality of three-dimensional point groups is unknown, the plurality of three-dimensional point groups are aligned and so-called registration is performed, so that the three-dimensional point group data are combined.

  This will be specifically described. In the portable information processing device 10, the environment related to the site 2 is performed by the procedure of (1) 3D (dimension) measurement, (2) filtering, (3) registration, and (4) mesh modeling. Model data is generated.

  That is, the portable information processing device 10 measures a plurality of three-dimensional point cloud data by performing 3D measurement at different measurement points at the site 2 (1). Then, the portable information processing device 10 removes the point cloud that becomes noise by performing filtering for each three-dimensional point cloud data (2). For example, for each point included in the three-dimensional point cloud data, the variance is calculated from a point closest to the point and a predetermined number of points. At this time, when the points are elements that form an article or the like, nearby points are also densely gathered, so the variance is smaller than the isolated points that become noise, and if the isolated points are noise, there are nearby points. Since it is sparse, the dispersion is larger than that of the constituent elements of the article. From this, as an example, a point whose variance is equal to or larger than the threshold value is regarded as noise and removed.

  After that, the portable information processing device 10 integrates the three-dimensional point group data as one piece of three-dimensional point group data by performing the alignment processing of the plurality of three-dimensional point group data after noise removal (3). Then, the portable information processing device 10 converts the integrated three-dimensional point group data into three-dimensional surface data by meshing processing (4). This reduces the data capacity and uses it as a model of the environment.

  As described above, various registration methods for aligning the three-dimensional point groups measured at different measurement points have been devised up to now, but as described in the background art section, the registration method is When using a target for alignment of the dimensional point cloud, it takes time and effort to set the target, and it is left to the workers on site to decide which point to install the target, so it is easy to use. Be damaged.

  Therefore, there is a need for a method that automatically performs alignment from the shape of the environment represented by 3D point cloud data without using a target. If a plurality of features are extracted and corresponding points are obtained according to an algorithm such as RANSAC (RANdom Sample Consensus), the calculation cost increases.

  From this, it is possible to calculate by calculating each three-dimensional point cloud horizontally by slicing each three-dimensional point cloud at a constant height and matching the point cloud included in the two-dimensional point cloud instead of processing the three-dimensional point cloud as it is. It is considered effective to increase efficiency.

  In this case, the condition is that a plurality of three-dimensional point clouds can be sliced in the horizontal direction at the same height with respect to the environment. At this time, all the measurements should be performed at the same height, but if you want to measure the environment of the site 2 without blind spots, you may want to change the leg length of the portable information processing device 10, or you may need to measure the desk or table. There may be a case where the portable information processing device 10 is placed in a place where measurement is desired. From such a background, a method of matching the height standard of each three-dimensional point cloud data in advance is required.

  As a method of adjusting the height reference of each three-dimensional point cloud data before slicing the three-dimensional point cloud, there is a method of detecting a observable floor surface at many measurement points and using the floor surface as a reference. For example, a method 1 in which all planes formed by vertically upward point groups are extracted and the plane having the lowest height is used as a reference plane, and all planes formed by vertically upward point groups are used. There are two methods, Method 2 which is extracted and the surface having the largest area is used as the reference surface.

  However, no matter which of the above two methods is adopted, a situation in which point plane data obtained by the two measurement points to be aligned is selected as a reference plane due to the problem of occlusion is assumed. There are restrictions on the range.

[Failure example of Method 1]
FIG. 2A is a diagram showing an example of occlusion. FIG. 2A shows an example of a situation in which height reference matching fails due to occlusion in the above method 1. In FIG. 2A, the three-dimensional point cloud data is measured with the portable information processing device 10 placed at the measurement point M1, and the portable information processing device 10 is placed at the measurement point M2. The case where the measurement of the three-dimensional point cloud data is performed in the state is shown. Further, in FIG. 2A, the surface with the lowest height among the planes extracted from the three-dimensional point cloud data measured at the measurement point M1 is shown by a thick solid line, while the 3 measured at the measurement point M2. Among the planes extracted from the dimensional point cloud data, the surface having the lowest height is indicated by a thick broken line. In the site 2A shown in FIG. 2A, a stair landing exists at a position lower than the floor surface. In the landing of such stairs, the laser reaches from the measurement point M2, but the laser does not reach from the measurement point M1 because the stairs become a blind spot. By this occlusion, the floor surface is extracted as the lowest surface from the three-dimensional point cloud data measured at measurement point M1, while the stair landing is the lowest surface from the three-dimensional point cloud data measured at measurement point M2. Is extracted as. As a result, the reference plane is displaced between the measurement point M1 and the measurement point M2, and the height alignment fails.

[Example of Method 2 failure]
FIG. 2B is a diagram showing an example of occlusion. FIG. 2B shows an example of a situation in which height reference matching fails due to occlusion in the above method 2. In FIG. 2B, three-dimensional point cloud data is measured with the portable information processing device 10 placed at the measurement point M3, and the portable information processing device 10 is placed at the measurement point M4. The case where the measurement of the three-dimensional point cloud data is performed in the state is shown. Further, in FIG. 2B, the plane having the largest area among the planes extracted from the three-dimensional point cloud data measured at the measurement point M3 is shown by a thick solid line, while the three-dimensional measurement at the measurement point M4 is performed. Among the planes extracted from the point cloud data, the surface having the largest area is indicated by the thick broken line. At the site 2B shown in FIG. 2B, a large conference table is placed in the conference room. At the measurement point M3, the portable information processing device 10 is placed on the floor, while at the measurement point M4, the conference table is installed. The case where the portable information processing device 10 is placed on the desk is shown. In this case, the floor surface can be widely scanned from the measurement point M3, while the floor surface can be partially scanned by the measurement point M4 placed on the table of the conference table, hidden behind the conference table. By such occlusion, the floor surface is extracted as the surface having the largest area from the three-dimensional point cloud data measured at the measurement point M3, while the floor space is extracted from the three-dimensional point cloud data measured at the measurement point M4. The surface on the desk is extracted as the surface with the largest area. As a result, a deviation occurs in the reference plane between the measurement points M3 and M4, and the height alignment fails.

  As shown in FIGS. 2A and 2B, in the method of detecting a specific place in the environment, not limited to the floor surface, and using it as a reference, the measurement point hides a part or all of the place to be the reference for occlusion. Weak in the situation.

  Therefore, the portable information processing apparatus 10 according to the present embodiment does not perform the height reference matching on a specific surface such as a floor. That is, in the portable information processing device 10 according to the present embodiment, the relative positional relationship between the planes existing in the environment of the site 2 between the three-dimensional point groups measured at different measurement points, that is, the altitude between the planes. By calculating an offset amount with which the differences match and matching the height reference, and performing two-dimensional registration, the synthesis accuracy of the three-dimensional point cloud data is increased in an environment with occlusion.

[Configuration of Portable Information Processing Device 10]
FIG. 3 is a block diagram of the functional configuration of the portable information processing device 10 according to the first embodiment. As shown in FIG. 3, the portable information processing device 10 includes a display unit 11, a storage unit 13, a three-dimensional point cloud data acquisition unit 15, and a control unit 17. The portable information processing device 10 may have functional units other than the functional units shown in FIG. 3, such as an input device and a communication interface.

  The display unit 11 is a device that displays various types of information.

  As one embodiment, a liquid crystal display or an organic EL (electroluminescence) display that realizes display by light emission can be mounted on the display unit 11, or a projector or the like that realizes display by projection can be mounted. , Both of these can be implemented. The display unit 11 can display information according to an instruction from the control unit 17, but as an example, the three-dimensional point cloud data aligned by the alignment unit 17c described later, or the three-dimensional point cloud. It is possible to display three-dimensional polygon data or the like in which the data is converted by meshing processing.

  The storage unit 13 is a storage device that stores data used for various programs such as an OS (Operating System) executed by the control unit 17, an application program that realizes the above-described offset amount calculation and two-dimensional registration. Is.

  As an embodiment, the storage unit 13 can be implemented as an auxiliary storage device in the portable information processing device 10. For example, a HDD (Hard Disk Drive), an optical disk, an SSD (Solid State Drive), or the like can be used as the storage unit 13. It should be noted that the storage unit 13 does not necessarily have to be implemented as an auxiliary storage device, and may be implemented as a main storage device in the portable information processing device 10. In this case, various semiconductor memory devices such as a RAM (Random Access Memory) and a flash memory can be used as the storage unit 13.

  For example, the storage unit 13 stores three-dimensional point cloud data that has been aligned by the alignment unit 17c described later, or three-dimensional polygon data obtained by converting the three-dimensional point cloud data by meshing processing. Besides, the contents related to the work support data displayed on the display unit 11 can be stored.

  The three-dimensional point cloud data acquisition unit 15 is a processing unit that acquires three-dimensional point cloud data.

  As one embodiment, the three-dimensional point cloud data acquisition unit 15 can be implemented by an LRF (Laser Range Finder) or the like. FIG. 4 is a diagram showing an example of a method of measuring three-dimensional point cloud data. As shown in FIG. 4, the three-dimensional point cloud data acquisition unit 15 irradiates light around the X axis from the center of the cylinder toward the circumference, and performs scanning for receiving the reflected light, thereby obtaining a vertical cross section Vs. You can get the distance. Then, the three-dimensional point cloud data acquisition unit 15 drives a motor (not shown), rotates the LRF around the Z axis, and repeats the above-described scanning to set the distance around the entire circumference of the portable information processing device 10 to 3 It can be acquired as dimension point cloud data. As a matter of course, the three-dimensional point cloud data acquisition unit 15 can be used at one site 2 such as the measurement points M1 and M2 at the site 2A shown in FIG. 2A or the measurement points M3 and M4 shown at FIG. 2B. It is possible to acquire three-dimensional point cloud data for each of a plurality of different measurement points. The three-dimensional point cloud data acquisition unit 15 may be activated when an instruction operation such as a measurement button is received from the worker 3, and the portable information processing device 10 is placed. It does not matter even if it is detected.

  The control unit 17 has an internal memory that stores various programs and control data, and executes various processes by these.

  As one embodiment, the control unit 17 is implemented as a central processing unit, so-called CPU (Central Processing Unit). The control unit 17 does not necessarily have to be implemented as a central processing unit, and may be implemented as an MPU (Micro Processing Unit) or an MCU (Micro Controller Unit). The control unit 17 can also be realized by a hard-wired logic such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  The control unit 17 calculates the offset amount and performs two-dimensional registration on a work area of a RAM such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory) implemented as a main storage device (not shown). The following processing unit is virtually realized by developing the information composition program stored as an application program for realizing the above as a process.

  As shown in FIG. 3, the control unit 17 includes a plane extraction unit 17a, an offset amount calculation unit 17b, and a position alignment unit 17c.

  The plane extraction unit 17a is a processing unit that extracts the plane of the site 2 from the 3D point cloud data.

  As one embodiment, the plane extraction unit 17a extracts a plane formed by the three-dimensional point cloud included in the three-dimensional point cloud data acquired by the three-dimensional point cloud data acquisition unit 15 according to an algorithm such as RANSAC. For example, the plane extraction unit 17a uses the three-dimensional point cloud included in the three-dimensional point cloud data as a sample, and randomly extracts three points from the sample. Subsequently, the plane extraction unit 17a further extracts, from the three-dimensional point group included in the three-dimensional point group data, a point group within a predetermined distance from the plane model defined by the three points randomly extracted from the sample. .. Here, the point group within a predetermined distance from the plane model is regarded as the point group existing on the plane model, and the subsequent processing will be described. Then, the plane extraction unit 17a determines whether the number of point groups existing on the plane model is equal to or larger than a predetermined threshold value. At this time, when the point group on the plane model is greater than or equal to the threshold, the plane extracting unit 17a determines the parameters defining the plane model, such as the coordinates of three points or the equation of the plane, and the point group included in the plane model. The plane data associated with is stored in the work area on the internal memory. On the other hand, when the number of point groups existing on the plane model is less than the threshold value, the plane extraction unit 17a does not store the plane data regarding the plane model. After that, the plane extraction unit 17a repeatedly executes random sampling of three points from the sample and storage of the plane data accompanying the random sampling for a predetermined number of trials. With such a plane extraction method, it is possible to obtain a plane model in which a certain number or more of point groups are within a certain distance in the normal direction from the plane model. Below, the part where the three-dimensional point group exists at a predetermined density or more on the plane defined by the plane model may be referred to as a “plane”.

  Here, the plane extraction unit 17a can also extract a horizontal plane from the three-dimensional point cloud data for each of the three-dimensional point cloud data measured at a plurality of measurement points. In this case, the plane extraction unit 17a calculates the direction of the horizontal plane between the three-dimensional point group data of each measurement point. Then, the plane extraction unit 17a converts the coordinates of each point included in the three-dimensional point group data for each three-dimensional point group data of each measurement point such that the positive direction of the Z axis is the normal direction of the horizontal plane. To do. Then, as described above, the plane extraction unit 17a extracts a horizontal plane from the 3D point cloud data for each 3D point cloud data of each measurement point according to an algorithm such as RANSAC.

  Furthermore, the plane extracting unit 17a can group any number of two or more three-dimensional point cloud data as registration execution targets according to the following criteria. For example, each time the three-dimensional point cloud data acquisition unit 15 acquires the three-dimensional point cloud data, the plane extraction unit 17a groups the three-dimensional point cloud data with the previously acquired three-dimensional point cloud data. By doing so, registration can be performed. Furthermore, the plane extracting unit 17a can also perform grouping by narrowing down to three-dimensional point cloud data acquired within a predetermined period, for example, 10 minutes, among the three-dimensional point cloud data acquired in the past. The plane extraction unit 17a can also accept a combination of three-dimensional point cloud data to be grouped, via an input device (not shown).

  In this way, when the horizontal plane is extracted from the three-dimensional point cloud data for each three-dimensional point cloud data of each measurement point, the plane extraction unit 17a indicates the height (z coordinate) of each horizontal plane, for example, shown in FIG. The height with the origin of the position of the laser light source irradiated by the LRF is calculated.

  The offset amount calculation unit 17b is a processing unit that calculates the offset amount between different measurement points. The “offset amount” here refers to a difference in height between the measurement points, and the heights of the measurement points are adjusted by adjusting the difference.

[Definition of offset amount problem]
Here, the offset amount calculation unit 17b uses a method of calculating an offset amount that does not depend on detection of a specific place such as a floor surface. That is, many horizontal planes existing in the environment such as the site 2 have a floor, a ceiling, a table, and the like, and in many cases, two or more common horizontal planes can be detected unless the two measurement points are widely separated. Use the finding that it is likely. Further, since the measurement accuracy of the laser scanner is high, the distance between the common horizontal planes can be detected as substantially the same value even for the three-dimensional point group data at different measurement points.

  From this, the offset amount calculation unit 17b can obtain the offset amount by matching the correspondence of the faces with the relative positional relationship (distance) of all horizontal planes extracted at each measurement point as a feature.

FIG. 5 is a diagram showing an example of a set of horizontal plane heights. FIG. 5 shows a site 2B where the occlusion shown in FIG. 2B occurs. FIG. 5 shows a set of heights of the horizontal plane extracted from the three-dimensional point cloud data measured at the measurement point M3 and a height of the horizontal plane extracted from the three-dimensional point cloud data measured at the measurement point M4. And the set is shown. As shown in FIG. 5, when the horizontal plane is extracted from the three-dimensional point cloud data for each of the three-dimensional point cloud data of the measurement points M3 and M4, the offset amount calculation unit 17b determines the height (z coordinate) of each horizontal plane. For example, the height with the origin of the position of the laser light source irradiated by the LRF shown in FIG. 4 is calculated. In this case, H ₁ {h ₁₁ , h ₁₂ , h ₁₃ } is obtained as a set of heights of the horizontal plane extracted from the three-dimensional point cloud data measured at the measurement point M3, and is also measured at the measurement point M4. H ₂ {h ₂₁ , h ₂₂ , h ₂₃ , h ₂₄ } is obtained as a set of horizontal plane heights extracted from the three-dimensional point cloud data. In the above example, h ₁₂ and h ₁₃ of the set H ₁ have negative values, and h ₂₂ , h _23, and h _{24 of} the set H ₂ have negative values.

The two sets of data on the height values thus obtained have different measured heights between the data, and therefore the height values are different even if they represent the height on the same horizontal plane. The difference between the height standards is the offset amount to be obtained. At this time, the height values do not become the same between the data even on the same horizontal plane, but in the example shown in FIG. 5, the distance between the floor surface and the horizontal plane on the desk of the conference table, If the distances are calculated by the combination of the same horizontal planes such as the distance between the horizontal plane and the top surface and the distance between the floor surface and the top surface, the three-dimensional point cloud data will have substantially the same value even if there is some measurement error. It is likely to be obtained. Therefore, it is possible to search for a corresponding surface among the three-dimensional point cloud data by using the distance between the horizontal planes as a feature. For example, of the distances between the horizontal planes extracted from the three-dimensional point cloud data measured at the measurement point M4, the distances between the horizontal planes extracted from the three-dimensional point cloud data measured at the measurement point M3, that is, the floor surface and searching for a distance of the top face _(h 11 _{-h 13)} close distance _(h 21 _{-h 23).}

The fact that the distances between corresponding horizontal planes are the same among these three-dimensional point cloud data is that, from another perspective, when the height offset value is converted by obtaining the reference offset amount Δh of the correct height, 3 This means that a plurality of values of the same height can be obtained among the dimension point cloud data. FIG. 6 is a diagram showing the relationship between the set of heights of the horizontal plane and the offset amount Δh. In FIG. 6, each element of the set H ₁ is shown by a solid thick line, while each element of the set H ₂ is shown by a dashed-dotted thick line. When the correct answer for the set H ₁ and the set H ₂ shown in the left part of FIG. 6 is the offset amount Δh, a correction for adding the offset amount Δh to each element included in the set H ₂ as shown in the right part of FIG. Doing the height of the set _{H 1} of _{h 14} a set of _{H 2 h 23} with height _{h 21} of the set between _{H 2 h 11} set _{H 1} matches match. As described above, when the reference offset amount Δh of the correct height is obtained and the height value is converted, it is highly possible that a plurality of values of the same height are obtained among the three-dimensional point cloud data. From this, it is possible to solve it as a problem of obtaining the offset amount Δh.

[Parameter definition]
Here, a set of heights of the horizontal plane extracted from the first three-dimensional point cloud data measured at different measurement points is H _1, and the height of the horizontal plane extracted from the second three-dimensional point cloud data is When the set is H ₂ , the set H ₁ and the set H ₂ can be represented by the following numerical sequences.

H ₁ = {h ₁₁ , h ₁₂ , ..., H _1i , ..., h _1M }
H ₂ = {h ₂₁ , h ₂₂ , ..., h _2j , ..., h _2N }

At this time, assuming that the offset amount is Δh, a set H ₂ ′ of horizontal plane heights extracted from the corrected second three-dimensional point cloud data of the offset amount Δh is represented as follows.

_{H 2 '= {h 21'} , h 22 ', ..., h 2j', ..., h 2N '}
= {H ₂₁ + Δh, h ₂₂ + Δh, ..., h _2j + Δh, ..., h _2N + Δh}

[Calculation Method 1 of Offset Amount Δh]
As one embodiment, the offset amount calculation unit 17b obtains the offset amount Δh using the evaluation function E of the following formula (1), which represents the degree of coincidence between H ₁ and H ₂ ′. That is, the offset amount calculation unit 17b changes the offset amount Δh by a predetermined shift amount within the range of the inequality sign in the following formula (3) determined by the maximum altitude difference D shown in the following formula (2), and An offset amount Δh that minimizes the evaluation function E of (1) is obtained. In the evaluation function E shown in the following formula (1), the smaller the value, the higher the degree of coincidence between H ₁ and H ₂ ′.

E = Σ _i Σ _j log (1+ | h _1i −h _2j ′ |) (1)
D = max _{i, j} | h _1i −h _2j | ... (2)
-D ≦ Δh ≦ D (3)

Here, log (1 + d) in the evaluation function is a function that is convex upward, so that if the same amount is approached, it is a function that can make the value smaller nearer than far away (the closer the slope is, the steeper the slope is). ). FIG. 7 is a diagram illustrating an example of a set of heights. In FIG. 7, for the sake of convenience of description, a situation in which h ₁₁ is included as an element of the set H ₁ and h ₂₁ ′ and h ₂₂ ′ are included as elements of the set H ₂ ′ is simply described. Further, in FIG. 7, a state S1 in which the height h ₁₁ and the height h ₂₁ ′ are the same is shown on the left side, while a state S2 in which the offset amount Δh is changed over the shift amount a is shown on the right side. In the case of the example shown in FIG. 7, the sum of the distances of h ₂₁ ′ and h ₂₂ ′ with respect to the height h _{11 is} the same between the states S1 and S2, but the evaluation value is calculated according to the above formula (1). By doing so, the evaluation value calculated in the state S1 can be calculated lower than the evaluation value calculated in the state S2, that is, the matching degree can be calculated higher.

FIG. 8 is a diagram showing an example of the evaluation value. The left part of FIG. 8 shows the evaluation values calculated in the state S1 shown in FIG. 7, while the right part of FIG. 8 shows the evaluation values calculated in the state S2 shown in FIG. Has been done. As shown in the left part of FIG. 8, in the case of the state S1, the evaluation value E is calculated by e ₁ + e ₂ according to the above equation (1). Among these, e ₁ has the same height h ₁₁ and height h ₂₁ ′, so log (1+ | h ₁₁ −h ₂₁ ′ |) is “0”, while e ₂ is log ( 1+ | h ₁₁ −h ₂₂ ′ |) is calculated. On the other hand, as shown in the right part of FIG. 8, in the case of the state S2, the evaluation value E is calculated by e ₃ + e ₄ according to the above equation (1). Among, _{e 3} is, log | '| _{e 4} together with the _{calculated, log ((1+ | h 11} -h 22 1+ h 11 -h 21)' |) to be calculated. At this time, e ₃ becomes larger by the shift amount a than e _1, and the value becomes larger as h ₂₁ ′ moves away from h ₁₁ , while e ₄ becomes larger by the shift amount a than e _2. , H ₂₂ ′ becomes smaller as h ₂₂ ′ approaches h ₁₁ , but since log (1 + d) is a convex function, the increment becomes larger than the decrement. Therefore, the evaluation value calculated in the state S1 is lower than the evaluation value calculated in the state S2, that is, the degree of coincidence can be calculated higher. On the other hand, when the slope of the evaluation function is “1”, that is, when the evaluation function is Σ _i Σ _j | h _1i −h _2j ′ |, the increment and decrease are the same and there is no difference in the evaluation value. , Is not suitable as a matching evaluation function.

[Calculation Method 2 of Offset Amount Δh]
As another embodiment, the offset amount calculation unit 17b calculates, for each combination of height values from the set H ₁ and the set H _2, an offset amount Δh with which the heights of the combinations match. Subsequently, the offset amount calculation unit 17b calculates the number c _{ij in} which the heights of H ₁ and H ′ ₂ are within a predetermined range, for example ± α, with the offset amount Δh. Then, the offset amount calculation unit 17b calculates the number c _ij for all the combinations, and obtains the offset amount Δh that gives the maximum number.

  The calculation method 1 and the calculation method 2 of the offset amount Δh can be adaptively switched and used. For example, the offset amount calculation unit 17b adopts the above calculation method 1 when the product of the number of horizontal planes extracted from each measurement point is equal to or more than a predetermined threshold, and when the product is less than the threshold, By adopting the above calculation method 2, the algorithm can be switched according to the calculation amount.

  The alignment unit 17c is a processing unit that performs alignment of three-dimensional point cloud data between different measurement points, that is, registration.

As one embodiment, the alignment unit 17c aligns the height reference between the three-dimensional point cloud data of each measurement point according to the offset amount Δh calculated by the offset amount calculation unit 17b. That is, the alignment unit 17c includes the 3D point cloud data to which the set H ₂ is assigned among the 3D point cloud data measured at different measurement points, that is, each included in the second 3D point cloud data. The offset amount Δh is added to the point. After that, as described in Related Document 1 below, the alignment unit 17c sets the three-dimensional point cloud included in the section of the predetermined height for each of the three-dimensional point cloud data to be aligned to a predetermined plane. For example, two-dimensional horizontal slice data is extracted by projecting onto the floor surface, and these are two-dimensionally registered.

[Related Document 1] "Automatic generation of integrated point cloud model for large-scale environment (3rd report)
Fully automatic alignment of multiple point clouds by point cloud pair alignment and match judgment ”
Yusuke Matsuyama, Hiroaki Date, Osamu Kanai, 2014 JSPE Spring Conference, Session ID: E45

  In addition, the alignment unit 17c may align the two-dimensional point group extracted by slicing each three-dimensional point group at the same height in the horizontal plane by the method described in Related Document 2 below. I don't care.

  [Related document 2] Masahiro Tomono, “Two-dimensional global scan matching method using invariant features in Euclidean transformation, Journal of the Robotics Society of Japan, Vol. 25, No. 3, pp390-401 (2007)”

  To explain this, in the two-dimensional point cloud, the normal line of each point can be calculated from the positions of the surrounding points, and the point cloud with directions can be obtained. In the case of two dimensions, if one correspondence between data points is determined, the coordinate conversion parameter (x, y, θ) for alignment is obtained. Therefore, the correspondence candidates of the directional points are evaluated from the positional relationship of other points and narrowed down to a predetermined number. As a result, candidates for coordinate conversion are narrowed down. Therefore, after performing a detailed alignment process for minimizing the alignment error for each candidate, the residual is evaluated and the alignment result with the smallest residual is adopted. .. Thereby, two-dimensional registration can be realized.

  By performing meshing processing on the three-dimensional point group data integrated into one as described above, the data can be converted into three-dimensional surface data. By using the three-dimensional point cloud data or the three-dimensional surface data integrated into one of these, support data for supporting the worker 3 can be created and projected by the projection AR.

[Process flow]
Next, a processing flow of the portable information processing device 10 according to the present embodiment will be described. FIG. 9 is a flowchart illustrating the procedure of the overall process according to the first embodiment. This process is started when the three-dimensional point cloud data acquisition unit 15 groups the three-dimensional point cloud data measured at different measurement points as a registration execution target.

  As shown in FIG. 9, the plane extraction unit 17a uses the height direction of the sensor coordinate system of the portable information processing apparatus 10, that is, the Z axis as a reference, and the normal line of the horizontal plane for each three-dimensional point cloud data of each measurement point. The direction is calculated (step S101).

  Then, the plane extraction unit 17a converts the coordinates of each point included in the three-dimensional point group data for each three-dimensional point group data of each measurement point such that the positive direction of the Z axis is the normal direction of the horizontal plane. Yes (step S102). Then, the plane extraction unit 17a extracts a horizontal plane from the three-dimensional point cloud data for each three-dimensional point cloud data of each measurement point according to an algorithm such as RANSAC (step S103).

  After that, the offset amount calculation unit 17b is irradiated with the height (z coordinate) of each horizontal plane extracted from the three-dimensional point group data of each measurement point, for example, the LRF shown in FIG. The height with the position of the laser light source as the origin is calculated (step S104).

  Then, the offset amount calculation unit 17b calculates the offset amount Δh by matching the correspondence of the faces using the relative positional relationship (distance) of all horizontal planes extracted at each measurement point as a feature (step S105).

  Subsequently, the alignment unit 17c corrects the position of each point of the three-dimensional point cloud data of the measurement point according to the offset amount Δh obtained in step S105 (step S106). Then, the alignment unit 17c extracts the two-dimensional point cloud data sliced horizontally at the same height for each of the three-dimensional point cloud data (step S107). After that, the alignment unit 17c executes registration with the two-dimensional point group as shown in FIG. 10 (step S108), and ends the process.

[Registration processing]
FIG. 10 is a flowchart of a registration process procedure according to the first embodiment. As shown in FIG. 10, the alignment unit 17c sets one of the two-dimensional point groups extracted in step S107 as a reference point group, the other two-dimensional point group as a correction point group, and the reference point group and the correction point group. A normal direction is calculated for each point (step S301). Specifically, the alignment unit 17c extracts the two points that are farthest apart from the points existing in the constant distance range around the point of interest, is orthogonal to the two points, and is irradiated by the origin, that is, the LRF. The vector that goes in the direction of the position of the laser light source is calculated as the normal vector.

  Then, the alignment unit 17c extracts a plurality of point correspondence candidates between the reference point group and the correction point group (step S302). Specifically, the alignment unit 17c uses the positional relationship of other points when the coordinate system is defined with reference to the position of each point and the normal direction as a feature (signature), and uses the signature of each combination of points as a signature. The degree of coincidence is evaluated, and those having a degree of coincidence equal to or higher than a predetermined value are extracted as point correspondence candidates.

  Then, the alignment unit 17c calculates coordinate conversion parameters (Δx, Δy, Δθ) for each point correspondence (step S303). Specifically, the alignment unit 17c calculates the coordinate conversion parameter if the two-dimensional point groups obtained at the two different measurement points correspond to each other, that is, if one point at the same location is determined. be able to.

  Then, the alignment unit 17c clusters the plurality of coordinate conversion parameters obtained in step S303 to calculate the coordinate conversion parameter of the center of gravity of a cluster having a predetermined number or more of elements (steps S304 and S305).

  After that, the alignment unit 17c sets the positional relationship of the point group after conversion with each parameter as an initial position, and performs detailed alignment by an ICP (Iterative Closest Point) algorithm (step S306). Finally, the alignment unit 17c evaluates the residual of each alignment result, selects the alignment result with the highest degree of matching as the final solution (step S307), and ends the process.

[One side of effect]
As described above, in the portable information processing device 10 according to the present embodiment, the relative positional relationship between the planes existing in the environment of the site 2 between the three-dimensional point groups measured at different measurement points, that is, The two-dimensional registration is executed by calculating the offset amount in which the height difference between the planes is the same and adjusting the height reference. Therefore, according to the portable information processing apparatus 10 according to the present embodiment, it is possible to improve the synthesis accuracy of the three-dimensional point cloud data in an environment with occlusion.

  Although the embodiments of the disclosed device have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. Therefore, other embodiments included in the present invention will be described below.

[Extraction of horizontal plane]
An application example of the process of extracting the horizontal plane will be described as a previous step of correcting the offset amount Δh. For example, when the installation surface of the portable information processing device 10 is tilted, or when the legs of the portable information processing device 10 are slightly different and the sensor is tilted with respect to the horizontal plane, the following processing is performed. The horizontal plane can be extracted.

  That is, the knowledge that the upward vector in the coordinate system of the sensor itself has a small angular difference from the normal vector of the horizontal plane even if the portable information processing device 10 is inclined is used. In other words, if the angle difference is large, in other words, if the tilt of the portable information processing device 10 is large, the portable information processing device 10 will fall. Therefore, paradoxically, when the tilt has not occurred, the coordinates of the sensor itself. It can be said that the angle difference between the upward vector in the system and the normal vector in the horizontal plane is small.

For example, the portable information processing device 10 extracts a plane from the 3D point cloud data. The plane extracted here is not limited to the plane orthogonal to the Z axis of the sensor coordinate system. Then, the portable information processing device 10 divides the previously extracted normal vector n _{k of} each plane and the vector in the Z-axis positive direction of the sensor coordinate system, that is, the upward vector v _s of the sensor coordinate system. A plane whose angle θ _k is within a predetermined range is extracted. However, when θ _k > 90 °, the portable information processing apparatus 10 sets θ _k = 180 ° −θ _{k to} include an opposite vector in the extraction target. In addition, although the case where the normal vector of each plane is used is illustrated here, the normal vector of each point group included in each plane may be used.

The portable information processing apparatus 10 calculates the vector v _h that is the median value of the distribution of the angles, with reference to the distribution of the angles thus extracted. At this time, when a certain number of horizontal planes exist in the site 2, the vector v _h becomes a normal vector of the horizontal plane. From this, the portable information processing apparatus 10 sets the normal vector n _k of each plane within a predetermined angle including the vector v _h that is the median of the angle distribution and the vector in the opposite direction. The corresponding plane is extracted as the horizontal plane.

FIG. 11: is a figure which shows the application example regarding the extraction of a horizontal surface. FIG. 11 shows a case where the portable information processing device 10 is placed with an inclination from the horizontal plane. In the case of the example shown in FIG. 11, five planes having normal vectors n _{1 to} n ₅ are extracted from the three-dimensional point cloud data. Then, by extracting the normal vector whose angle θ _k formed with the upward vector v _{s of} the sensor coordinate system is within a certain range, the normal vector n ₅ is removed, while the normal vector n ₅ is removed. The vectors n _{1 to} n ₄ are extracted. Then, when the angles θ _k are sorted in ascending or descending order, the normal vectors n ₁ , n ₂ , and n ₄ have almost the same value, and thus the median is one of n ₁ , n ₂ , and n ₄ . become. As a result, even when the portable information processing device 10 is placed with an inclination with respect to the horizontal plane, the horizontal plane can be extracted.

Distributed and integrated
In addition, each component of each illustrated device may not necessarily be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part of the device may be functionally or physically distributed / arranged in arbitrary units according to various loads or usage conditions. It can be integrated and configured. For example, the three-dimensional point cloud data acquisition unit 15, the plane extraction unit 17a, the offset amount calculation unit 17b, or the alignment unit 17c may be connected via a network as an external device of the portable information processing device 10. In addition, another device has a three-dimensional point cloud data acquisition unit 15, a plane extraction unit 17a, an offset amount calculation unit 17b, or an alignment unit 17c, and the portable information described above can be obtained by network cooperation and cooperation. The functions of the processing device 10 may be realized.

[Information synthesis program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes an information synthesizing program having the same functions as those in the above embodiments will be described with reference to FIG.

  FIG. 12 is a diagram illustrating a hardware configuration example of a computer that executes the information composition program according to the first and second embodiments. As illustrated in FIG. 12, the computer 100 includes an operation unit 110a, a speaker 110b, a camera 110c, a display 120, and a communication unit 130. Further, the computer 100 has a CPU 150, a ROM 160, an HDD 170, and a RAM 180. Each of these units 110 to 180 is connected via a bus 140.

  As shown in FIG. 12, the HDD 170 stores an information composition program 170a that exhibits the same functions as the plane extraction unit 17a, the offset amount calculation unit 17b, and the alignment unit 17c described in the first embodiment. The information synthesizing program 170a may be integrated or separated similarly to the respective components of the plane extraction unit 17a, the offset amount calculation unit 17b, and the alignment unit 17c shown in FIG. That is, the HDD 170 does not necessarily need to store all the data described in the first embodiment, and the data used for the processing may be stored in the HDD 170.

  Under such an environment, the CPU 150 reads out the information composition program 170a from the HDD 170 and expands it in the RAM 180. As a result, the information composition program 170a functions as an information composition process 180a, as shown in FIG. The information composition process 180a expands various data read from the HDD 170 in an area allocated to the information composition process 180a in the storage area of the RAM 180, and executes various processes using the expanded data. For example, the processing shown in FIGS. 9 to 10 is included as an example of the processing executed by the information composition process 180a. In the CPU 150, not all the processing units shown in the above-described first embodiment need to operate, and the processing unit corresponding to the processing to be executed may be virtually realized.

  Note that the above information composition program 170a does not necessarily have to be stored in the HDD 170 or the ROM 160 from the beginning. For example, the information synthesizing program 170a is stored in a “portable physical medium” such as a flexible disk, a so-called FD, a CD-ROM, a DVD disk, a magneto-optical disk, an IC card, which is inserted into the computer 100. Then, the computer 100 may acquire the information composition program 170a from these portable physical media and execute it. Further, the information composition program 170a is stored in another computer or a server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, etc., and the computer 100 acquires the information composition program 170a from these. You may make it execute.

10 portable information processing device 11 display unit 13 storage unit 15 three-dimensional point cloud data acquisition unit 17 control unit 17a plane extraction unit 17b offset amount calculation unit 17c alignment unit

Claims (4)

Acquisition of point cloud information based on the distance from the own device to the object using the reflected light reflected from the object while irradiating while scanning the light at each of different first measurement position and second measurement position Department,
An extraction unit that extracts point cloud information belonging to a plane area from the point cloud information acquired by the acquisition unit for each of the first measurement position and the second measurement position ,
Based on the vertical distance between the planar regions extracted from the first measurement position and the vertical distance between the planar regions extracted from the second measurement position, A matching unit that calculates the offset amount between the first measurement position and the second measurement position by matching the correspondence relationship of the planar area between the measurement position and the second measurement position ,
An information processing device, comprising: an alignment unit that performs alignment using the point cloud information corrected according to the offset amount calculated by the calculation unit. The information processing apparatus according to claim 1, wherein the extraction unit extracts point cloud information regarding a horizontal plane region for each of the first measurement position and the second measurement position .   The extraction unit is a normal vector of a plane area in which an angle formed by a vector corresponding to a vertical direction of a coordinate system in which the point cloud information is acquired by the device itself and a normal vector of the plane area is within a predetermined range. And extracting, as the horizontal plane area, a plane area having a normal vector within a predetermined range from the median value of the distribution of the distribution of the angles of the normal vector of the extracted flat area. The information processing device according to claim 2. From the first measurement position or the second measurement position to the object using the reflected light that is emitted while scanning the light at each of different first measurement position and second measurement position and is reflected from the object A process of acquiring point cloud information based on the distance of
A process of extracting point cloud information belonging to a plane area from the acquired point cloud information for each of the first measurement position and the second measurement position ;
Based on the vertical distance between the planar regions extracted from the first measurement position and the vertical distance between the planar regions extracted from the second measurement position, A process of calculating an offset amount between the first measurement position and the second measurement position by matching the correspondence relationship of the planar area between the measurement position and the second measurement position ;
An information synthesizing program characterized by causing a computer to execute a process of performing alignment using the point group information corrected according to the calculated offset amount.

Images