JP6950208B2 - Information processing programs, information processing methods, and information processing equipment - Google Patents

Description

The present invention relates to an information processing program, an information processing method, and an information processing device.

Conventionally, there is a distance sensor that measures the distance to an object by irradiating the object with a laser and measuring the time until the reflected light is returned. In addition, there is a so-called 2D (two-dimensional: 2-Dimensions) sensor that measures the distance to an object while rotating the optical system in the distance sensor and scanning in the horizontal direction. In addition, there is a technique called 3D measurement that converts the position and shape of a three-dimensional object into data by rotating a 2D sensor with a driving device.

The three-dimensional measurement point cloud information obtained by 3D measurement is used for, for example, various simulators and applications such as CAD (Computer Aided Design). The measurement point is a point of an object irradiated with a laser by a 2D sensor. An application such as CAD can represent a space to be measured on a computer space based on information on the position of a measurement point or the position of a measurement point based on the distance from a 2D sensor to the measurement point, for example.

As a prior art, for example, when recognizing an object based on the distance image data acquired based on the pulsed laser emitted and reflected by the object, the distance image data is converted to an edge which is a part of the object. There is a technique for extracting a plane of an object by using such information (see, for example, Patent Document 1 below). Further, as a prior art, for example, there is a technique of detecting a straight line from a projected image obtained by projecting three-dimensional coordinate data onto a plane and detecting a plane on the three-dimensional coordinate data from the straight line (see, for example, Patent Document 2 below). ).

Japanese Unexamined Patent Publication No. 2006-317221 Japanese Unexamined Patent Publication No. 2014-85940

However, if there is an obstacle when measuring the space to be measured by the sensor, for example, the sensor cannot measure the distance of the portion where the laser is blocked by the obstacle. Therefore, conventionally, for example, when a three-dimensional model is generated using the measurement point cloud information obtained by this measurement, a model representing the shape of the portion where the laser is blocked cannot be generated.

In one aspect, the present invention is an information processing program, information processing method, and information capable of generating a model representing the shape of a portion not measured by an obstacle contained in a reference surface such as a floor surface, a wall surface, or a ceiling surface. It is an object of the present invention to provide a processing apparatus.

According to one aspect of the present invention, three-dimensional measurement point group information based on the distance from the measuring device to the object measured by the measuring device is acquired, and the measurement is performed from the acquired measurement point group information. The first measurement point group information existing on the reference plane measured by the apparatus and the second measurement point group information existing in a specific direction with respect to the reference plane are extracted, and the extracted first measurement point group information is extracted. The first model information indicating the first model based on the group information is acquired, the second model information indicating the second model based on the extracted second measurement point group information is acquired, and the acquired first model information and the acquired first model information are used. The information processing program, the information processing method, and the information processing apparatus that generate the third model information indicating the third model including the first model and the second model based on the acquired second model information are proposed. Will be done.

According to one aspect of the present invention, it is possible to generate a model representing the shape of a portion that is not measured by an obstacle included in a reference surface such as a floor surface, a wall surface, or a ceiling surface.

FIG. 1 is an explanatory diagram showing an example of one operation by the information processing device. FIG. 2 is an explanatory diagram showing an example of occlusion. FIG. 3 is an explanatory diagram showing an example of an information processing device 100 for performing 3D measurement. FIG. 4 is a block diagram showing a hardware configuration example of the information processing device 100. FIG. 5 is a block diagram showing a functional configuration example of the information processing apparatus 100. FIG. 6 is an explanatory view showing a specific example of the floor surface and the ceiling surface. FIG. 7 is an explanatory view showing a specific example of the wall surface. FIG. 8 is an explanatory view showing a second method example of modeling a reference plane which is a floor surface. FIG. 9 is an explanatory view showing a third method example of modeling a reference plane which is a floor surface. FIG. 10 is an explanatory diagram showing an example of a modeling method when a wall surface is used as a reference plane. FIG. 11 is a flowchart showing a processing procedure example 1 performed by the information processing apparatus 100. FIG. 12 is a flowchart showing a processing procedure example 2 performed by the information processing apparatus 100. FIG. 13 is a flowchart showing a processing procedure example 3 performed by the information processing apparatus 100. FIG. 14 is an explanatory diagram showing an example in which the measurement point cloud information 101 is modeled as it is. FIG. 15 is an explanatory diagram showing an example in which the measurement point cloud information 101 is separated and modeled. FIG. 16 is an explanatory diagram showing a system example.

The information processing program, the information processing method, and the embodiment of the information processing apparatus according to the present invention will be described in detail with reference to the drawings below.

FIG. 1 is an explanatory diagram showing an example of one operation by the information processing device. The information processing device 100 is a computer that acquires model information indicating a three-dimensional model based on the measurement point cloud information 101 obtained by 3D measurement. A three-dimensional model is an object represented in computer space. The computer space is a space set up to represent an object on a computer.

Further, in FIG. 1, the information processing device 100 is a measuring device itself that performs 3D measurement, but is not limited to this. The information processing device 100 may be, for example, a PC (Personal Computer) or a server capable of communicating with a measuring device that performs 3D measurement.

Here, the measurement point group information 101 has, for example, position information based on the distance from the measuring device to the measurement point for each measurement point. The information processing device 100 represents the position of each measurement point using a local Cartesian coordinate system such as the x-axis, y-axis, and z-axis. More specifically, the measurement point cloud information 101 includes a coordinate value indicating the position of the measurement point in the coordinate system C used in the information processing apparatus 100 for each measurement point. The coordinate values are the x-coordinate value, the y-coordinate value, and the z-coordinate value.

For example, CAD generates model information indicating a three-dimensional model based on, for example, measurement point cloud information 101. The model information is three-dimensional surface information that represents the space to be measured or the shape of an object by a surface. The model information may be, for example, CAD information such as information representing a three-dimensional shape by vertices and triangles, or analytical model information representing a three-dimensional shape by small elements such as a tetrahedron or a cube. Well, not particularly limited.

In addition, there are places where the measurement is not performed due to the state in which the object in the foreground hides the object behind (hereinafter referred to as occlusion). For example, in CAD, when a three-dimensional model is created using the measurement point cloud information obtained by this measurement, there is a problem that a model representing the shape of the occlusion portion cannot be generated. An example of modeling based on occlusion and measurement point cloud information with occlusion will be described with reference to FIG. For example, in order to measure the space to be measured without blind spots, it is conceivable to perform 3D measurement at a plurality of measurement points. However, it takes time and effort for the measurer to perform 3D measurement at a plurality of measurement points. Further, when a plurality of measurement point cloud information is acquired at different measurement points in order to measure the space to be measured without blind spots, it may be difficult to combine the plurality of measurement point cloud information.

Therefore, in the present embodiment, the information processing apparatus 100 separates the measurement point cloud information 101 obtained by the 3D measurement into the point cloud information existing on the floor surface and the point cloud information existing above the floor surface. Model information showing a model that combines each model modeled for each separated point cloud information is generated. As a result, the information processing apparatus 100 can generate a model that more faithfully represents the shape of the occlusion portion of the reference plane without the trouble of the measurer such as performing the measurement at a plurality of measurement points. Therefore, the labor of the measurer can be saved. In particular, the information processing device 100 is useful for spaces such as floors, ceilings, and rooms with walls. For example, when the reference plane is the floor plane, a model representing the shape of the occlusion portion of the floor plane can be generated. Therefore, for example, a three-dimensional model that is more faithful to the actual room can be generated without any hassle.

First, the information processing device 100 acquires three-dimensional measurement point cloud information 101 based on the distance from the measuring device measured by the measuring device to the object. The measuring device includes, for example, a distance sensor. The measuring device, for example, scans while irradiating light and measures the distance from the measuring device to the object by using the reflected light reflected from the object. The measuring device will be described later with reference to FIG. The three-dimensional measurement point cloud information 101 indicates each position in the three dimensions of the measurement point cloud in which the distance was measured. As a result, the information processing apparatus 100 can acquire the measurement point cloud information 101 obtained by the 3D measurement. The information processing device 100 may acquire the measurement point cloud information 101 by 3D measurement, or may acquire the measurement point cloud information 101 measured by another measuring device via a network or the like. In the example of FIG. 1, the information processing device 100 includes a measuring device.

Next, the information processing device 100 exists in the first measurement point cloud information corresponding to the reference plane measured by the measurement device from the acquired measurement point cloud information 101 and in a specific direction with respect to the reference plane. The second measurement point cloud information is extracted. The reference surface is a reference surface such as a floor surface, a ceiling surface, and a wall surface. The reference plane is automatically specified by the information processing apparatus 100 as described later. Further, the reference plane may be a plane designated by the user from among a plurality of planes specified by the measurement point cloud information 101. The first measurement point group information 521 corresponding to the reference plane is, for example, information on the measurement point group existing on the reference plane. The reference plane is substantially on the reference plane, and the measurement points existing on the reference plane include the measurement points existing in the vicinity of the reference plane by setting an error range. The specific direction is set according to the reference plane. In the example of FIG. 1, the floor surface will be described as a reference surface. Further, in the example of FIG. 1, the floor is shown below the measurement point existing on the floor surface for easy understanding. When the reference plane is the floor plane, the specific direction is, for example, a direction above the floor plane. Specifically, the information processing apparatus 100 has, for example, the first measurement point cloud information corresponding to the floor surface and the height above the floor surface from the acquired measurement point cloud information 101. The second measurement point cloud information is extracted. As a result, the measurement point group on the floor surface and the measurement point group existing above the floor surface are separated.

Next, the information processing apparatus 100 acquires the first model information 121 indicating the first model 111 based on the extracted first measurement point cloud information. Specifically, the information processing apparatus 100 acquires the first model information 121 indicating the first model 111 by CAD by giving the first measurement point cloud information to, for example, CAD. This gives a model of the floor surface. There is a part of the floor surface that does not have measurement points, but when modeling, if the measurement point cloud information 101 corresponding to the floor surface is provided, a surface is formed between the measurement points corresponding to the floor surface. NS. In this way, the surface of the model is formed so that the portion without the measurement point becomes the floor surface. Therefore, a model representing the shape of the occlusion portion of the floor surface can be obtained.

Further, the information processing apparatus 100 acquires the second model information 122 indicating the second model 112 based on the extracted second measurement point cloud information. Specifically, the information processing apparatus 100 acquires the second model information 122 indicating the second model 112 by CAD by giving the second measurement point cloud information to, for example, CAD. This gives a model of obstacles above the floor. The order of generation processing of the first model information 121 and the second model information 122 is not particularly limited.

Next, the information processing apparatus 100 shows a third model 113 including the first model 111 and the second model 112 based on the acquired first model information 121 and the acquired second model information 122. 3 Generate model information 123. Specifically, the information processing apparatus 100 generates the third model information 123 including the first model information 121 and the second model information 121.

When the entire measurement point cloud information 101 is modeled as it is, since there are no measurement points in the occlusion portion of the floor surface, a model in which the floor surface and the obstacle are integrated may be generated. On the other hand, the information processing apparatus 100 generates the first model 111 of the floor surface and the second model 112 of the obstacle separately, so that the floor surface and the obstacle are not integrated. Then, the information processing apparatus 100 can obtain a model that more faithfully represents the shape of the occlusion portion on the floor surface.

FIG. 2 is an explanatory diagram showing an example of occlusion. In FIG. 2 (1), an obstacle is placed on the floor in the space to be measured. The obstacle is, for example, an object such as a desk or a box, and is not particularly limited. In FIG. 2 (1), when the measuring instrument irradiates the laser from the left side to the right side, there is a portion on the floor surface that becomes occlusion due to an obstacle. In FIGS. 2 (1) and 2 (2), the square points are the measurement points.

As described above, in an application such as CAD, for example, model information representing the space to be measured by a surface can be generated based on the measurement point cloud information. When an application such as CAD generates model information indicating a model 200 from measurement point cloud information having an occlusion part, for example, a surface is created between close measurement points in order to interpolate the part without the measurement point cloud. I have something to do. More specifically, as shown in FIG. 2 (2), in an application such as CAD, a surface may be created between the measurement point p1 and the measurement point p2. A model 200 is created in which the floor surface and the obstacle are integrated by the surface s1, and the distinction between the floor surface and the obstacle may be unclear. As described above, the model created based on the measurement point cloud information may not be able to represent the floor or obstacles. As described with reference to FIG. 1, in the present embodiment, the information processing apparatus 100 has measurement point cloud information corresponding to the floor surface and measurement point cloud information existing above the floor surface (measurement points corresponding to obstacles). Group information) and model information that combines each model modeled for each is generated. As a result, the information processing apparatus 100 can obtain a model that faithfully represents the shape of the occlusion portion.

FIG. 3 is an explanatory diagram showing an example of an information processing device 100 for performing 3D measurement. In FIG. 3, the information processing device 100 includes a driving device 301, a measuring device 302, and a control device 303. The information processing device 100 is installed and used on a horizontal floor surface, for example. In FIG. 3, the shape of the housing of the information processing device 100 is, for example, substantially the same regardless of the direction in which the information processing device 100 is viewed, but is not limited to this.

Further, the tripod portion of the information processing device 100 can be expanded and contracted, for example. The height of the information processing device 100 can be changed by expanding and contracting the tripod portion of the information processing device 100. The height of the information processing device 100 can be changed from, for example, 30 cm to 130 cm by the tripod portion.

The drive device 301 is a motor that rotates in the pan direction d1 about a rotation shaft 310 (corresponding to the z-axis in FIG. 3). In the following description, the rotation angle in the pan direction d1 (the angle with respect to the x-axis in FIG. 3) may be expressed as “rotation angle θh (θh = 0 to 360 ° (degrees))”. However, it is assumed that the front direction and the x-axis direction when the drive device 301 is in the initial position coincide with each other.

The measuring device 302 is a 2D sensor that measures the distance from the own device to the object by irradiating the object with light (for example, a laser) while scanning in the tilt direction d2 and using the time until the reflected light is received. be. The tilt direction d2 is a direction of rotation about the tilt axis 320. That is, the measuring device 302 rotates the optical system (for example, the light emitting unit 411, the light receiving unit 412, etc. shown in FIG. 4 described later) around the tilt axis 320, and the direction indicated by the thick arrow in FIG. 3 (irradiation direction). ) Is irradiated with light.

In the information processing device 100, the measuring device 302 is attached to the driving device 301 so that the tilt direction d2 is perpendicular to the pan direction d1. Then, the measuring device 302 irradiates the object with light (hereinafter, referred to as “laser”) while scanning in the tilt direction d2 while being moved along the circular orbit around the rotation axis 310 to reach the object. Measure the distance.

At this time, the measuring device 302 irradiates the laser in the tilt direction d2 in the range of “0 to 360 °”. The tilt angle θv is an angle of the tilt direction d2 with respect to the rotation axis 310. More specifically, for example, the measuring device 302 performs this measurement each time it is moved from 0 ° to 360 ° in the tilt direction d2 by a predetermined angle. For example, when the predetermined angle is 0.25 °, the measuring device 302 measures about 1440 points at 25 [msec] per lap. Further, the measuring device 302 performs this measurement every time, for example, the driving device 301 moves in the pan direction d1 by a predetermined angle from "0 to 360 °". However, it is not necessary to measure from 0 ° to 360 ° in both the tilt direction and the pan direction. If either one performs the measurement from 0 to 180 °, the measurement in all directions can be performed. Further, the measurement range may be limited by specifying an angle, not limited to all directions. As a result, 3D measurement can be performed using the measuring device 302 which is a 2D sensor.

(Example of hardware configuration of information processing device 100)
FIG. 4 is a block diagram showing a hardware configuration example of the information processing device 100. In FIG. 6, the information processing device 100 includes a CPU (Central Processing Unit) 401, a memory 402, an I / F (Interface) 403, a driving device 301, and a measuring device 302. Further, each component is connected by a bus 400.

Here, the CPU 401 controls the entire information processing apparatus 100. The memory 402 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash ROM, and the like. Specifically, for example, a flash ROM or ROM stores various programs, and the RAM is used as a work area of the CPU 401. The program stored in the memory 402 is loaded into the CPU 401 to cause the CPU 401 to execute the coded process.

The I / F 403 is connected to a wired or wireless network and is connected to another computer (eg, a user's personal computer) via the network 410. Then, the I / F 403 controls the interface between the network 410 and the inside of the own device, and controls the input / output of data from another computer. The control device 303 shown in FIG. 3 includes, for example, a CPU 401, a memory 402, and an I / F 403.

The drive device 301 is a motor that rotates about the rotation shaft 310 in the pan direction d1. The measuring device 302 includes a light emitting unit 411, a light receiving unit 412, a driving unit 413, and a sensor control unit 414. The light emitting unit 411 is a light source that irradiates a laser, and is, for example, a semiconductor laser. The light receiving unit 412 receives the reflected light. The drive unit 413 rotates the light emitting unit 411 around the tilt shaft 320 in the tilt direction d2. The sensor control unit 414 irradiates the object with the laser while scanning in the tilt direction d2, and measures the distance from the own device to the object using the time until the reflected light is received. The sensor control unit 414 outputs the coordinate value of the object in the coordinate system C as the coordinate data of the measurement point to the memory 402 or the like based on the distance.

In addition to the above-described components, the information processing device 100 may include, for example, a disk drive, a disk, an SSD (Solid State Drive), an input device, a display, and the like.

(Example of functional configuration of information processing device 100)
FIG. 5 is a block diagram showing a functional configuration example of the information processing apparatus 100. The information processing device 100 includes a acquisition unit 501, a control unit 502, and a storage unit 510. The storage unit 510 is realized by, for example, a memory 402 or the like. The acquisition unit 501 is realized by the driving device 301 and the measuring device 302. Further, the control unit 502 includes a first specific unit 511, a second specific unit 512, an extraction unit 513, a third specific unit 514, an additional unit 515, a first acquisition unit 516, and a second acquisition unit 517. And a generation unit 518. The processing of the control unit 502 from the first specific unit 511 to the generation unit 518 is realized by, for example, a memory 402 or the like. The CPU 401 shown in FIG. 4 is coded in a program stored in an accessible memory 402 or the like. Then, the CPU 401 reads the program from the memory 402 or the like and executes the process coded in the program. As a result, the processing of the control unit 502 is realized. Further, the processing result of the control unit 502 is stored in, for example, a memory 402 or the like.

The acquisition unit 501 acquires the measurement point cloud information 101. Specifically, the acquisition unit 501 acquires the measurement point group information 101 based on the distance from the self-information processing device 100 to the object by scanning while irradiating the light and using the reflected light reflected from the object. do.

Next, the first specific unit 511 specifies the measured reference plane based on the acquired measurement point cloud information 101. Specifically, the first specific unit 511 specifies the distribution of the number of measurement points in the height direction by counting the number of measurement points in the height direction, for example, based on the measurement point cloud information 101. Here, the height direction can be specified, for example, based on the coordinate values of each axis of the measurement point cloud information 101. Further, for example, when the information processing device 100 is installed on a surface horizontal to the floor surface and measurement is performed, the height direction is a direction parallel to the z-axis direction shown in FIG. Then, the first specific unit 511 has a first reference plane based on the measurement point group information 101 on the upper side in the height direction of the measurement point group information 101, and a first reference plane based on the measurement point group information 101 on the lower side in the height direction. 2 Specify the reference plane.

The upper measurement point group information 101 is, for example, the highest in the height range when the height range from the height of the highest measurement point to the height of the lowest measurement point is 100 [%]. The measurement point group information 101 is included in the range from the height to the height of a predetermined number [%] in the direction opposite to the height direction. The predetermined number [%] is not particularly limited, but is, for example, about 10 [%]. The lower measurement point cloud information 101 is, for example, measurement point cloud information 101 included in a range from the lowest height in the height range to a predetermined number [%] in the height direction. The predetermined number [%] is not particularly limited, but is, for example, about 10 [%].

The first specific unit 511 specifies a plane having the highest number of measurement points in the upper measurement point cloud information 101 as a ceiling surface. The first specific unit 511 specifies a plane having the highest number of measurement points in the lower measurement point cloud information 101 as a floor surface. An example of specifying the floor surface and the ceiling surface by the first specific unit 511 will be described in detail with reference to FIG.

The second specific unit 512 is, for example, based on the distribution of the number of measurement points in the height direction obtained from the acquired measurement point cloud information 101, from the measurement point cloud information 101, the fourth according to the floor surface. The measurement point cloud information and the fifth measurement point cloud information according to the ceiling surface are extracted. Then, the second specific unit 512 is high between each vertical plane with respect to the floor surface based on the extracted fourth measurement point cloud information and each vertical plane with respect to the ceiling surface based on the extracted fifth measurement point cloud information. Identify a reference plane that is a vertical plane whose positions match in each direction other than the vertical direction.

Here, the process of extracting the vertical plane from the measurement point cloud information 101 will be briefly described. Specifically, the second specific unit 512 is a measurement point included in the measurement point cloud information (for example, the fourth measurement point cloud information and the fifth measurement point cloud information) according to an algorithm such as RANSAC (Randam Sample consensus). Extract the vertical plane formed by the group.

More specifically, the second specific unit 512 uses, for example, a point cloud specified by the measurement point cloud information as a sample, and randomly extracts three points from this sample. Subsequently, the second specific unit 512 further extracts a point cloud within a predetermined distance from the vertical plane model determined by three points randomly extracted from the sample from the point cloud specified by the measurement point cloud group information. .. A point cloud within a predetermined distance from the vertical plane model is regarded as a point cloud existing on the vertical plane model. This vertical plane model is a model used to define a vertical plane, unlike the three-dimensional model represented on the computer space by CAD described above.

Then, the second specific unit 512 determines whether or not the number of point groups existing on the vertical plane model is equal to or greater than the threshold value. In the second specific unit 512, when the point cloud existing on the vertical plane model is equal to or larger than the threshold value, the vertical plane in which the parameter defining the vertical plane model and the point cloud existing on the vertical plane model are associated with each other. The data is stored in the storage unit 510 or the like. Parameters that define the vertical plane model include, for example, the coordinates of three points or the equation of the vertical plane. When the number of point clouds existing on the vertical plane model is less than the threshold value, the second specific unit 512 repeatedly executes random sampling of three points from the sample and storage of the plane data associated therewith over a predetermined number of trials. By such a plane extraction method, it is possible to obtain a vertical plane model in which a certain number or more of point clouds are within a certain distance in the normal direction from the vertical plane model. The constant distance is the minimum distance between a plurality of vertical plane models parallel to each other. In other words, for example, the distance between the plane models of a plurality of plane models parallel to each other is a certain distance or more. The second specifying unit 512 specifies the plane defined by the vertical plane model as a vertical plane. Although the method of specifying the vertical plane has been described here, the horizontal plane substantially orthogonal to the vertical plane can also be specified in the same manner.

In this way, each vertical plane with respect to the floor surface and each vertical plane with respect to the ceiling surface can be obtained. The second specifying portion 512 identifies a vertical plane whose position other than the height direction coincides between each vertical plane with respect to the floor surface and each vertical plane with respect to the ceiling surface. A vertical plane that coincides with a position other than the height direction is a wall surface. The wall surrounds all four sides of the room. It is presumed that the wall surface exists between the ceiling surface and the floor surface. The vertical planes present on both the vertical plane with respect to the floor surface and the vertical plane with respect to the ceiling surface are likely to be wall surfaces. An example of specifying the wall surface by the second specific unit 512 will be described in detail with reference to FIG. 7.

Next, an example of modeling will be described using three methods. In the first method, as shown in FIG. 1, the second measurement point cloud information 522 existing in a specific direction with respect to the reference plane is separated from the measurement point cloud information 101, and each measurement point cloud information 101 Each model information that models is generated and synthesized. In the second method, the measurement point cloud information corresponding to the missing portion of the reference plane without the measurement point is added to the measurement point cloud information 101, and the model information obtained by modeling the added measurement point cloud information 101. To generate. The third method combines the first method and the second method.

First, the first method of modeling will be described.

From the measurement point cloud information 101 acquired by the acquisition unit 501, the extraction unit 513 includes the first measurement point cloud information 521 corresponding to the reference plane and the second measurement point cloud information existing in a specific direction with respect to the reference plane. 522 and are extracted. The specific direction is set according to, for example, a reference plane. When the reference plane is the floor surface, the specific direction is the direction from the floor surface to the ceiling surface, and the height direction toward the higher side. When the reference plane is the ceiling plane, the specific direction is the direction from the ceiling plane to the floor plane, and the height direction toward the lower side. When the reference plane is a wall surface, the specific direction is a direction from the wall surface toward the wall surface facing the wall surface.

More specifically, the extraction unit 513 extracts the second measurement point cloud information 522 existing in a specific direction with respect to the reference plane from the measurement point cloud information 101 acquired by the acquisition unit 501. Then, the extraction unit 513 extracts the measurement point cloud information 101 other than the second measurement point cloud information 522 from the acquired measurement point cloud information 101 as the first measurement point cloud information 521.

Then, the first acquisition unit 516 acquires the model information indicating the first model 111 based on the first measurement point cloud information 521. Specifically, the first acquisition unit 516 gives the first measurement point cloud information 521 to an application such as CAD. Then, the first acquisition unit 516 acquires the model information generated by the application such as CAD. For applications such as CAD, the information processing device 100 may be executed, or a device different from the information processing device 100 may be executed.

Then, the second acquisition unit 517 acquires the model information indicating the second model 112 based on the second measurement point cloud information 522. Specifically, the second acquisition unit 517 gives the second measurement point cloud information 522 to the CAD. Then, the second acquisition unit 517 acquires the model information generated by the application such as CAD.

Then, the generation unit 518 generates the third model information 123 indicating the third model 113 including the first model 111 and the second model 112 based on the first model information 121 and the second model information 122. do. For example, when the first model information 121 is P1, the second model information 122 is P2, and the third model information 123 is P, the generation unit 518 is generated as "P = {P1, P2}". Just do it. The first method of modeling is as shown in FIG.

By the way, for example, a plurality of objects such as desks and shelves may be installed apart from each other on the floor. When the measurement point cloud information corresponding to the desk and the measurement point cloud information corresponding to the shelf are modeled together, the desk and the shelf may be integrated. Therefore, the second acquisition unit 517 groups the measurement points included in the second measurement point group information 522 based on the distance between the measurement points included in the extracted second measurement point group information 522. Specifically, the second acquisition unit 517 sets, for example, measurement points whose distances are equal to or less than a threshold value into the same group. The second acquisition unit 517 acquires the second model information 122 indicating the second model 112 based on the information of the measurement points included in each group for each group of the plurality of grouped groups. Then, the generation unit 518 includes the first model information 121, the second model information 122 acquired for each group, the first model 111, and the second model 112 for each group. The third model information 123 indicating the model 113 is generated. The details of the generation method are as described in the first method.

Next, the second method of modeling will be described.

The extraction unit 513 extracts the first measurement point cloud information 521 corresponding to the reference plane from the measurement point cloud information 101 acquired by the acquisition unit 501.

Then, the third specific unit 514 identifies a missing portion of the reference plane in which the density of the measurement point group included in the first measurement point group information 521 is equal to or less than a predetermined density, based on the first measurement point group information 521. .. For example, the third specific unit 514 may divide the reference plane into a plurality of regions and determine whether or not the density of the measurement point group is equal to or less than a predetermined density for each region. More specifically, the third specific unit 514 identifies the region as a missing portion when the number of measurement point groups is equal to or less than the threshold value for each region. The threshold value is determined based on the distance between the information processing device 100 and the reference plane. When the distance between the information processing device 100 and the reference plane is long, the number of measurement points per area is small, and when the distance between the information processing device 100 and the reference plane is short, the number of measurement points per area is large. The number of measurement points per region is determined based on the resolutions in the tilt direction d2 and the pan direction d1.

Further, the third specific unit 514 identifies a missing portion in which the density of the measurement point cloud is equal to or less than a predetermined density based on the image in which each measurement point of the measurement point group included in the first measurement point group information 521 is colored. You may. Specifically, the third specific unit 514 generates image information indicating an image in which each measurement point of the measurement point group included in the first measurement point group information 521 is colored. Then, the third specific unit 514 may specify the uncolored portion from the image indicated by the generated image information as a missing portion whose density of the measurement point group is equal to or less than a predetermined density.

The addition unit 515 generates a third measurement point cloud information including the measurement point cloud information 101 acquired by the acquisition unit 501 and the measurement point cloud information corresponding to the missing portion. The third measurement point cloud information generated here is also referred to as the added measurement point cloud information 101. Specifically, the addition unit 515 generates measurement point cloud information corresponding to the missing portion. The measurement point group information corresponding to the missing portion is information indicating the position of the measurement point group when the measurement points of the missing portion are provided so as to have a predetermined density or more. For example, when the reference plane is a plane substantially perpendicular to the z-axis direction, the additional portion 515 includes the same z-coordinate value as the z-coordinate value of each measurement point included in the first measurement point group information 521, and the missing portion includes the z-coordinate value. The measurement point cloud information 101 including the corresponding x-coordinate value and y-coordinate value is generated. Then, the addition unit 515 adds the generated measurement point cloud information 101 to the acquired measurement point cloud information 101. More specifically, for example, when the measurement point group information 101 is Dk, the generated measurement point group information is Dx, and the added measurement point group information 101 is D, the additional unit 515 is set to "D = {Dk". , Dx} ".

Then, the first acquisition unit 516 acquires model information indicating a model based on the measurement point cloud information 101 after addition by the addition unit 515. The method of acquiring model information by the first acquisition unit 516 is the same as the acquisition method described in the first method. As for the second method, FIG. 8 shows a simple example in which the floor surface is used as the reference surface.

Further, the information processing apparatus 100 may be configured so that either the first method or the second method can be selected. In the first method, the processing time can be shortened as compared with the second method. On the other hand, in the second method, a model that more faithfully represents the shape of the occlusion portion can be obtained as compared with the first method when the missing portion is large. For example, if the missing portion of the reference plane is large, it is not clear what kind of measurement point is used for the missing portion when creating model information by an application such as CAD. Therefore, for example, the information processing apparatus 100 performs the second method when the missing portion is larger than the predetermined size, and performs the first method when the missing portion is smaller than the predetermined size.

Finally, a third method of modeling will be described. The third method is a combination of the first method and the second method.

From the measurement point cloud information 101 acquired by the acquisition unit 501, the extraction unit 513 includes the first measurement point cloud information 521 corresponding to the reference plane and the second measurement point cloud information existing in a specific direction with respect to the reference plane. 522 and are extracted. The first measurement point cloud information 521 corresponding to the reference plane is, for example, the first measurement point cloud information 521 existing on the reference plane.

Based on the first measurement point group information 521, the third specific unit 514 identifies a missing portion of the reference plane in which the density of the measurement point group included in the first measurement point group information 521 is equal to or less than a predetermined density. The specific method is as described in the second method.

The addition unit 515 generates a third measurement point group information including the extracted first measurement point group information 521 and the measurement point group information corresponding to the missing portion. The method of generating the measurement point cloud information corresponding to the missing portion is as described in the second method. The third measurement point cloud information generated here is also referred to as the added first measurement point cloud information 521.

Next, the first acquisition unit 516 acquires model information indicating the first model 111 based on the added first measurement point cloud information 521 by the addition unit 515. The second acquisition unit 517 acquires model information indicating the second model 112 based on the second measurement point cloud information 522. The method of acquiring model information by the first acquisition unit 516 and the second acquisition unit 517 is as described in the first method.

Then, the generation unit 518 synthesizes the model information indicating the first model 111 and the model information indicating the second model 112. The synthesis method is as described in the first method. As for the third method, FIG. 9 shows a simple example in which the floor surface is used as the reference surface.

When modeling is performed by CAD as in the second method, it is clear how a surface is created between the part of the reference surface to which the measurement point is added and the part in contact with the reference surface. Not. Therefore, with respect to the third method, by modeling the reference plane and the portion in contact with the reference plane separately, a model that more faithfully represents the shape of the portion in contact with the reference plane can be obtained.

The information processing device 100 may be configured so that any one of the first method to the third method can be selected depending on the user and various conditions.

Further, as described above, an example in the case where the reference plane is a wall surface specified by the second specific portion 512 is shown in FIG.

Further, the extraction process by the extraction unit 513 when there are a plurality of reference planes will be described. The method of specifying the reference plane is as described above. Therefore, there may be a plurality of reference surfaces such as a floor surface, a ceiling surface, and a wall surface.

The extraction unit 513 extracts the first measurement point cloud information 521 corresponding to each of the plurality of measured reference planes from the measurement point cloud information 101. Further, the extraction unit 513 exists in a specific direction set for each reference plane of the plurality of measured reference planes, and extracts the second measurement point cloud information 522 other than the first measurement point cloud information 521. The specific direction is set by each reference plane. The particular direction for the floor is towards the ceiling. A particular direction for the ceiling surface is towards the floor surface. The particular direction for the wall is the direction towards the opposite wall. When the reference surface is a floor surface, a ceiling surface, or a wall surface, the extraction unit 513 extracts the measurement point cloud information 101 in the area surrounded by the floor surface, the ceiling surface, and the wall surface from the acquired measurement point cloud information 101. It is extracted as the second measurement point cloud information 522.

FIG. 6 is an explanatory view showing a specific example of the floor surface and the ceiling surface. The first specific unit 511 specifies, for example, the distribution 600 of the number of measurement points in the height direction based on the acquired measurement point cloud information 101. In the distribution 600, the vertical axis is the height and the horizontal axis is the number of measurement points. In the distribution 600, the height range from the height of the highest measurement point MAX (Maximum) to the height of the lowest measurement point MIN (Minimum) is 100 [%].

Then, the first specific unit 511 is included in the range from the height MAX to the height of 10 [%] in the direction of lowering the height from the acquired measurement point cloud information 101 based on the distribution 600. The measurement point cloud information 101 is extracted. The measurement point cloud information 101 extracted here is the upper measurement point cloud information 101 described above. This 10 [%] is a predetermined number and is not particularly limited. The first specific unit 511 specifies a horizontal plane having the largest number of measurement points in the distribution in the height direction of the measurement point cloud specified by the extracted measurement point cloud information 101 as the ceiling surface.

Further, the first specific unit 511 is included in the range from the height MIN to the height of 10 [%] in the direction in which the height increases from the acquired measurement point cloud information 101 based on the distribution 600. The measurement point cloud information 101 is extracted. The measurement point cloud information 101 extracted here is the lower measurement point cloud information 101 described above. This 10 [%] is a predetermined number and is not particularly limited. The first specific unit 511 specifies the horizontal plane having the largest number of measurement points in the distribution in the height direction of each measurement point group specified by the extracted measurement point group information 101 as the floor surface.

FIG. 7 is an explanatory view showing a specific example of the wall surface. FIG. 7 (1) shows the measurement point cloud information 101 in the area near the ceiling surface and the measurement point cloud information 101 in the area near the floor surface. 7 (2-1) and 7 (2-2) show a vertical plane in the measurement point cloud information 101 in the region near the ceiling surface and a vertical plane in the measurement point cloud information 101 in the region near the floor surface. FIG. 7 (3) shows a state in which the vertical planes are overlapped.

As shown in FIG. 7 (1), in the distribution 600, the second specific unit 512 has a height of 20 [%] from the acquired measurement point cloud information 101 in the direction in which the height decreases from the height MAX. The measurement point cloud information 101 included in the above range is extracted. This 20 [%] is a predetermined number and is not particularly limited. The measurement point cloud information 101 extracted here is the measurement point cloud information 101 in the region near the ceiling surface.

The second specific unit 512 extracts the measurement point cloud information 101 included in the range from the height MIN to the height of 10 [%] in the height direction from the acquired measurement point cloud information 101 in the distribution 600. do. This 10 [%] is a predetermined number and is not particularly limited. The measurement point cloud information 101 extracted here is the measurement point cloud information 101 in the area near the floor surface.

The second specific unit 512 extracts a plane perpendicular to the ceiling surface based on the measurement point cloud information 101 in the region near the ceiling surface. The process of extracting the vertical plane is as described above. The second specific unit 512 extracts a plane perpendicular to the floor surface, for example, based on the measurement point cloud information 101 in the region near the floor surface. The process of extracting the vertical plane is as described above. Here, each vertical plane is a plane in a direction substantially parallel to the height direction. FIG. 7 (2-1) shows each first edge of the vertical plane when the vertical plane with respect to the ceiling plane is viewed from the direction opposite to the height direction (top). FIG. 7 (2-2) shows each second edge of the vertical plane when the plane perpendicular to the floor plane is viewed from the direction opposite to the height direction (above).

The second specifying portion 512 specifies the vertical planes of the matching edges as the wall surface (reference plane) when the first edge and the second edge are overlapped. The matching edges in FIG. 7 (3) are thick line portions. For example, the vertical planes of the edges that do not match in the vertical planes of the region near the ceiling plane are, for example, vertical planes representing an object attached to the ceiling. For example, the vertical planes of the edges that do not match in the vertical planes of the area near the ceiling plane are vertical planes representing an object such as a shelf placed on the floor.

FIG. 8 is an explanatory view showing a second method example of modeling a reference plane which is a floor surface. The third specific unit 514 is a missing portion 800 of the reference surface in which the density of the measurement points included in the first measurement point group information 521 is equal to or less than a predetermined density based on the first measurement point group information 521 corresponding to the floor surface. To identify. The addition unit 515 adds, for example, the measurement point cloud information corresponding to the missing portion 800 to the acquired measurement point cloud information 101.

Then, the first acquisition unit 516 acquires model information indicating the model based on the measurement point cloud information 101 added by the addition unit 515. In CAD, when modeling, a surface is not created between the measurement point p1 and the measurement point p2, and the distance from the measurement point p1 is closer than the measurement point p2 between the measurement point p3 and the measurement point p1. It is presumed to create a face in. Therefore, the information processing apparatus 100 can create a model that more faithfully represents the shape of the occlusion portion on the floor surface.

FIG. 9 is an explanatory view showing a third method example of modeling a reference plane which is a floor surface. The extraction unit 513 uses the measurement point cloud information 101 acquired by the acquisition unit 501 to obtain the first measurement point cloud information 521 corresponding to the floor surface and the second measurement point cloud information 521 existing in a specific direction with respect to the floor surface. 522 and are extracted. As a result, the measurement point cloud information 101 is separated into a first point cloud information 521 corresponding to the floor surface and a second measurement point cloud information 522 existing in a specific direction with respect to the floor surface. If the floor is the reference plane, the particular direction is the height direction.

The third specific unit 514 is a missing portion 800 of the floor surface in which the density of the measurement points included in the first measurement point group information 521 is equal to or less than a predetermined density based on the first measurement point group information 521 corresponding to the floor surface. To identify. The addition unit 515 adds the measurement point cloud information corresponding to the missing portion 800 to the first measurement point cloud information 521, for example.

Then, the first acquisition unit 516 acquires the first model information 121 indicating the first model 111 based on the first measurement point cloud information 521 to which the measurement point cloud information corresponding to the missing portion 800 is added by the addition unit 515. ..

The second acquisition unit 517 acquires the second model information 122 indicating the second model 112 based on the extracted second measurement point cloud information 522. Then, the generation unit 518 synthesizes the first model information 121 and the second model information 122. As a result, the information processing apparatus 100 can create a model that more faithfully represents the shape of the occlusion portion (missing portion 800) on the floor surface.

FIG. 10 is an explanatory diagram showing an example of a modeling method when a wall surface is used as a reference plane. In FIG. 10 (1), each vertical plane specified as a reference plane by the second specific portion 512 is shown by a thick line. In FIG. 10 (2), the vertical plane after the closed loop is shown by a thick line. Here, as a modeling method when the wall surface is used as a reference plane, the above-mentioned third method will be used.

The extraction unit 513 performs a closed loop processing on the end of each vertical surface specified as the reference surface by the second specific unit 512. Then, the extraction unit 513 extracts the second measurement point cloud information 522 inside the region surrounded by the vertical plane after the closed loop processing from the acquired measurement point cloud information 101. Specific directions are set for each of the vertical planes, and the extraction unit 513 can set a region surrounded by the vertical planes based on these specific directions. In addition, the extraction unit 513 extracts the first measurement point cloud information 521 existing on the vertical plane from the acquired measurement point cloud information 101. The extraction unit 513 extracts the sixth measurement point cloud information outside the region surrounded by the vertical plane from the acquired measurement point cloud information 101.

The third specific unit 514 identifies the missing portion 800 in the vertical plane based on the first measurement point cloud information 521. Then, the addition unit 515 adds the measurement point cloud information corresponding to the missing portion 800 to the first measurement point cloud information 521.

The first acquisition unit 516 acquires the first model information 121 indicating the first model 111 based on the first measurement point cloud information 521 added by the addition unit 515 and the extracted sixth measurement point cloud information. The second acquisition unit 517 acquires the second model information 122 indicating the second model 112 based on the extracted second measurement point cloud information 522. The generation unit 518 generates the third model information 123 indicating the third model 113 including the first model 111 and the second model 112.

As a result, the information processing apparatus 100 can create a model that more faithfully represents the shape of the occlusion portion of the wall surface.

(Example of processing procedure performed by the information processing apparatus 100)
Next, an example of a processing procedure performed by the information processing apparatus 100 will be described with reference to FIGS. 11 to 13. An example of the processing procedure in the first method of modeling described above will be described with reference to FIG. An example of the processing procedure in the second method of modeling described above will be described with reference to FIG. An example of the processing procedure in the third method of modeling described above will be described with reference to FIG.

FIG. 11 is a flowchart showing a processing procedure example 1 performed by the information processing apparatus 100. The information processing device 100 acquires the measurement point cloud information 101 (step S1101). Next, the information processing apparatus 100 specifies each reference plane (step S1102). The information processing device 100 specifies at least one reference surface of, for example, a floor surface, a ceiling surface, a wall surface, and the like. The reference plane may be a vertical plane or a horizontal plane that can be specified from the measurement point cloud information 101 and is designated by the user.

The information processing device 100 extracts the first measurement point cloud information 521 corresponding to each reference plane (step S1103). The information processing apparatus 100 extracts the second measurement point cloud information 522 existing in the set direction for each reference plane (step S1104). The information processing apparatus 100 generates first model information 121 indicating the first model 111 based on the first measurement point cloud information 521 (step S1105). The information processing apparatus 100 generates the second model information 122 indicating the second model 112 based on the second measurement point cloud information 522 (step S1106). In step S1105 and step S1106, the information processing apparatus 100 generates the first model information 121 and the second model information 122, but the present invention is not limited to this. For example, when a device other than the information processing device 100 executes CAD, the information processing device 100 transfers the first measurement point group information 521 and the second measurement point group information 522 to the other device via the network 410. Then, the first model information 121 and the second model information 122 are acquired from other devices.

The information processing apparatus 100 generates third model information 123 indicating the third model 113 including the first model 111 and the second model 112 based on the first model information 121 and the second model information 122 (step). S1107), a series of processes is completed.

FIG. 12 is a flowchart showing a processing procedure example 2 performed by the information processing apparatus 100. The information processing device 100 acquires the measurement point cloud information 101 (step S1201). Next, the information processing apparatus 100 specifies each reference plane (step S1202). Then, the information processing apparatus 100 extracts the first measurement point cloud information 521 corresponding to each reference plane (step S1203).

Based on the first measurement point group information 521, the information processing device 100 identifies a missing portion 800 of the reference plane where there is no measurement point specified by the first measurement point group information 521 (step S1204). The information processing device 100 generates the measurement point cloud information corresponding to the specified missing portion 800 based on the first measurement point cloud information 521 (step S1205). Then, the information processing apparatus 100 adds the measurement point cloud information corresponding to the generated missing portion 800 to the measurement point cloud information 101 acquired in step S1201 (step S1206).

The information processing device 100 generates model information indicating a model based on the added measurement point cloud information 101 (step S1207), and ends a series of processes.

FIG. 13 is a flowchart showing a processing procedure example 3 performed by the information processing apparatus 100. The information processing device 100 acquires the measurement point cloud information 101 (step S1301). Next, the information processing apparatus 100 specifies each reference plane (step S1302). The information processing device 100 extracts the first measurement point cloud information 521 corresponding to each reference plane (step S1303). The information processing apparatus 100 extracts the second measurement point cloud information 522 existing in the direction set for each reference plane with respect to each reference plane (step S1304). In step S1304, the information processing apparatus 100 selects the measurement points included in the first measurement point cloud information 521 among the measurement points existing in the directions set for each reference surface with respect to each reference surface. , Is not extracted as the second measurement point cloud information 522.

Next, the information processing apparatus 100 identifies a missing portion 800 of the reference plane that does not have a measurement point specified by the first measurement point group information 521 based on the first measurement point group information 521 (step S1305). The information processing apparatus 100 generates the measurement point cloud information corresponding to the specified missing portion 800 based on the first measurement point cloud information 521 (step S1306). Then, the information processing apparatus 100 adds the generated measurement point cloud information 101 to the first measurement point cloud information 521 (step S1307). The information processing device 100 generates first model information 121 indicating the first model 111 based on the added first measurement point cloud information 521 (third measurement point cloud information) (step S1308). The information processing apparatus 100 generates the second model information 122 indicating the second model 112 based on the second measurement point cloud information 522 (step S1309). The information processing apparatus 100 generates third model information 123 indicating the third model 113 including the first model 111 and the second model 112 based on the first model information 121 and the second model information 122 (step). S1310), the series of processes is completed.

Next, a comparative example of the case where the measurement point cloud information 101 is modeled as it is and the case where the measurement point cloud information 101 is separated and modeled based on the reference plane will be described with reference to FIGS. 14 and 15. ..

FIG. 14 is an explanatory diagram showing an example in which the measurement point cloud information 101 is modeled as it is. FIG. 15 is an explanatory diagram showing an example in which the measurement point cloud information 101 is separated and modeled. 14 (1) and 15 (1) show the space to be measured. In the space to be measured, there is a desk in the room. Hiding behind the desk, part of the floor becomes an occlusion part.

Therefore, as shown in FIG. 14 (2), when the entire measurement point cloud information 101 is modeled as it is, the desk and the floor are integrated in the occlusion portion in the model.

As shown in FIG. 15 (2), the information processing apparatus 100 has measurement point cloud information corresponding to a reference surface such as a floor surface, a wall surface, and a ceiling surface, and measurement point cloud information corresponding to an area surrounded by the wall surface. Of these, the measurement point cloud information, which is above the floor surface and below the ceiling surface, is modeled. As a result, modeling is performed based on the measurement point cloud information corresponding to the desk, so that a model that more faithfully represents the actual shape of the desk included in the measurement target can be obtained. In addition, since the model is modeled on a reference surface such as a floor surface, a wall surface, or a ceiling surface, a model that more faithfully represents the actual shape of the floor included in the measurement target can be obtained. Then, as shown in FIG. 15 (3), the information processing apparatus 100 generates model information including the two modeled models. As a result, in the model shown in FIG. 15, the desk and the floor are not integrated in the occlusion portion.

In this way, the information processing apparatus 100 can create a model that more faithfully represents the actual space to be measured, as compared with the case where the measurement point cloud information 101 is modeled as it is.

FIG. 16 is an explanatory diagram showing a system example. In the present embodiment, the case where the information processing device 100 is the measuring device itself has been described as an example, but the present invention is not limited to this. A part of the functional parts of the information processing apparatus 100 shown in FIG. 5 may be realized by another device.

The system 1600 includes an information processing device 100, which is a measuring device, a device 1601, and a device 1602. The information processing device 100, the device 1601, and the device 1602 are connected via, for example, a network 410. In the example of FIG. 16, the device 1601 is a PC, and the device 1602 is a portable terminal device, but the present invention is not limited to this. Examples of the device 1601 and the device 1602 include a PC, a server, and a portable terminal device. For example, the acquisition unit 501 may be realized by the information processing device 100 which is a measuring device, and the control unit 502 and the storage unit 510 may be realized by the device 1601. Further, a part of the functions of the control unit 502 may be realized by the information processing device 100, and the remaining functions of the control unit 502 may be realized by the device 1601.

Further, for example, the device 1602 may notify the information processing device 100, which is a measuring device, of the measurement start instruction received by the operation input by the user. As a result, the user can perform 3D measurement without directly operating the measuring device.

The device 1601 and the device 1602 may have a hardware configuration such as a CPU, a disk, a memory, an I / F, an input device, and an output device such as a display, and are not particularly limited. Further, various measurement point cloud information and various model information may be stored and managed in a database server (not shown) or the like.

As described above, the information processing apparatus 100 uses the measurement point cloud information obtained by 3D measurement as the first measurement point cloud information on the reference plane and the second measurement point cloud in a specific direction with respect to the reference plane. It is divided into information, and model information is generated by combining each model modeled for each measurement point cloud information. As a result, the information processing apparatus 100 can generate a model that more faithfully represents the shape of the occlusion portion of the reference plane without the trouble of the measurer such as performing the measurement at a plurality of measurement points. Therefore, the labor of the measurer can be saved. In particular, the information processing device 100 is useful for spaces such as floors, ceilings, and rooms with walls. For example, if the reference plane is the floor, a model can be obtained that more faithfully represents the occlusion portion of the floor. Therefore, for example, a three-dimensional model that more faithfully represents a room can be generated without any hassle.

By the way, if the missing portion where the measurement point does not exist is small in the reference plane, a model showing the shape of the missing portion by the measurement point close to the missing portion in the measurement point group existing on the reference plane can be obtained. However, especially when the missing part is large, it is not clear what kind of measurement point is used for the missing part when modeling. Therefore, the information processing apparatus 100 adds the measurement point cloud information corresponding to the missing portion of the reference plane where there is no measurement point to the measurement point cloud information according to the reference plane, and models the model based on the added measurement point cloud information. To become. As a result, the information processing apparatus 100 can generate a model that more faithfully represents the occlusion portion even when the missing portion is large. For example, when the reference plane is the floor surface, measurement points corresponding to the occlusion portion of the floor surface are added, so that a model that more faithfully represents the occlusion portion of the floor surface can be generated.

Further, the information processing apparatus 100 estimates a reference plane which is at least one of a floor surface and a ceiling surface based on the distribution of the number of measurement points in the height direction obtained from the acquired measurement point cloud information. As a result, when the space to be measured is in a room, a model that more faithfully represents the occlusion portion of the floor surface and the occlusion portion of the ceiling surface can be obtained. Therefore, the information processing apparatus 100 can generate, for example, a model that more faithfully represents the room to be measured without any trouble.

Further, the information processing apparatus 100 specifies a vertical surface that coincides between each vertical surface with respect to the floor surface and each vertical surface with respect to the ceiling surface as a wall surface. As a result, when the space to be measured is in a room, a model that more faithfully represents the occlusion portion of the wall surface can be obtained. Therefore, the information processing apparatus 100 can generate, for example, a model that more faithfully represents the room without any hassle.

When there are a plurality of reference planes, the information processing apparatus 100 exists in the first measurement point cloud information corresponding to each of the plurality of reference planes and in a specific direction set for each of the plurality of reference planes. The second measurement point cloud information other than the one measurement point cloud information is extracted. In this way, when there are a plurality of reference planes, the information processing apparatus 100 models each of the plurality of reference planes and the portion other than the reference plane to more accurately model the portion other than the reference plane. Can be transformed into. For example, when a room having a wall, a floor, and a ceiling and a desk installed on the floor is used as the space to be measured, the wall, the floor, the ceiling, and the desk are modeled separately. Therefore, the information processing device 100 can create a model that looks more like a desk because the model corresponding to the desk is not integrated with the wall, floor, ceiling, and the like.

Further, when the information processing apparatus 100 has a set of a plurality of distributions in the three-dimensional distribution of the measurement point cloud specified by the second measurement point cloud information other than the first measurement point cloud information corresponding to the reference plane, Model by set. For example, when there are a plurality of objects on the floor such as a desk, a shelf, and a trash can, there is a risk that the plurality of objects will be integrated when modeling. Therefore, the information processing apparatus 100 can be modeled for each object by modeling for each set of distributions. Therefore, the information processing device 100 can suppress the integration of a plurality of objects and can create a model that looks more like an object.

The information processing device 100 adds the measurement point cloud information corresponding to the missing portion of the reference plane where there is no measurement point to the measurement point cloud information acquired by 3D measurement, and models based on the added measurement point cloud information. Generate the converted model information. As a result, the information processing apparatus 100 can generate a model that more faithfully represents the shape of the occlusion portion of the reference plane without the trouble of the measurer such as performing the measurement at a plurality of measurement points. Therefore, the labor of the measurer can be saved. In particular, the information processing device 100 is useful for spaces such as floors, ceilings, and rooms with walls. For example, when the reference surface is the floor surface, a model that faithfully represents the shape of the occlusion portion of the floor surface can be obtained. Therefore, for example, a three-dimensional model that more faithfully represents a room can be generated without any hassle.

The information processing method described in the present embodiment can be realized by executing an information processing program prepared in advance on a computer such as a personal computer or a workstation. This information processing program is recorded on a computer-readable recording medium such as a magnetic disk, an optical disk, or a USB (Universal Serial Bus) flash memory, and is executed by being read from the recording medium by the computer. Further, the information processing program may be distributed via a network such as the Internet.

Further, the information processing apparatus 100 described in the present embodiment is an IC for a specific purpose (hereinafter, simply referred to as “ASIC”) such as a standard cell or a structured ASIC (Application Specific Integrated Circuit), or an FPGA (Field Program Metal Array). ), Etc., can also be realized by PLD (Programmable ASIC Device). Specifically, for example, the information processing device 100 is manufactured by defining the function of the above-mentioned information processing device by a HDL (Hardware Description Language) description, logically synthesizing the HDL description, and giving it to an ASIC or PLD. be able to.

The following additional notes are further disclosed with respect to the above-described embodiment.

(Appendix 1) To the computer
Acquire three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device.
From the acquired measurement point cloud information, the first measurement point cloud information existing on the reference plane measured by the measuring device and the second measurement point cloud information existing in a specific direction with respect to the reference plane. And, extract,
The first model information indicating the first model based on the extracted first measurement point cloud information is acquired, and the first model information is acquired.
The second model information indicating the second model based on the extracted second measurement point cloud information is acquired, and the second model information is acquired.
Based on the acquired first model information and the acquired second model information, a third model information indicating a third model including the first model and the second model is generated.
An information processing program characterized by executing processing.

(Appendix 2) To the computer
Based on the first measurement point cloud information, a portion of the reference plane where the density of each measurement point specified by the first measurement point cloud information is equal to or less than a predetermined density is specified.
A third measurement point group information including the first measurement point group information and the measurement point group information corresponding to the specified portion is generated.
Let the process be executed
The process of acquiring the first model information is
Instead of acquiring the first model information indicating the first model based on the first measurement point cloud information, the first model information indicating the first model based on the third measurement point cloud information is acquired.
The information processing program according to Appendix 1, wherein the information processing program is characterized by the above.

(Appendix 3) To the computer
Based on the distribution in the height direction of the number of each measurement point specified by the acquired measurement point cloud group information, the reference surface which is at least one of the floor surface and the ceiling surface is specified.
Let the process be executed
The process of extracting the first measurement point cloud information and the second measurement point cloud information is
From the acquired measurement point cloud information, the first measurement point cloud information corresponding to the specified reference plane, the second measurement point cloud information existing in the specific direction with respect to the reference plane, and the second measurement point cloud information. The information processing program according to Appendix 1 or 2, characterized in that

(Appendix 4) To the computer
Based on the distribution of the number of measurement points in the height direction obtained from the acquired measurement point group information, among the measurement point group information, the fourth measurement point group information according to the floor surface and the measurement point group information. Extract the 5th measurement point cloud information according to the ceiling surface,
Between each vertical plane with respect to the floor surface based on the extracted fourth measurement point cloud group information and each vertical plane with respect to the ceiling surface based on the extracted fifth measurement point cloud group information, other than the height direction. Identify the reference plane, which is a vertical plane with matching positions in each direction.
Let the process be executed
The process of extracting the first measurement point cloud information and the second measurement point cloud information is
From the acquired measurement point cloud information, the first measurement point cloud information corresponding to the specified reference plane, the second measurement point cloud information existing in the specific direction with respect to the reference plane, and the second measurement point cloud information. The information processing program according to any one of Supplementary notes 1 to 3, wherein the information processing program is characterized by extracting information processing.

(Appendix 5) The extraction process is
From the measurement point cloud information, the first measurement point cloud information corresponding to each of the plurality of reference planes measured by the measuring device and the reference plane of each of the plurality of reference planes measured by the measuring device. Extracts the second measurement point cloud information other than the first measurement point cloud information, which exists in the specific direction set for.
The information processing program according to any one of Supplementary notes 1 to 4, wherein the information processing program is characterized by the above.

(Appendix 6) To the computer
Acquire three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device.
From the acquired measurement point cloud information, the first measurement point cloud information existing on the reference plane measured by the measuring device is extracted.
Based on the extracted first measurement point cloud information, a missing portion of the reference plane in which the density of each measurement point specified by the first measurement point cloud information is equal to or less than a predetermined density is specified.
A third measurement point group information including the measurement point group information and the measurement point group information corresponding to the specified missing portion is generated.
Acquire model information indicating a model based on the generated third measurement point cloud information.
An information processing program characterized by executing processing.

(Appendix 7) The computer
Acquire three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device.
From the acquired measurement point cloud information, the first measurement point cloud information existing on the reference plane measured by the measuring device and the second measurement point cloud information existing in a specific direction with respect to the reference plane. And, extract,
The first model information indicating the first model based on the extracted first measurement point cloud information is acquired, and the first model information is acquired.
The second model information indicating the second model based on the extracted second measurement point cloud information is acquired, and the second model information is acquired.
Based on the acquired first model information and the acquired second model information, a third model information indicating a third model including the first model and the second model is generated.
An information processing method characterized by executing processing.

(Appendix 8) The computer
The measuring device acquires three-dimensional measurement point cloud information based on the distance from the measuring device to the object.
From the acquired measurement point cloud information, the first measurement point cloud information existing on the reference plane measured by the measuring device is extracted.
Based on the extracted first measurement point cloud information, a missing portion of the reference plane in which the density of each measurement point specified by the first measurement point cloud information is equal to or less than a predetermined density is specified.
A third measurement point group information including the measurement point group information and the measurement point group information corresponding to the specified missing portion is generated.
Acquire model information indicating a model based on the generated third measurement point cloud information.
An information processing method characterized by executing processing.

(Appendix 9) Obtain three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device.
From the acquired measurement point cloud information, the first measurement point cloud information existing on the reference plane measured by the measuring device and the second measurement point cloud information existing in a specific direction with respect to the reference plane. And, extract,
The first model information indicating the first model based on the extracted first measurement point cloud information is acquired, and the first model information is acquired.
The second model information indicating the second model based on the extracted second measurement point cloud information is acquired, and the second model information is acquired.
Based on the acquired first model information and the acquired second model information, a third model information indicating a third model including the first model and the second model is generated.
An information processing device characterized by having a control unit.

(Appendix 10) Three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device is acquired.
From the acquired measurement point cloud information, the first measurement point cloud information existing on the reference plane measured by the measuring device is extracted.
Based on the extracted first measurement point cloud information, a missing portion of the reference plane in which the density of each measurement point specified by the first measurement point cloud information is equal to or less than a predetermined density is specified.
A third measurement point group information including the measurement point group information and the measurement point group information corresponding to the specified missing portion is generated.
Acquire model information indicating a model based on the generated third measurement point cloud information.
An information processing device characterized by having a control unit.

100 Information processing device 101 Measurement point cloud information 111 1st model 112 2nd model 113 3rd model 121 1st model information 122 2nd model information 123 3rd model information 301 Drive device 302 Measuring device 303 Control device 501 Acquisition unit 502 Control Part 510 Storage part 511 1st specific part 512 2nd specific part 513 Extraction part 514 3rd specific part 515 Addition part 516 1st acquisition part 517 2nd acquisition part 518 Generation part 521 1st measurement point cloud information 522 2nd measurement point Group information 600 distribution

Claims (5)

On the computer
Gets the three-dimensional measurement point group information based on the distance to the object body from the measured by the measuring device the measuring device,
Based on the distribution in the height direction of the number of each measurement point specified by the acquired measurement point cloud group information, a reference surface which is at least one of a floor surface and a ceiling surface is specified.
From the acquired measurement point cloud information, the first measurement point cloud information corresponding to the specified reference plane and the second measurement point cloud information existing in a specific direction with respect to the reference plane are extracted. ,
The first model information indicating the first model based on the extracted first measurement point cloud information is acquired, and the first model information is acquired.
The second model information indicating the second model based on the extracted second measurement point cloud information is acquired, and the second model information is acquired.
Based on the acquired first model information and the acquired second model information, a third model information indicating a third model including the first model and the second model is generated.
An information processing program characterized by executing processing. On the computer
Based on the first measurement point cloud information, a portion of the reference plane where the density of each measurement point specified by the first measurement point cloud information is equal to or less than a predetermined density is specified.
A third measurement point group information including the first measurement point group information and the measurement point group information corresponding to the specified portion is generated.
Let the process be executed
The process of acquiring the first model information is
Instead of acquiring the first model information indicating the first model based on the first measurement point cloud information, the first model information indicating the first model based on the third measurement point cloud information is acquired.
The information processing program according to claim 1. On the computer
Based on the distribution of the number of measurement points in the height direction obtained from the acquired measurement point group information, among the measurement point group information, the fourth measurement point group information according to the floor surface and the measurement point group information. Extract the 5th measurement point cloud information according to the ceiling surface,
Between each vertical plane with respect to the floor surface based on the extracted fourth measurement point cloud group information and each vertical plane with respect to the ceiling surface based on the extracted fifth measurement point cloud group information, other than the height direction. Identify the reference plane, which is a vertical plane with matching positions in each direction.
Let the process be executed
The process of extracting the first measurement point cloud information and the second measurement point cloud information is
From the acquired measurement point cloud information, the first measurement point cloud information corresponding to the specified reference plane, the second measurement point cloud information existing in the specific direction with respect to the reference plane, and the second measurement point cloud information. The information processing program according to claim 1 or 2, wherein the information processing program is characterized in that. The computer
Acquire three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device.
Based on the distribution in the height direction of the number of each measurement point specified by the acquired measurement point cloud group information, a reference surface which is at least one of a floor surface and a ceiling surface is specified.
From the acquired measurement point cloud information, the first measurement point cloud information corresponding to the specified reference plane and the second measurement point cloud information existing in a specific direction with respect to the reference plane are extracted. ,
The first model information indicating the first model based on the extracted first measurement point cloud information is acquired, and the first model information is acquired.
The second model information indicating the second model based on the extracted second measurement point cloud information is acquired, and the second model information is acquired.
Based on the acquired first model information and the acquired second model information, a third model information indicating a third model including the first model and the second model is generated.
An information processing method characterized by executing processing. Acquire three-dimensional measurement point cloud information based on the distance from the measuring device to the object measured by the measuring device.
Based on the distribution in the height direction of the number of each measurement point specified by the acquired measurement point cloud group information, a reference surface which is at least one of a floor surface and a ceiling surface is specified.
From the acquired measurement point cloud information, the first measurement point cloud information corresponding to the specified reference plane and the second measurement point cloud information existing in a specific direction with respect to the reference plane are extracted. ,
The first model information indicating the first model based on the extracted first measurement point cloud information is acquired, and the first model information is acquired.
The second model information indicating the second model based on the extracted second measurement point cloud information is acquired, and the second model information is acquired.
Based on the acquired first model information and the acquired second model information, a third model information indicating a third model including the first model and the second model is generated.
An information processing device characterized by having a control unit.

Images