JP2018106643A - Spatial model processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate correction of a spatial model generated by position alignment of a plurality of local regions with a three-dimensionally measured point group.SOLUTION: Spatial model storage means 20 stores a spatial model obtained by integrating, for each of a plurality of local regions in a space, a local spatial model formed with a dot group at a local coordinate system of the local region with a common coordinate system common to the plurality of local regions. Reference plane estimation means 30 detects a portion corresponding to an object plane where an area in the spatial model is largest, and estimates a plane along the aforementioned portion as a reference plane. Input means 41 causes a user to input a line segment expressing a position, on the reference plane, of the object plane orthogonal to the reference plane. Spatial model correction means 32 sets a target surface including the aforementioned line segment and orthogonal to the estimated reference plane, extracts, from the spatial model, a focused portion corresponding to the target surface and having a position and a shape, and moves the local spatial model in a manner such that the focused portion matches the target surface at the common coordinate system.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a spatial model processing apparatus that corrects a spatial model, and more particularly to a spatial model processing apparatus that corrects a spatial model generated by aligning a point group obtained by three-dimensionally measuring a plurality of local regions.

  When generating a three-dimensional model (spatial model) of a wide space, three-dimensional measurement is performed for each local area with overlapping each other while moving in the space, and multiple local areas obtained by measurement of each local area are measured. Point cloud (point cloud) was aligned and connected.

  In other words, using the shape represented by the point cloud of the overlap part as a clue, the multiple local areas are three-dimensionally measured in each local coordinate system, and then converted into a common coordinate system to align them. To generate a spatial model.

JP 2006-170031 A

  However, the local region may include a plain surface such as a plain wall that is dominant and has few features in two or three dimensions. In addition, the local area may include a local area that includes an adjacent room that is three-dimensionally measured through the door and has few features related to the adjacent room. For this reason, an alignment error is likely to occur in a local region with few features, and a local coordinate transformation shift occurs in a spatial model in which a plurality of local regions are connected. Even if the displacement for each local region is very small, the displacement accumulates as the number of local regions to be connected increases, and there may be a large displacement in the entire spatial model. For example, a wall that is originally a flat surface may be a bent wall, or a plurality of flat surfaces may be layered walls with gaps.

  Such a shift is caused by a small number of features in the local region. Therefore, in order to drastically reduce the information, it is necessary to add information. For example, the user moves the local coordinate system of the local area in the spatial model (rotates or translates), or the user sets a marker in the spatial model to add a new feature. Without it, it cannot be drastically reduced.

  However, the work for correcting the data of the three-dimensional coordinate system by the user requires a great deal of labor and skill. For example, for a local area where walls are dominant, work is done on a large plane with few clues, and for a local area including the next room through the door, it is a sensitive work to be performed on the part of the next room with a narrow measurement range. Therefore, a lot of trial and error and experience are required.

  The present invention has been made in view of the above problems, and an object thereof is to provide a spatial model processing apparatus capable of correcting a spatial model of a three-dimensional coordinate system without requiring a great deal of labor and skill.

  (1) The space model processing device according to the present invention is a space composed of point groups obtained by three-dimensional measurement in a space where an object plane serving as a reference plane and an object plane perpendicular to the reference plane exist. An apparatus for correcting a model, wherein a common coordinate system is used for each of a plurality of local regions in the space, the local space model including the point cloud in the local coordinate system of the local region being shared by the plurality of local regions And a space model storage means for storing the space model integrated in step (b), a portion corresponding to the object plane having the largest area in the space model is detected, and a plane along the portion is estimated as the reference plane Reference plane estimation means, input means for allowing a user to input a line segment representing the position of the object plane perpendicular to the reference plane on the reference plane, and the reference plane including the line segment A target plane perpendicular to the estimation reference plane estimated by the estimation unit is set, a target portion having a position and a shape corresponding to the target surface is extracted from the spatial model, and the target portion on the target plane in the common coordinate system And a spatial model correction unit that moves the local spatial model so as to match.

  (2) The spatial model processing device according to (1) above generates a density image in which pixel values are determined according to the density of the point group in a projection image obtained by projecting the spatial model onto the estimation reference plane. The image processing apparatus may further include a generation unit and a display unit that displays the density image, and the input unit may cause the user to input the line segment on the density image.

  (3) In the spatial model processing device according to (2), the density image generation unit detects two planes perpendicular to the estimation reference plane and orthogonal to each other from the spatial model, and each of the two planes. The direction of the line segment that is the projection image on the estimated reference plane is set to the horizontal direction and the vertical direction of the density image.

  According to the spatial model processing apparatus of the present invention, it is possible to effectively correct a spatial model of a three-dimensional coordinate system with little effort and skill.

It is a block diagram which shows the structure of the outline of the space model processing apparatus which concerns on embodiment of this invention. It is a functional block diagram of the outline of the space model processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the point group which is a space model before correction | amendment processing. It is a schematic diagram explaining the coordinate transformation of the space model using the maximum plane and two planes perpendicular to this and perpendicular to each other. FIG. 4 is a schematic diagram illustrating an example of a density image generated from the space model illustrated in FIG. 3 and a line segment group input on the density image. It is a schematic diagram which shows the example of the target surface group calculated from the line segment group illustrated in FIG. FIG. 7 is a schematic diagram illustrating a spatial model obtained by correcting the spatial model illustrated in FIG. 3 based on the target plane group illustrated in FIG. 6. It is a schematic flowchart of operation | movement of a space model processing apparatus. It is a schematic flowchart of operation | movement of a space model processing apparatus.

  Hereinafter, a spatial model processing apparatus 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The space model processing device 1 corrects a space model composed of point groups obtained by three-dimensional measurement for a space in which an object plane parallel to a reference plane and a perpendicular object plane exist.

  For example, three-dimensional measurement can be performed using a three-dimensional measurement device such as an RGBD camera. The RGBD camera is a TOF (Time Of Flight) type distance sensor that measures the distance from the time it takes for the projected laser light to reciprocate with the object surface, and a color that captures an RGB image of the same field of view in synchronization with the distance sensor. It is a device that combines cameras. Here, the distance sensor scans the laser beam two-dimensionally and measures the directions and distances of a plurality of reflection points within the scanning range. Based on the direction and distance, the three-dimensional coordinates of the point group including the reflection points are determined in a local coordinate system based on the position and orientation of the three-dimensional measuring apparatus. Note that the three-dimensional measurement apparatus may be a combination of a distance sensor that acquires a distance image and a monochrome camera that acquires a grayscale image. Further, the distance sensor is not limited to the TOF method, and may employ another method such as a pattern light projection method.

  For example, on one floor of a building, a horizontal plane is used as a reference plane, and floors and ceilings exist as object planes parallel to the horizontal plane, while walls and the like exist as vertical object planes between a plurality of rooms and sections. As described above, the three-dimensional measurement of the entire space in which a plurality of rooms, sections, and the like are arranged is divided into the three-dimensional measurement in a plurality of local regions that are a part of the space. Specifically, a three-dimensional measuring device such as an RGBD camera is installed in each local area and measured. Therefore, a point group obtained by three-dimensionally measuring an object in each local region is represented by three-dimensional coordinate data in a coordinate system (local coordinate system) set for each local region. Then, a local space model composed of point groups obtained in each local region is integrated in a common coordinate system common to a plurality of local regions to generate an overall space model. The spatial model processing apparatus 1 according to the present invention performs a process of correcting a positional relationship shift between a plurality of local spatial models constituting a spatial model.

  FIG. 1 is a block diagram showing a schematic configuration of the space model processing apparatus 1. The spatial model processing device 1 includes a storage unit 2, a control unit 3, and a user interface unit 4.

  The storage unit 2 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 2 stores various programs and various data, and inputs / outputs such information to / from the control unit 3.

  The control unit 3 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The control unit 3 reads out and executes the program from the storage unit 2 to function as each unit described later. In addition, the control unit 3 stores various data in the storage unit 2 and reads the various data from the storage unit 2. The control unit 3 is also connected to the user interface unit 4 to output and display an image or the like on the user interface unit 4 and acquire a coordinate value or the like input by the user using the user interface unit 4.

  The user interface unit 4 is a device that displays an image or the like input from the control unit 3 to the user, and outputs a coordinate value or the like input by operating by the user to the control unit 3. Specifically, the user interface unit 4 includes a display device such as a liquid crystal display or CRT (Cathode Ray Tube), and a pointing device such as a mouse or an operation input device such as a keyboard.

  FIG. 2 is a schematic functional block diagram of the space model processing apparatus 1. The storage unit 2 functions as the space model storage unit 20 or the like. The control unit 3 also functions as a reference plane estimation unit 30, a density image generation unit 31, a spatial model correction unit 32, and the like. The user interface unit 4 functions as display means 40 and input means 41.

  The spatial model storage unit 20 stores a three-dimensional point group (local space model) for each of a plurality of local regions and a spatial model obtained by integrating these three-dimensional point groups. Here, as described above, three-dimensional measurement is performed corresponding to each local region, but the specific range of each local region is substantially determined by the spatial measurement range at each measurement position of the three-dimensional measurement device. Determined. Further, as already described, the three-dimensional point group obtained by measuring each local region is represented by a local coordinate system for each local region. On the other hand, a spatial model in which three-dimensional point groups of all local regions are integrated is represented by a common coordinate system that is commonly used in all local regions.

  When integrating each local space model into a common coordinate system, by rotating and translating the local coordinate system in the common coordinate system (by moving it) so as to obtain consistency in the overlapping part of adjacent local regions ) The local space model is aligned. Specifically, the difference (error) in the overlapping portion between the surface shapes represented by the three-dimensional point groups of the adjacent local regions is minimized. Further, in addition to the difference in the surface shape represented by the three-dimensional point group in the overlapping portion, the alignment may be performed so as to minimize the difference in the RGB value or the difference in the gray value of the three-dimensional point group. The spatial model generated by such alignment is stored in advance in the storage unit 2 as target data for correction processing by the spatial model correction unit 32. The storage unit 2 can store the spatial model after the correction process.

  FIG. 3 is a diagram illustrating an example of a point group that is a space model before correction processing, and the space model includes two rooms. The space model 100 shown in FIG. 3A corresponds to a perspective view of the room viewed obliquely from above, and the space model 110 shown in FIG. 3B shows the point cloud shown in FIG. 3A viewed from directly above the room. Hit the top view. In this space model, two local space models obtained by three-dimensional measurement in two rooms are aligned and integrated.

  In FIG.3 (b), two rooms are located in the center part and the left side. The three-dimensional point group 111 and the three-dimensional point group 113 are point groups measured in a local region corresponding to the left room, and the three-dimensional point group 112 and the three-dimensional point group 114 are local corresponding to the central room. It is a point cloud measured in a region. The three-dimensional point group 111 and the three-dimensional point group 112 correspond to the same wall separating the two rooms, and should originally form one plane, but because of the alignment error due to the small number of wall features, There is a gap between the two point groups 111 and 112. The three-dimensional point group 113 and the three-dimensional point group 114 are originally walls arranged in a straight line, but the feature for connecting the rooms can be obtained only by measurement through the door, and the feature of the connecting part between the rooms is An alignment error occurs due to a small amount of the error, and the error spills over and the point groups 113 and 114 are not aligned on a straight line.

  The reference plane estimation unit 30 reads a space model from the space model storage unit 20, detects a portion corresponding to the object plane having the largest area from the read space model, and estimates a plane along the portion as a reference plane. The reference plane estimation unit 30 then outputs the detected reference plane information to the density image generation unit 31 and the space model correction unit 32. Specifically, for example, the reference plane estimation unit 30 detects a plurality of planes by analyzing the point group constituting the space model, determines a plane having the maximum area from the detected plurality of planes, The normal vector in the common coordinate system of the determined plane and the distance from the origin are calculated. The orientation of the reference plane is determined by the normal vector. Further, the distance between the origin of the common coordinate system and the reference plane can be arbitrarily set, and can be set to the same distance as the maximum plane, for example.

  Note that the reference plane estimation unit 30 may extract the point group as a portion corresponding to the object plane having the largest area from the space model and then determine the reference plane from the point group. In this case, the area of the plane corresponding to the point group can be determined based on the distribution range of the point group. In addition, when downsampling processing or the like is performed so that the density of the point group is constant on the object surface, the area of the plane can be determined based on the number of constituent points of the point group.

  Here, typically, the floor is detected as the largest object plane from the spatial model that measures the indoor space, and the ground is detected as the largest object plane from the spatial model that measures the outdoor space. A horizontal plane in the space model is estimated as a plane.

  The density image generation unit 31 reads the space model from the space model storage unit 20 and projects a point group constituting the read space model onto the reference plane (estimated reference plane) input from the reference plane estimation unit 30. Then, the density image generation unit 31 generates a density image in which pixel values are determined according to the density of the point group in the projection image. For example, the density image generating unit 31 generates a density image in which the luminance value is set according to the density of the point group.

  Specifically, first, the density image generating unit 31 projects all points constituting the spatial model onto the estimation reference plane and equally divides the projection area of the estimation reference plane into blocks corresponding to the pixels of the density image. The projection points are counted for each block. Next, the density image generation means 31 generates a density image by setting a higher luminance value for the pixel corresponding to the block as the number of projection points of the block increases. For example, it is possible to set a luminance value that is proportional to the number of projection points of a block corresponding to each pixel by setting the luminance value of 0 projection points to 0 and the luminance value of the maximum number of projection points in all blocks to 255. In this case, the luminance value of the pixel corresponding to the position of the object plane perpendicular to the estimation reference plane is high luminance, and typically a linear region corresponding to the wall is particularly high luminance. Therefore, the density image is an image in which the user can easily recognize the position of the wall.

  The brightness value may be reversed. That is, the density image may be generated by setting a lower luminance value to the pixel corresponding to the block as the number of projection points of the block is larger. In that case, typically, a linear region corresponding to the wall has particularly low luminance, and the density image is an image in which the user can easily recognize the position of the wall.

  Further, the density image generating means 31 detects two planes that are perpendicular to the estimation reference plane and orthogonal to each other from the spatial model, and the direction of the line segment that is a projection image of each of the two planes on the estimation reference plane is determined as the density image. Are set in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction). At this time, as the two planes, it is preferable to preferentially detect a pair having a large area among a pair of planes that satisfy the condition of being perpendicular to the estimation reference plane and orthogonal to each other. For example, the density image generating means 31 can detect the pair having the largest area or the pair having the largest total area among the plane pairs. In many cases, the angles of walls intersecting each other, such as a room, are right angles. Thus, by setting the coordinate axis of the density image in this way, an input line segment by the user described later is in the X-axis direction or Y-axis direction of the density image. Therefore, it becomes easy to obtain an easy and accurate input by the user.

  Setting the X axis and the Y axis along the estimation reference plane corresponds to defining the orthogonal coordinate system XYZ for the spatial model with the normal direction of the estimation reference plane as the Z axis. That is, the coordinate conversion of the space model is performed using the maximum plane detected by the reference plane estimation unit 30 and the density image generation unit 31 and two planes perpendicular to and perpendicular to each other. FIG. 4 is a schematic diagram for explaining the coordinate transformation. The space model 200 is a space model stored in advance in the space model storage unit 20, and is the same as the space model 100 illustrated in FIG. The coordinate system of the space model 200 can be set arbitrarily. For example, two orthogonal coordinate axes of the imaging surface of an RGBD camera installed in a certain local area become the X axis and the Y axis, and the Z axis can be set in the optical axis direction of the camera. The reference plane estimation means 30 detects, for example, a point group 210 corresponding to the floor surface as a portion corresponding to the maximum plane from the space model 200, and estimates a horizontal plane along this as a reference plane. In addition, for example, the density image generation unit 31 corresponds to a wall having the second largest area after the floor surface which is the largest plane among the walls erected on the floor surface in the space model 200 as a surface perpendicular to the reference plane. A point group 220 and a point group 230 corresponding to another wall orthogonal to the wall are detected. The normal direction of the estimated reference plane is the Z axis, the direction of the line segment obtained by projecting the plane defined by the point group 220 onto the estimated reference plane is the Y axis, and the plane defined by the point group 230 is estimated. An orthogonal coordinate system XYZ having the X-axis as the direction of the line segment obtained by projection onto the reference plane is defined, and a space model 240 in which the coordinates of the point group are converted corresponding to this is generated.

  The display unit 40 displays the density image generated by the density image generation unit 31. Specifically, the display device of the user interface unit 4 functions as the display unit 40.

  The input means 41 allows the user to input a line segment representing the position on the reference plane of the object plane perpendicular to the reference plane. Specifically, the input device of the user interface unit 4 functions as the input unit 41. The input means 41 is preferably configured to allow the user to specify the position of the line segment on the density image. Typically, a line that represents a wall that forms a room if it is indoors and a wall that forms a building if it is outdoor are input. Note that the line segment input by the user using the input unit 41 is used by the spatial model correction unit 32 to be described later to designate the position of the target vertical plane when correcting the position of the local space model.

  FIG. 5 is a schematic diagram illustrating a density image 300 (FIG. 5A) generated from the space models 100 and 110 illustrated in FIG. 3 and an example of a line segment group 310 input on the density image 300 (FIG. 5B). FIG.

  The space model correction unit 32 is a plane including the line segment input by the input unit 41 and is perpendicular to the reference plane (estimated reference plane) estimated by the reference plane estimation unit 30 as a target plane in the correction process. Set. Then, the spatial model correction unit 32 extracts a target portion having a position and a shape corresponding to the target surface from the spatial model stored in the spatial model storage unit 20 and locally matches the target portion with the target surface in the common coordinate system. Move operation to rotate and translate the space model. Specifically, the portion of interest extracted by the space model correction unit 32 is a point group distributed in a position and shape corresponding to the target plane, and the space model correction unit 32 has a distribution area of the point group approximately on the target plane. The local space model is moved so as to overlap with each other, thereby generating a corrected space model composed of a point group whose coordinates are converted. The corrected space model is stored in the space model storage means 20. This storage process may be a process of updating the uncorrected space model with the corrected space model.

  FIG. 6 is a schematic diagram illustrating an example of the target surface group 400 calculated from the line group 310 illustrated in FIG. 5, and corresponds to a perspective view of the space in which the target surface group 400 is disposed as viewed obliquely from above.

  FIG. 7 is a schematic diagram showing a space model 500 obtained by correcting the space models 100 and 110 illustrated in FIG. 3 based on the target surface group 400 shown in FIG. 6, and corresponds to a plan view seen from directly above the room. That is, FIG. 3B is a diagram of the spatial model before correction viewed from above, and FIG. 7 is a diagram of the spatial model after correction viewed from above. The wall three-dimensional point groups 501 and 502 shown in FIG. 7 correspond to the wall three-dimensional point groups 111 and 112 illustrated in FIG. FIG. 7 shows that the deviation generated in the three-dimensional point groups 111 and 112 is corrected in the three-dimensional point groups 501 and 502, and the two point groups 501 and 502 are arranged substantially in a straight line. Similarly, the three-dimensional point groups 503 and 504 of the walls corresponding to the three-dimensional point groups 113 and 114 of the wall illustrated in FIG. 3 are also corrected and aligned in a substantially straight line.

  Next, the operation of the space model processing apparatus 1 will be described. FIG. 8 and FIG. 9 are schematic flowcharts of the operation of the space model processing apparatus 1.

  The control unit 3 reads the spatial model from the spatial model storage unit 20 (step S1) and operates as the reference plane estimation unit 30. The reference plane estimation means 30 detects a plane from the spatial shape represented by the three-dimensional point group constituting the space model (step S2). Specifically, the reference plane estimation means 30 defines a normal vector in the common coordinate system and a distance from the origin for each point included in the three-dimensional point group, and the point included in the three-dimensional point group is determined as a normal vector. By clustering the direction and distance from the origin, each point is divided into clusters for each plane. Note that, for example, the normal vector at each point can be defined based on the orientation of the minute surface estimated using points in the vicinity of the point, and the distance from the origin is a plane including the minute estimated surface. Can be defined based on the length of the perpendicular drawn from the origin. In addition, the reference plane estimation unit 30 stores the clustering result in the storage unit 2 for reuse in a later step.

  The reference plane estimation means 30 detects the maximum plane from the planes detected in step S2, and estimates the reference plane (step S3). For example, when the point cloud is down-sampled so as to have a constant density on the object surface, the reference plane estimation unit 30 determines the cluster having the largest number of belonging points among the clusters for each plane. Can be determined. The reference plane estimation means 30 calculates the normal vector (a, b, c) of the maximum plane in the common coordinate system and the distance d from the origin, and derives the plane equation ax + by + cz + d = 0 of the estimated reference plane based on the result. To do.

  The control unit 3 operates as the density image generation unit 31, and the read spatial model is input to the density image generation unit 31. The density image generating means 31 performs coordinate conversion on the three-dimensional point group of the space model with the estimated reference plane as a plane of Z = 0 in the common coordinate system (step S4). Specifically, the density image generating means 31 performs coordinate transformation in the common coordinate system so that the distance from the origin is 0 and the X and Y components of the normal vector are 0 respectively with respect to the estimated reference plane calculated in step S3. An expression is derived, and the coordinates of each point included in the three-dimensional point group are converted by the calculated coordinate conversion expression. Incidentally, the plane equation of the estimated reference plane is cz = 0.

  The density image generating means 31 detects two planes that are substantially perpendicular to the estimation reference plane and are substantially perpendicular to each other from among the spatial shapes represented by the three-dimensional point group constituting the spatial model (step S5). For example, with respect to the above-described cluster for each plane, normal vectors in the common coordinate system are calculated in descending order of area, and an angle formed with the normal vector of the estimation reference plane is calculated. In addition, you may approximate in the order with many number of points instead of making it into the order with a big area. If the calculated angle is 85 degrees to 95 degrees, the plane is regarded as vertical and the plane is selected as a candidate for the first plane. Further, with respect to the above-described cluster for each plane, normal vectors are calculated in descending order of area after the selected first plane candidate, and an angle formed with the selected normal vector of the first plane is calculated. If the calculated angle is 85 degrees to 95 degrees, it is regarded as vertical, and the candidate for the plane and the first plane are determined as the second plane and the first plane, respectively. If the second plane cannot be determined, the first plane candidate cluster selected so far is removed and the first plane candidate is selected again. Note that instead of diverting the clustering result in step S2, the density image generating means 31 may perform the clustering described in the description of step S2 again.

The density image generating unit 31 performs coordinate conversion so that each of the two planes determined in step S5 becomes a plane of X = 0 and a plane of Y = 0 in the common coordinate system XYZ (step S6). Further, the density image generation means 31 obtains the X coordinate minimum value X _MIN and the Y coordinate minimum value Y _MIN of the three-dimensional point group constituting the spatial model for the convenience of density image generation, and the point (X _MIN , Y Coordinate conversion for moving the origin of the common coordinate system to _MIN , 0) is performed (step S7). By moving the origin in this way, the X and Y coordinates of all point groups constituting the space model become non-negative values, and can be adjusted to the definitions of general horizontal coordinates and vertical coordinates in the image of the display device. .

  The density image generation unit 31 projects all three-dimensional point groups constituting the spatial model onto the estimation reference plane Z = 0 (step S8), sets a luminance value according to the density of the projection points, and sets the density. An image is generated (step S9). The density image generation unit 31 outputs the generated density image to the user interface unit 4.

  The user interface unit 4 functions as the display unit 40 and the input unit 41, displays the density image on the display device (step S10), and confirms whether or not the correction completion from the user is input (step S11). The correction process of the spatial model, which will be described later, is executed in a loop process until there is a correction completion in step S11. On the other hand, when the correction completion is input (YES in step S11), the correction process is Is terminated. In step S10, when the loop process has already been started, the line already specified as the line segment for designating the target surface is displayed superimposed on the density image.

  If there is no input indicating that the correction is completed in step S11 (NO in step S11), a series of processes S12 to S18 for correcting the space model are executed. First, the control part 3 receives the line segment input by a user (step S12). Specifically, when a mouse or the like forming the input unit 41 is operated by the user and a line segment is input on the density image, the coordinates of the line segment are output from the input unit 41 to the control unit 3. Also, the input line segment is stored in the storage unit 2, while the control unit 3 reads the already input line segment from the storage unit 2.

  When the line segment is input, the control unit 3 operates as the spatial model correction unit 32. The space model correction unit 32 calculates a target plane that includes each line segment input by the user and is perpendicular to the estimated reference plane in the common coordinate system (step S13). Each target plane is surrounded by two sides that are perpendicular to the estimated reference plane at each end point of the line segment that is the basis of the calculation, and an upper side and a lower side that are passed horizontally between the two sides. Specified by a square. The lower side of the target surface is defined by a line segment that connects between the intersections of the two sides and the maximum plane. The upper side of the target plane may be a line segment having a maximum height in the space model and horizontal to the maximum plane.

  The space model correction unit 32 detects a plane corresponding to the target surface from the space shape represented by the three-dimensional point group constituting the space model (step S14). Specifically, the normal of the plane is calculated from the cluster of each plane and the normal of each target surface is calculated, and the angle formed by the normal between each plane and each target surface and the distance between the surfaces are compared. Then, a plane whose normal line direction difference and distance from each target plane is less than a predetermined value is detected as a plane corresponding to the target plane. Incidentally, the distance between the surfaces can be, for example, the length of a perpendicular line from the center point of the three-dimensional point group corresponding to the plane to the target surface. The predetermined value can be an upper limit value of the alignment error estimated through a previous experiment (for example, a value corresponding to 10 cm in real space). In addition, the space model correction unit 32 excludes a target surface from which a corresponding plane is not detected from subsequent processing targets. Note that the spatial model correction unit 32 may perform clustering again in step S14 instead of diverting the clustering result in step S2.

  The spatial model correction unit 32 identifies which local region each of the planes represented by the three-dimensional point group constituting the spatial model is included (step S15). In this process, the local region to which the cluster of each plane other than the cluster of the maximum plane belongs is specified.

  The spatial model correction unit 32 determines, for each local region identified as including a plane in step S15, the difference in position regarding rotation / translation between the plane and the target plane for which the correspondence between the plane is detected in step S14. Is calculated (step S16). This difference becomes closer to 0 as the positions and orientations of the two are similar, and becomes larger as they are shifted.

  The space model correction unit 32 calculates a correction amount for each local region that minimizes the difference calculated in step S16 in the entire space model (step S17). Since the 3D point cloud of the local region is already aligned in the common coordinate system, the difference in the rotation / translation amount between the 3D point clouds of the adjacent local regions is 0 (that is, the homogeneous transformation representing rotation / translation). The matrix is close to the identity matrix. In this situation, imposing an additional condition of reducing the difference in rotation / translation amount between the target plane and the plane corresponding to the target plane in the local area, the position and orientation of the corresponding plane in the local area is brought closer to the target plane, In addition, optimization is performed so that the position and orientation of the three-dimensional point group in the local area in the common coordinate system is maintained as much as possible.

An example of this optimization processing method will be described. The number of local area (or local space model) constituting a spatial model is N, the i local spatial model (1 ≦ i ≦ N) and P _i.

For any two P _i and P _j having an overlapping region, the local space model is correlated in the overlapping region, and the rotational translation between the P _i and P _j portions associated with each other The parameter Rt _{i_j} is obtained. That is, when performing a coordinate transformation corresponding to Rt _{I_j} to one of the parts and P _j moieties of P _i associated with each other, suitably matched to the other. Rt _{i — j} can be expressed by a vector composed of three degrees of freedom of translation (x, y, z) and three degrees of freedom of rotation (ρ, θ, φ). In particular,
Rt _{i_j} = (ρ _{i_j} , θ _{i_j} , φ _{i_j} , x _{i_j} , y _{i_j} , z _{i_j} )
It becomes the form.

A position and orientation parameter composed of three degrees of freedom (x, y, z) of translation and three degrees of freedom of rotation (ρ, θ, φ) of P _i aligned in the common coordinate system is represented by a vector X _i . In particular,
X _i = (ρ _i , θ _i , φ _i , x _i , y _i , z _i )
It becomes the form.

Two _P i with overlapping regions, as long as the difference in position and orientation of _{P j (X} i -X _j) corresponds to _{Rt I_j, P} i, the position and orientation of _{P j} by the coordinate conversion based on _{Rt I_j} Although aligned generally _(X i -X _j) is not necessarily be canceled by coordinate conversion by _{Rt I_j,} in this _case, P i, the position and orientation of the _{P j} are not completely matched, _{(X i -X j) -Rt} i_j remains as an error.

The spatial model input as a correction target to the spatial model processing device 1 is, for example, the difference between the (X _i −X _j ) and Rt _{i_j} (Mahalanobis distance) for two P _i and P _j having overlapping regions. The local space model is aligned to minimize the whole. Specifically, X _i is determined so as to minimize the error ε _A defined by the following equation as the sum of the Mahalanobis distances for all combinations of two P _i and P _j having overlapping regions. Here, C _{i — j} is a covariance matrix corresponding to the six elements of the vector X _i .

  As described above, the spatial model aligned in this way may be displaced between the originally same planes of the overlapping portions of the local space model. On the other hand, the space model correction means 32 guides the position and orientation of the corresponding plane of the local space model to approach the target surface based on the user's instruction according to the above-described additional conditions, and the aligned space model is described above. Correct the gap so that it is reduced or eliminated.

Specifically, the correction can be performed by minimizing an error defined by using the Mahalanobis distance, as in the above-described alignment of the local space model in the space model. Here, for any P _i having a plane corresponding to the target surface, it represents the rotation translation parameters between the plane and the target surface and Rt _{i_ref.} Rt _{i_ref} can be expressed by a six-component vector similar to Rt _{i_j} . Further, the position and orientation parameter of the target surface is represented by a six-component vector X _ref similar to X _i .

The spatial model correction unit 32 corrects X _i so as to minimize the Mahalanobis distance regarding the difference (X _i −X _ref ) between the position and orientation parameters between P _i and the target plane and Rt _{i_ref in the} entire spatial model. Specifically, at step S17, using the error epsilon _R defined by the following equation as the sum of the Mahalanobis distances for all P _i having a plane corresponding to the target surface, to minimize the error epsilon _R X _i is required. The difference from the X _i before correction of the X _i is calculated as a correction amount of the position and orientation. Here, C _{i_ref} is a covariance matrix, similar to C _{i_j} .

In other words, the spatial model correction unit 32 performs correction for alignment with the target surface in addition to the alignment that has already been performed, so that the error sum ε defined by, for example, ε = ε _A + ε _R is minimized. X _i is given as an optimal local region arrangement. Note that the weights of ε _A and ε _R in ε can be changed. For example, using a constant λ (where λ> 0), ε is generally ε = λ · ε _A + (1−λ). · ε can be _R.

  The spatial model correcting unit 32 corrects the spatial model by rotating and translating the local spatial model of each local region by the correction amount for the local region calculated in step S17, and the corrected spatial model is stored in the spatial model storage unit. 20 (step S18). That is, correction for converting the local coordinates of each local region is performed, and each point included in the three-dimensional point group of each local region is rotated and translated by the correction amount. After correction, the process returns to step S10.

[Modification]
In the above-described embodiment, the input unit 41 inputs a line segment in response to a user operation. However, the input unit 41 may be an external file input unit such as a USB interface. That is, the input means 41 reads an image file in which one or more line segments are drawn on a plane corresponding to a reference plane such as a floor surface, and inputs the line segments of the file as line segment information indicating the position of the target plane. To do.

  In this case, the generation of the density image by the density image generation unit 31 and the display of the density image by the display unit 40 can be omitted.

  Alternatively, in that case, the density image may be used to cause the user to input an instruction for rotation / translation of the image drawn in the image file. For example, the display means 40 superimposes and displays the image of the read file on the density image, and the mouse displays two points on the density image and two points on the line segment in the image file corresponding to those points. Two-dimensional rotation / translation is performed on the image file so that two points on the line segment coincide with two points on the density image, and the line segment after the rotation / translation is output to the space model correction means 32.

  DESCRIPTION OF SYMBOLS 1 Spatial model processing apparatus, 2 Storage part, 3 Control part, 4 User interface part, 20 Spatial model storage means, 30 Reference plane estimation means, 31 Density image generation means, 32 Spatial model correction means, 40 Display means, 41 Input means .

Claims (3)

A space model processing device for correcting a space model composed of point clouds obtained by three-dimensional measurement for a space in which an object plane serving as a reference plane and an object plane perpendicular to the reference plane exist,
For each of a plurality of local regions in the space, the spatial model in which the local space model composed of the point group in the local coordinate system of the local region is integrated in a common coordinate system common to the plurality of local regions is stored. A spatial model storage means,
A reference plane estimating means for detecting a portion corresponding to the object plane having the largest area in the spatial model and estimating a plane along the portion as the reference plane;
Input means for allowing a user to input a line segment representing a position on the reference plane of an object plane perpendicular to the reference plane;
A target plane that includes the line segment and is perpendicular to the estimated reference plane estimated by the reference plane estimation means is set, and a portion of interest having a position and shape corresponding to the target plane is extracted from the spatial model, and the common coordinate system A spatial model correcting means for moving the local spatial model so that the target portion is aligned with the target surface at
A spatial model processing device comprising: Density image generation means for generating a density image in which pixel values are determined according to the density of the point group in a projection image obtained by projecting the spatial model onto the estimation reference plane;
Display means for displaying the density image,
The input means allows the user to input the line segment on the density image;
The spatial model processing apparatus according to claim 1, wherein:   The density image generation means detects two planes that are perpendicular to the estimation reference plane and orthogonal to each other from the spatial model, and determine the direction of the line segment that is a projection image of each of the two planes on the estimation reference plane. The spatial model processing apparatus according to claim 2, wherein the density image is set in a horizontal direction and a vertical direction.

Images