JP2019219248A - Point group processor, method for processing point group, and program - Google Patents

Abstract

To obtain a background point group of a background region in an actual space more precisely.SOLUTION: The point group processor includes: a background extraction unit for extracting a background point group as a point group of background region from a three-dimensional point group corresponding to each of different timings obtained by measuring an actual space by LIDAR from the same point at the timings; and a background integration unit for generating an integrated background point group by integrating at least a part of the background point groups extracted from the point groups for the different timings.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a point cloud processing device, a point cloud processing method, and a program.

  In recent years, demands for three-dimensional measurement of real space and three-dimensional modeling of measurement data have been increasing. A method of realizing three-dimensional measurement of a real space and three-dimensional modeling of measurement data includes, for example, a three-dimensional laser that detects a response to active light (near infrared light), such as LIDAR (Laser Imaging Detection and Ranging). There is a method using a sensor.

  Regarding three-dimensional measurement, Patent Literature 1 adds a new point cloud generated by processing information obtained from an image sensor for an area having a low point density in a point cloud obtained using LIDAR. Thus, a technique for increasing the resolution of a point cloud has been disclosed. In the method of obtaining a depth map from infrared dots, a region that does not sufficiently reflect infrared light is a defective region in the depth map. In this regard, Patent Literature 2 discloses a method of interpolating a defective area using an image having RGB color information. The depth map may be used for generating a point cloud.

Patent No. 5778237 JP 2006-522923 A

  According to the techniques described in Patent Literature 1 and Patent Literature 2, a point cloud or a depth map expressing a real space at a certain point in time can be obtained. However, Patent Literature 1 and Patent Literature 2 do not disclose a technique for extracting a background point group belonging to a background area such as a floor or a wall in a real space from the entire point group. Even if there is a technology to extract the background point cloud from the whole point cloud obtained by measurement at a certain point in time, if some background area is hidden by the object, part of the background point cloud is missing I will.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a new and improved technique capable of obtaining a background point group belonging to a background area of a real space with higher accuracy. The present invention provides a point cloud processing apparatus, a point cloud processing method, and a program.

  In order to solve the above-described problems, according to an aspect of the present invention, a three-dimensional point group corresponding to each timing obtained by measuring a real space with the LIDAR from the same point at each of a plurality of timings. A background extraction unit that extracts a background point group that is a point group belonging to a background area, and integrating by integrating at least a part of each background point group extracted from the point group corresponding to each of the plurality of timings. And a background integration unit for generating a background point cloud.

  The background integration unit extracts a new partial point group from the new background point group extracted by the background extraction unit, and by repeatedly integrating the new partial point group with the integrated background point group, The integrated background point cloud may be updated repeatedly.

  The background integration unit may delete some of the points constituting the integrated background point group when the number of points constituting the integrated background point group exceeds an upper limit.

  Some of the points forming the integrated background point group may be random points forming the integrated background point group.

  The background integration unit may randomly extract the new partial point group from the new background point group.

  The background integration unit may randomly extract the new partial point group from a different range for each of the plurality of new background point groups.

  The point cloud processing device may further include an object detection unit that detects a region where an object is present by calculating a difference between a certain point cloud and the integrated background point cloud.

  The object detection unit is configured to obtain an integrated background point obtained by integrating a point group corresponding to a first timing and a partial point group extracted from a point group corresponding to a second timing before the first timing. The area where the object exists may be detected by calculating the difference from the group.

  The background area may include at least one of a floor area and a wall area.

  According to another aspect of the present invention, there is provided a three-dimensional image corresponding to each timing, which is obtained by measuring a real space with the LIDAR from the same point at each of a plurality of timings. From the point group, extracting a background point group that is a point group belonging to a background area, and integrating at least a part of each background point group extracted from the point group corresponding to each of the plurality of timings, Generating an integrated background point cloud.

  According to another embodiment of the present invention, there is provided a computer configured to measure a real space by using a LIDAR from the same point at each of a plurality of timings. A background extraction unit that extracts, from the three-dimensional point group, a background point group that is a point group belonging to a background area; By integration, a program for functioning as a background integration unit that generates an integrated background point cloud is provided.

  According to the present invention described above, it is possible to obtain a background point group belonging to a background area in a real space with higher accuracy.

It is an explanatory view showing a point cloud processing system by an embodiment of the present invention. FIG. 3 is an explanatory diagram illustrating a specific example of a real space. FIG. 3 is an explanatory diagram showing a background point group extracted from the real space point group shown in FIG. 2. FIG. 1 is an explanatory diagram illustrating a configuration of a point cloud processing device 200 according to an embodiment of the present invention. It is explanatory drawing which shows the specific example of extraction of the partial point group Pi. It is explanatory drawing which shows the specific example of an integrated background point group. 5 is a flowchart illustrating an operation of the point cloud processing device 200 according to the embodiment of the present invention. It is an explanatory view showing a first modification. It is an explanatory view showing a second modification. FIG. 2 is a block diagram illustrating a hardware configuration of a point cloud processing apparatus 200.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  Further, in this specification and the drawings, a plurality of components having substantially the same function and configuration may be distinguished from each other by adding different alphabets after the same reference numeral. However, unless it is necessary to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, each of the plurality of constituent elements is given only the same reference numeral.

<Overview of point cloud processing system>
The embodiment of the present invention relates to a point cloud processing system that detects an object existing in a real space by processing a point cloud obtained by measuring a real space by LIDAR (Laser Imaging Detection and Ranging). Prior to a detailed description of an embodiment of the present invention, an outline of a point cloud processing system according to an embodiment of the present invention will be described.

  FIG. 1 is an explanatory diagram showing a point cloud processing system according to an embodiment of the present invention. As shown in FIG. 1, the point cloud processing system according to the embodiment of the present invention includes a LIDAR 100 and a point cloud processing device 200. The LIDAR 100 and the point cloud processing apparatus 200 may be connected by wire, may be connected wirelessly, or may be connected via a network.

(LIDAR)
The LIDAR 100 is a measuring device that measures a three-dimensional environment in a real space. The LIDAR 100 is installed at a point to be supported by a tripod 110, for example, as shown in FIG. The LIDAR 100 repeats the measurement using the laser from the same installed point over a plurality of frames to obtain a three-dimensional point group of each frame. The LIDAR 100 outputs the point cloud obtained by the measurement to the point cloud processing device 200.

  The number of laser transmission / reception sensors provided in the LIDAR 100, the laser scanning method, and the measurable range are not particularly limited. For example, the number of laser transmission / reception sensors may be 8, 16, 32, 64, 128, or the like. The laser scanning method may be a method of changing the direction of the laser along the horizontal direction or a method of changing the direction of the laser along the vertical direction. Further, the measurable range is 120 ° or all directions in a horizontal field of view, and may be 20 °, 30 ° or 40 ° in a vertical field of view.

(Point cloud processing device)
The point cloud processing device 200 detects an object area where an object exists in a real space based on the point cloud input from the LIDAR 100. The detection of the object region can be realized by separating (segmenting) a point group belonging to a background region (fixed objects such as floors and walls) from a point group input from the LIDAR 100. Therefore, the point cloud processing apparatus 200 also has a function of extracting a background point cloud, which is a point cloud belonging to the background area, from the point cloud input from the LIDAR 100. The object to be detected by the point cloud processing device 200 is not particularly limited. For example, the point cloud processing device 200 may detect a moving object such as a person or a vehicle. Here, a specific example of extraction of a background point cloud by the point cloud processing device 200 will be described with reference to FIGS.

  FIG. 2 is an explanatory diagram showing a specific example of the real space. The real space shown in FIG. 2 is a space surrounded by a floor surface F and a wall surface W, and an object O exists on the floor surface F. In this specification, it is assumed that the LIDAR 100 is installed such that the XY plane is a plane horizontal to the floor F, and the XZ plane or the YZ plane is a plane horizontal to the wall surface W.

  FIG. 3 is an explanatory diagram illustrating a background point group extracted from the point group in the real space illustrated in FIG. 2. As shown in FIG. 3, the background point group is scattered in the floor surface F and the wall surface W in a region other than the portion shielded by the object O. Note that FIG. 3 merely shows the background point group simply. In practice, a background point group including more points may be obtained, or points may be distributed in an arc shape. .

(Organization of issues)
The object detection described above can be accurately performed by using a background point group having no shortage. On the other hand, as shown in FIG. 3, in the background point group in which the point group of the portion shielded by the object し is missing, a part of the point group to be treated as the background area may be detected as the object. .

  Thus, the present inventor has come up with an embodiment of the present invention with the above circumstances in mind. The point cloud processing device 200 according to the embodiment of the present invention can obtain the background point cloud belonging to the background area of the real space with higher accuracy. Hereinafter, the configuration and operation of the point cloud processing apparatus 200 according to the embodiment of the present invention will be sequentially described in detail.

<Configuration of point cloud processing device>
FIG. 4 is an explanatory diagram showing the configuration of the point cloud processing device 200 according to the embodiment of the present invention. As shown in FIG. 4, the point cloud processing apparatus 200 according to the embodiment of the present invention includes a point cloud storage unit 220, a background extraction unit 230, a background point cloud storage unit 240, a background integration unit 250, an object detection unit 260, a display. It has a control unit 270 and a display unit 280.

(Point cloud storage unit)
The point cloud storage unit 220 stores the point cloud of each frame input from the LIDAR 100. It is assumed that the point group includes a background point group belonging to the background region and an object point group belonging to the object region without being distinguished. Hereinafter, the point group input from the LIDAR 100 may be referred to as a point group Po in some cases.

(Background extraction unit)
The background extraction unit 230 extracts a background point group belonging to a background area from the point group Po stored in the point group storage unit 220. The background area includes, for example, a floor surface and a wall surface, and the background point group may include a floor surface point group Pof belonging to the floor surface and a wall surface point group Pow belonging to the wall surface, as shown in FIG. The floor surface is the largest plane in the point group Po horizontal to the XY plane, and the wall surface is estimated to be the largest plane in the point group Po horizontal to the XZ plane or the YZ plane. For this reason, the background extraction unit 230 obtains the floor surface point group Pof and the wall surface point group Pow by executing RANSAC (Random Sampling Consensus). Hereinafter, a procedure for estimating a point group constituting a planar shape by RANSAC will be described, and then a specific method for obtaining a floor point group Pof and a wall point group Pow by RANSAC will be described.

By executing RANSAC on the point group Po using three parameters, a point group P_plane forming a planar shape is output. The three parameters include a normal direction vpl of the plane shape to be estimated, an error threshold value dist_pl of a distance between a point group forming the plane and the plane, and an error threshold value of an angle between the point group forming the plane and the plane. angle_pl corresponds to this. These parameters can be set manually. In summary, the point group P_plane is expressed as follows.
P_plane: = RANSAC (Po, vpl, dist_pl, angle_pl)

Then, the background extraction unit 230 can obtain the floor surface point group Pof by applying the Z-axis direction to the above-described vpl. The process of obtaining the floor surface point group Pof is expressed as follows.
Pof: = RANSAC (Po, Z, dist_pl, angle_pl)

Furthermore, the background extraction unit 230 can obtain a wall surface point group Pow_Y belonging to a wall surface horizontal to the XZ plane by applying the Y-axis direction to the above-described vpl. In addition, the background extraction unit 230 can obtain a wall surface point group Pow_X belonging to a wall surface horizontal to the YZ plane by applying the X-axis direction to the above-described vpl. A point group obtained by combining the wall surface point group Pow_Y and the wall surface point group Pow_X is a wall surface point group Pow. The process of obtaining the wall surface point group Pow is expressed as follows.
Pow_Y: = RANSAC (Po, Y, dist_pl, angle_pl)
Pow_X: = RANSAC (Po, X, dist_pl, angle_pl)
Pow: = Pow_Y + Pow_X

(Background point cloud storage unit)
The background point cloud storage unit 240 stores the background point cloud. Specifically, when the floor point group Pof and the wall surface point group Pow extracted by the background extraction unit 230 by the above-described processing are point groups extracted from the point group Po of the first frame, the background point group storage unit 240 stores the floor surface point group Pof and the wall surface point group Pow as an initial background point group. Further, the background point group storage unit 240 stores an integrated background point group obtained by the background integration unit 250 described later.

(Background Integration Department)
The background integration unit 250 compares the initial background point group stored in the background point group storage unit 240 and the partial point group Pi of the background point group (floor point group Pof and wall surface point group Pow) extracted from the new frame. By performing integration, an integrated background point group Pb is generated. When the integrated background point group Pb is already stored in the background point group storage unit 240, the background integration unit 250 determines the integrated background point group Pb and the partial points of the background point group extracted from the new frame. By integrating the group Pi, the integrated background point group Pb is updated.

Here, the partial point group Pi of the floor surface point group Pof and the wall surface point group Pow extracted from the new frame may be a point group randomly extracted from the floor surface point group Pof and the wall surface point group Pow. Alternatively, it may be a point cloud extracted according to a rule specified by the user. The process in which the background integration unit 250 extracts such a partial point group Pi is expressed as follows. Pi_size is the number of points included in the partial point group Pi, and is a parameter given by the user. The number of Pi_size may be, for example, 0.3% to 1.0% of the number of points of the floor point group Pof and the wall point group Pow extracted from the new frame.
Pi: = Extract_pi (Pow + Pof, random or user rule, Pi_size)

Then, the background integration unit 250 generates or updates the integrated background point group Pb by adding the above-described partial point group Pi to the initial background point group or the integrated background point group Pb. This processing is expressed as follows.
Pb: = Pb + Pi

  The extraction of the partial point group Pi and the generation of the integrated background point group Pb will be described more specifically with reference to FIGS.

  FIG. 5 is an explanatory diagram showing a specific example of extraction of the partial point group Pi. FIG. 5 illustrates a floor point group Pof and a wall point group Pow extracted by the background extraction unit 230 in a frame subsequent to the frame illustrated in FIG. When FIG. 3 is compared with FIG. 5, the position where the object O is moving and the point group is missing due to being blocked by the object O is different.

  The background integration unit 250 extracts, for example, six partial point groups Pi indicated by thick lines in FIG. 5 from the floor surface point group Pof and the wall surface point group Pow illustrated in FIG. Then, the background integration unit 250 generates the integrated background point group Pb shown in FIG. 6 by adding the partial point group Pi to the initial background point group shown in FIG. In the integrated background point group Pb illustrated in FIG. 6, a point group is added at a position where the point group is missing due to being blocked by the object O in FIG. By repeating such addition of the partial point group Pi over a plurality of frames to update the integrated background point group Pb, it is possible to improve the accuracy of the integrated background point group Pb.

  However, when the addition of the partial point group Pi is repeated, the score of the integrated background point group Pb increases too much, and the storage area and the computational resources are squeezed. Therefore, the maximum number of points Npbmax is set as the upper limit of the number of points of the integrated background point group Pb, and the background integration unit 250 performs processing on the integrated background point group Pb according to the relationship between the number of points of the integrated background point group Pb and the maximum number of points Npbmax. May be performed.

  Specifically, when the score of the integrated background point group Pb is less than the maximum score Npbmax, the background integration unit 250 updates the integrated background point group Pb by adding the partial point group Pi to the integrated background point group Pb. .

  On the other hand, when the number of points of the integrated background point group Pb is equal to or larger than the maximum number of points Npbmax, the background integration unit 250 randomly deletes the same number of points P_d as the partial point group Pi from the integrated background point group Pb. Then, the background integration unit 250 updates the integrated background point group Pb by adding the partial point group Pi to the integrated background point group Pb after the elimination of the point Pb_d.

The above processing is organized as follows.
Pb: = Pb + Pi (when the maximum score is Npbmax> Pb)
Pb: = Pb−Pb_d + Pi (when the maximum score is Npbmax ≦ Pb)

(Object detection unit)
The object detection unit 260 detects an area where an object exists (object area) using the point cloud Po and the integrated background point cloud Pb. For this purpose, the object detection unit 260 calculates the difference between the point group Po and the integrated background point group Pb. In order to calculate the difference at high speed, the object detection unit 260 converts the point group Po and the integrated background point group Pb into an Octree structure to obtain Oct_Po and Oct_Pb. Then, the object detection unit 260 obtains the point group information Diff_Po_Pb of the difference between Oct_Po and Oct_Pb. Furthermore, the object detection unit 260 obtains a difference point group Pd that is a difference between the point group Po and the integrated background point group Pb by applying Diff_Po_Pb to the point group Po.

The process for obtaining the difference point group Pd described above is expressed as follows.
Oct_Po: = Octree (Po)
Oct_Pb: = Octree (Pb)
Diff_Po_Pb: = Octdiff (Oct_Po, Oct_Pb)
Pd: = Octindice (Po, Diff_Po_Pb)

  Then, the object detection unit 260 obtains an object region Region_d by performing clustering on the difference point group Pd. Specifically, the object detection unit 260 first converts the difference point group Pd into a Kdtree structure in order to perform clustering, and obtains Kd_Pd. Then, the object detection unit 260 performs clustering using Kd_Pd, the allowable distance dist_c between each cluster, the minimum number of cluster point cloud points Ncmin, and the maximum number of cluster point cloud points Ncmax, and performs a cluster Pc of the point cloud forming the object. Get. By this clustering, zero to a plurality of Pc are obtained. Note that dist_c, Ncmin and Ncmax may be parameters specified by the user.

  Further, the object detection unit 260 calculates an object region Region_d where the object exists from the Pc group. Region_d has a list structure, and each element stores the number of point groups of each Pc, the maximum and minimum coordinates of the cluster, the cluster size, the center of gravity of the cluster, the distance from the cluster to the LIDAR 100, the type of object, and the like. The object detection unit 260 calculates the number of point groups and the coordinates of the cluster by searching for the point group of Pc, and calculates the cluster size, the center of gravity of the cluster, the distance from the cluster to the LIDAR 100, The type of the object can be calculated.

The processing from the above-described clustering to detection of the object region Region_d is expressed as follows. Kdtree is a function for converting an argument into a Kdtree structure, Clustering is a function for outputting a cluster group constituting an object in a point group, Region_f is the number of point groups of each cluster, maximum / minimum coordinates of the cluster, This function outputs a list of structures including the cluster size, the center of gravity of the cluster, the distance from the cluster to the LIDAR 100, and the type of object.
Kd_Pd: = Kdtree (Pd)
Pc: = Clustering (Kd_Pd, dist_c, Ncmin, Ncmax)
Region_d: = Region_f (Pc)

(Display control unit)
The display control unit 270 uses the point cloud Po and the object region Region_d read from the point cloud storage unit 220 to generate a detection result screen indicating the detection result of the object in the point cloud Po. The detection result screen is generated by arranging the point group Po, arranging a cluster region (cube) using the maximum and minimum coordinates of the cluster obtained from Region_d, and using the maximum and minimum coordinates. Calculating each vertex of the cube, and drawing a line between the vertices of each cube so as to be an edge of the cube, for each object existing in Region_d.

(Display)
Display unit 280 displays the detection result screen generated by display control unit 270. Thereby, all the object regions existing in the point group Po are visualized.

<Operation of point cloud processing device>
The configuration of the point cloud processing device 200 according to the embodiment of the present invention has been described above. Subsequently, the operation of the point cloud processing device 200 according to the embodiment of the present invention will be described with reference to FIG.

  FIG. 7 is a flowchart showing the operation of the point cloud processing device 200 according to the embodiment of the present invention. As shown in FIG. 7, first, the point cloud storage unit 220 stores the point cloud Po of the first frame input from the LIDAR 100 (S304).

  Then, the background extraction unit 230 extracts a background point group (floor point group Pof and wall surface point group Pow) belonging to the background area from the point group Po stored in the point group storage unit 220 (S308). The background point group storage unit 240 stores the background point group extracted by the background extraction unit 230 as an initial background point group (S312).

  Thereafter, when the point group Po of the new frame is input from the LIDAR 100, the point group storage unit 220 stores the point group Po of the new frame (S316). Subsequently, the background extraction unit 230 extracts a background point group from the point group Po of the new frame (S320), and the background integration unit 250 extracts a partial point group Pi from the background point group (S324).

  Here, when the integrated background point group Pb is stored in the background point group storage unit 240, the background integration unit 250 determines whether or not the score of the integrated background point group Pb is equal to or more than the maximum number of points Npbmax (S328). ). If the score of the integrated background point group Pb is equal to or greater than the maximum score Npbmax, or if the integrated background point group Pb is not stored in the background point group storage unit 240 (S328 / No), the process proceeds to S336. On the other hand, when the number of points of the integrated background point group Pb is equal to or greater than the maximum number of points Npbmax (S328 / Yes), the background integration unit 250 deletes the same number of point groups as the partial point group Pi from the integrated background point group Pb (S332). .

  Next, the background integration unit 250 updates (generates) the integrated background point group Pb by integrating the partial point group Pi with the integrated background point group Pb (the first round is the initial background point group) (S336). Then, the object detection unit 260 calculates the difference between the point group Po of the new frame and the integrated background point group Pb to detect the object region (S340), and the display unit 280 displays the object region in the point group Po of the new frame. It is superimposed and displayed (S344).

  Thereafter, the processing of S316 to S344 is repeated until the measurement by the LIDAR 100 ends (S348).

<Effects>
According to the embodiment of the present invention described above, various effects can be obtained.

  For example, the background integration unit 250 updates the integrated background point group Pb by extracting a partial point group Pi from the background point group of the new frame and adding the partial point group Pi. In contrast, all the background point groups of the new frame are added, and after the number of points of the integrated background point group Pb reaches the maximum number of points Npbmax, some of the point groups of the integrated background point group Pb are added to the new frame. It is also conceivable to replace all of the background point clouds with. However, in this method, when the object moves, although the background point cloud at the position of the object before the movement can be compensated for, the original background point cloud at the position of the object after the movement is rapidly lost. I will. On the other hand, according to the embodiment of the present invention, it is possible to improve the robustness of the integrated background area Pb with respect to the movement of the object.

  Further, the background integration unit 250 randomly (probabilistically) extracts the partial point group Pi from the background point group of the new frame. According to such a configuration, it is possible to interpolate the loss of the point group without specifying the position where the point group is missing in the background point group.

  After the number of points in the integrated background point group Pb reaches the maximum number of points Npbmax, the background integration unit 250 randomly deletes points included in the integrated background point group Pb. According to this configuration, even if a point group that does not belong to the background area is included in the integrated background point group Pb, the point group can be gradually eliminated, and as a result, the integrated background point group Pb It is possible to improve the accuracy.

<Modification>
The embodiment of the invention has been described. Hereinafter, some modified examples of the embodiment of the present invention will be described. Each of the modified examples described below may be applied to the embodiment of the present invention alone, or may be applied to the embodiment of the present invention in combination. Further, each modification may be applied instead of the configuration described in the embodiment of the present invention, or may be additionally applied to the configuration described in the embodiment of the present invention.

(First Modification)
In the above, the example in which the background integration unit 250 randomly extracts the partial point group Pi from the entire background point group has been described. The first modified example is a modified example relating to the extraction of the partial point group Pi. The background integration unit 250 may randomly extract the partial point group Pi from within a different range for each frame. The first modified example will be described more specifically with reference to FIG.

  FIG. 8 is an explanatory diagram showing a first modification. As shown in FIG. 8, the background integration unit 250 divides the background point group into a plurality of regions D1 to D3, and in the background point group of one frame, the background point group is randomly divided from the point group included in any of the regions D1 to D3. May be extracted. For example, in the n-th frame, the background integration unit 250 may randomly extract the partial point group Pi from the point group included in the region D1, such as five point groups indicated by thick lines. Then, the background integration unit 250 randomly extracts the partial point group Pi from the point group included in the region D2 in the (n + 1) th frame, and randomly extracts the partial point group Pi from the point group included in the region D3 in the (n + 2) th frame. May be extracted.

(Second Modification)
In the above description, an example has been described in which the integrated background point group Pb is updated using the point group Po of the new frame, and after the integrated background point group Pb is updated, object detection is performed using the integrated background point group Pb. The second modification is a modification relating to the relationship between updating of the integrated background point group Pb and object detection. In the second modification, the object detection unit 260 determines the point group Po (n) of the frame n and the partial point group Pi extracted from the point group Po (n-1) of the frame n-1 obtained earlier. The object region is detected by calculating the difference from the integrated background point group Pb (n-1) obtained by the integration of (n-1). Hereinafter, the second modified example will be described more specifically with reference to FIG.

  FIG. 9 is an explanatory diagram showing a second modification. The processing of S304 to S316 is as described with reference to FIG. As shown in FIG. 9, the background integration unit 250 updates the integrated background point group Pb using the point group Po of the new frame (S320, S324, S328, S332, S336). The object detection unit 260 detects an object region in parallel with the update of the integrated background point group Pb by the background integration unit 250.

  That is, the object detection unit 260 calculates the difference between the point group Po of the new frame and the integrated background point group Pb updated in the previous frame, and detects an object region (S322). Then, the display unit 280 superimposes and displays the object area on the point group Po of the new frame (S326).

  According to such a configuration, object detection can be performed without waiting for the integrated background point group Pb to be updated using the point group Po of the new frame. Therefore, the responsiveness and the immediacy of the display of the object area can be improved. It is possible to improve.

<Hardware configuration>
The embodiment of the invention has been described. The information processing such as the extraction of the background point cloud and the update of the integrated background point cloud is realized by cooperation between software and hardware of the point cloud processing device 200 described below.

  FIG. 10 is a block diagram illustrating a hardware configuration of the point cloud processing apparatus 200. The point cloud processing apparatus 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and a host bus 204. Further, the point cloud processing device 200 includes a bridge 205, an external bus 206, an interface 207, an input device 208, a display device 209, an audio output device 210, a storage device (HDD) 211, a drive 212, A network interface 215.

  The CPU 201 functions as an arithmetic processing device and a control device, and controls overall operations in the point cloud processing device 200 according to various programs. Further, the CPU 201 may be a microprocessor. The ROM 202 stores programs used by the CPU 201, operation parameters, and the like. The RAM 203 temporarily stores programs used in the execution of the CPU 201, parameters that appropriately change in the execution, and the like. These are mutually connected by a host bus 204 composed of a CPU bus and the like. By the cooperation of the CPU 201, the ROM 202, the RAM 203, and the software, the functions of the above-described background extraction unit 230, background integration unit 250, object detection unit 260, and display control unit 270 can be realized.

  The host bus 204 is connected to an external bus 206 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 205. Note that the host bus 204, the bridge 205, and the external bus 206 do not necessarily need to be separately configured, and these functions may be mounted on one bus.

  The input device 208 includes an input unit such as a mouse, a keyboard, a touch panel, a button, a microphone, a sensor, a switch, and a lever for a user to input information, and an input that generates an input signal based on an input by the user and outputs the input signal to the CPU 201. It is composed of a control circuit and the like. By operating the input device 208, the user of the point cloud processing apparatus 200 can input various data to the point cloud processing apparatus 200 or instruct a processing operation.

  The display device 209 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device, and a lamp. The audio output device 210 includes an audio output device such as a speaker and headphones.

  The storage device 211 is a data storage device configured as an example of a storage unit of the point cloud processing device 200 according to the present embodiment. The storage device 211 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 211 is composed of, for example, a hard disk drive (HDD) or a solid storage drive (SSD), or a memory having an equivalent function. The storage device 211 drives a storage and stores programs executed by the CPU 201 and various data.

  The drive 212 is a reader / writer for a storage medium, and is built in or external to the point cloud processing device 200. The drive 212 reads information recorded on a removable storage medium 24 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 203 or the storage device 211. The drive 212 can also write information on the removable storage medium 24.

  The network interface 215 is a communication interface including, for example, a communication device for connecting to a network. The network interface 215 may be a wireless LAN (Local Area Network) compatible communication device or a wire communication device that performs wired communication.

<Supplement>
As described above, the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to such examples. It is apparent that those skilled in the art to which the present invention pertains can conceive various changes or modifications within the scope of the technical idea described in the appended claims. It is understood that these also belong to the technical scope of the present invention.

  For example, each step in the processing of the point cloud processing apparatus 200 in the present specification does not necessarily need to be processed in a time series in the order described in the flowchart. For example, each step in the processing of the point cloud processing device 200 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

  In the above description, an example in which the integrated background point group Pb is generated by adding a partial point group to the initial background point group has been described. However, each of the partial point groups Pi extracted from the background point group of a plurality of frames is described. The integrated background point group Pb may be generated by the integration.

  The target measured by the LIDAR 100 according to the embodiment of the present invention is not particularly limited. For example, the object measured by the LIDAR 100 may be indoors, outdoors, or a construction site.

  In addition, a computer program for causing hardware such as a CPU, a ROM, and a RAM built in the point cloud processing apparatus 200 to exhibit the same functions as those of the configuration of the point cloud processing apparatus 200 can be created. Also, a storage medium storing the computer program is provided.

100 LIDAR
110 Tripod 200 Point cloud processing unit 220 Point cloud storage unit 230 Background extraction unit 240 Background point cloud storage unit 250 Background integration unit 260 Object detection unit 270 Display control unit 280 Display unit

Claims (11)

Background extraction for extracting a background point group, which is a point group belonging to a background area, from a three-dimensional point group corresponding to each timing, obtained by measuring a real space by LIDAR from the same point at each of a plurality of timings Department and
A background integration unit that generates an integrated background point cloud by integrating at least a part of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
A point cloud processing device comprising:   The background integration unit extracts a new partial point group from the new background point group extracted by the background extraction unit, and by repeatedly integrating the new partial point group with the integrated background point group, The point cloud processing device according to claim 1, wherein the integrated background point cloud is repeatedly updated.   The point group processing according to claim 2, wherein the background integration unit deletes some of the points constituting the integrated background point group when the number of points constituting the integrated background point group exceeds an upper limit. apparatus.   The point cloud processing device according to claim 3, wherein some of the points forming the integrated background cloud are random points forming the integrated background cloud.   The point group processing device according to claim 2, wherein the background integration unit randomly extracts the new partial point group from the new background point group.   The point group processing device according to claim 5, wherein the background integration unit randomly extracts the new partial point group from a different range for each of the plurality of new background point groups. The point cloud processing device,
The point cloud processing device according to claim 1, further comprising an object detection unit configured to calculate a difference between a certain point cloud and the integrated background point cloud to detect a region where an object is present.   The object detection unit is configured to obtain an integrated background point obtained by integrating a point group corresponding to a first timing and a partial point group extracted from a point group corresponding to a second timing before the first timing. The point cloud processing device according to claim 7, wherein a region where the object exists is detected by calculating a difference from the cloud.   The point cloud processing device according to claim 1, wherein the background area includes at least one of a floor area and a wall area. Extracting a background point group, which is a point group belonging to a background region, from a three-dimensional point group corresponding to each timing obtained by measuring the real space by LIDAR from the same point at each of a plurality of timings; ,
By integrating at least a part of each background point group extracted from the point group corresponding to each of the plurality of timings, to generate an integrated background point group,
And a point cloud processing method. Computer
Background extraction for extracting a background point group, which is a point group belonging to a background area, from a three-dimensional point group corresponding to each timing, obtained by measuring a real space by LIDAR from the same point at each of a plurality of timings Department and
A background integration unit that generates an integrated background point cloud by integrating at least a part of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
Program to function as

Images