JP6495559B2 - Point cloud processing system - Google Patents

Description

  The present invention relates to a point cloud processing system, method, and program.

  A technique for photographing the sky with an unmanned air vehicle such as a drone and measuring the distance to the ground with a radar is known. An attempt has been made to acquire three-dimensional point cloud data of a field such as a construction site using this technique (see, for example, Patent Document 1). The point cloud data includes three-dimensional coordinate data and color data. Then, using these three-dimensional coordinate data and color data, a soil volume calculation for making the designed land is performed from an aerial image taken by an unmanned air vehicle.

JP 2006-194515 A

  By the way, after acquiring point cloud data, there is a need to extract or delete only a desired region. For example, when a certain vacant area is photographed, there is a need to extract only the forest part from the photographed vacant area image or to delete it. At that time, only the desired forest part is selected by image analysis. For example, it is difficult to select a predetermined region such as a forest appropriately even if artificial intelligence is selected by distinguishing the color or altitude of a point. There are many cases.

  The present invention has been made in view of such a demand, and an object of the present invention is to provide a system that makes it possible to accurately extract or delete a desired region from point cloud data.

  The present invention provides the following solutions.

The invention according to the first feature is a point cloud processing system for processing three-dimensional point cloud data,
Obtaining means for obtaining three-dimensional point cloud data including the color and three-dimensional coordinates of a point to be processed;
Image display means for displaying the three-dimensional point cloud data in a predetermined dimension based on the three-dimensional coordinates;
Accepting means for accepting a change in parameters relating to the analysis method of the point to be processed;
A point cloud processing system comprising: a re-display unit that re-displays the point cloud reflecting the received parameter change.

  According to the invention relating to the first feature, it is possible to change the parameters related to the analysis method of the point to be processed for the three-dimensional point cloud data and redisplay the point cloud reflecting the change. . Therefore, it is possible to provide a system that can accurately extract or delete an area desired by a user from a point cloud image.

The invention according to the second feature is the invention according to the first feature,
Learning means for learning a correlation between the three-dimensional point cloud data and the parameter;
Provided is a point group processing system further comprising a recommendation unit that recommends a parameter to be set with new three-dimensional point group data based on a result learned by the learning unit.

  According to the second aspect of the invention, the correlation between the 3D point cloud data and the parameters is learned, and parameters to be set with the new 3D point cloud data are recommended based on the learning result. Therefore, it is possible to further increase the accuracy of accurately extracting or deleting a region desired by the user from the point cloud image.

  According to the present invention, it is possible to provide a system that can accurately extract or delete a region desired by a user from a point cloud image.

FIG. 1 is a block diagram showing a hardware configuration and software functions of a point cloud processing system 1 in the present embodiment. FIG. 2 is a flowchart showing a point cloud processing method according to this embodiment. FIG. 3 is an example when a two-dimensional image of three-dimensional point cloud data is displayed on the image display unit 34. FIG. 4 is a display example on the image display unit 34 when a parameter change is received. FIG. 5 is an example when the two-dimensional image of the three-dimensional point cloud data is redisplayed on the image display unit 34.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

<Configuration of Point Cloud Processing System 1>
FIG. 1 is a block diagram for explaining the hardware configuration and software functions of a point cloud processing system 1 according to this embodiment. The point cloud processing system 1 is a system for processing 3D point cloud data.

  The point cloud processing system 1 is connected to an aerial imaging apparatus 10 that captures an imaging target, wirelessly communicates with the aerial imaging apparatus 10, a controller 20 that controls the aerial imaging apparatus 10, and the aerial imaging apparatus 10. And a point cloud processing device 30 for processing an image (three-dimensional point cloud data).

[Aerial Camera 10]
The aerial imaging device 10 is not particularly limited as long as it is a device that can shoot a subject to be photographed from the sky. For example, the aerial imaging apparatus 10 may be a radio controlled airplane or an unmanned aerial vehicle called a drone. In the following description, it is assumed that the aerial imaging apparatus 10 is a drone.

  The aerial imaging apparatus 10 is powered by a battery 11 that functions as a power source for the aerial imaging apparatus 10, a motor 12 that operates with electric power supplied from the battery 11, and the motor 12. And a rotor 13.

  The aerial imaging device 10 also includes a control unit 14 that controls the operation of the aerial imaging device 10, a position detection unit 15 that transmits position information of the aerial imaging device 10 to the control unit 14, and a control signal from the control unit 14. A driver circuit 16 for driving the motor 12, a camera 17 for taking an aerial image of an object to be photographed in accordance with a control signal from the control unit 14, a control program executed by the microcomputer of the control unit 14, and the like are stored in advance. 17 includes a storage unit 18 for storing an image taken by the camera.

  The aerial imaging apparatus 10 includes a wireless communication unit 19 that performs wireless communication with the controller 20.

  These components are mounted on a main body structure (frame or the like) having a predetermined shape. As for a main body structure (frame or the like) having a predetermined shape, a similar one to a known drone may be adopted.

[Battery 11]
The battery 11 is a primary battery or a secondary battery, and supplies power to each component in the aerial imaging apparatus 10. The battery 11 may be fixed to the aerial imaging apparatus 100 or may be detachable.

[Motor 12, rotor 13]
The motor 12 functions as a drive source for rotating the rotor 13 with electric power supplied from the battery 11. By rotating the rotor 13, the aerial imaging device 10 can be levitated and flying.

[Control unit 14]
The controller 14 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like.

  Further, the control unit 14 implements the photographing module 141 by reading a predetermined program.

  The control unit 14 controls the motor 12 to perform flight control (control of ascending, descending, horizontal movement, etc.) of the aerial imaging apparatus 10. The control unit 14 controls the attitude of the aerial imaging apparatus 10 by controlling the motor 12 using a gyro (not shown) mounted on the aerial imaging apparatus 10.

[Position detection unit 15]
The position detection unit 15 includes a LIDAR (Laser Imaging Detection and Ranging) technique and a GPS (Global Positioning System) technique. By using these LIDAR technology and GPS technology together, the latitude, longitude, and height (for example, the height of a tree) of each point constituting the three-dimensional point cloud data of the image captured by the camera 17 are detected.

[Driver circuit 16]
The driver circuit 16 has a function of applying a voltage designated by a control signal from the control unit 14 to the motor 12. As a result, the driver circuit 16 can drive the motor 12 in accordance with the control signal from the control unit 14.

[Camera 17]
The camera 17 has a function of converting (imaging) an optical image captured by a lens into an image signal by an imaging element such as a CCD or a CMOS. The type of the camera 17 may be selected as appropriate according to the image analysis method to be photographed.

[Storage unit 18]
The storage unit 18 is a device that stores data and files, and includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card. The storage unit 18 stores a control program storage area (not shown) for storing in advance a control program executed by the microcomputer of the control unit 14, and image data captured by the camera 17. 3D points to be stored together with the 3D coordinate data detected in (the data of the latitude, longitude, and height (eg, tree height) of each point constituting the 3D point cloud data of the image taken by the camera 17) A group data storage area (not shown) and the like are provided.

  The data stored in the three-dimensional point cloud data storage area can be transferred to the point cloud processing device 30 through a portable recording medium such as a USB memory or an SD card.

[Wireless communication unit 19]
The wireless communication unit 19 is configured to be able to perform wireless communication with the controller 20 and receives a remote control signal from the controller 20.

[Controller 20]
The controller 20 has a function of operating the aerial imaging apparatus 10. The controller 20 includes an operation unit 21 that is used by a user to steer the aerial imaging apparatus 10, a control unit 22 that controls the operation of the controller 20, a control program that is executed by a microcomputer of the control unit 22, and the like. A storage unit 23 to be stored, a wireless communication unit 24 that wirelessly communicates with the aerial imaging apparatus 10, and an image display unit 25 that displays a predetermined image to the user are provided.

  The wireless communication unit 24 is configured to be able to wirelessly communicate with the aerial imaging apparatus 10 and receives a remote control signal toward the aerial imaging apparatus 10.

  The image display unit 25 may be integrated with a control device that controls the aerial imaging apparatus 10, or may be a separate body from the control device. If integrated with the control device, the number of devices used by the user can be reduced, and convenience is enhanced. When the control device is separate from the control device, examples of the image display unit 25 include portable terminal devices such as smartphones and tablet terminals that can be wirelessly connected to the wireless communication unit 19 of the aerial imaging device 10. When the control device is separate from the control device, an existing control device that does not have the image display unit 25 can be applied.

[Point Cloud Processing Device 30]
The point cloud processing device 30 has a function of processing three-dimensional point cloud data of a photographed image photographed using the camera of the aerial imaging device 10. The point cloud processing device 30 includes an input unit 31 for a user to input command information, a control unit 32 for controlling the operation of the point cloud processing device 30, a control program executed by a microcomputer of the control unit 32, and the like. Is stored in advance, and an image display unit 34 for displaying a predetermined image to the user.

  The control unit 32 implements an acquisition module 321, an image display module 322, a reception module 323, a redisplay module 324, a learning module 325, and a recommendation module 326 by reading a predetermined program.

  The storage unit 33 stores a 3D point cloud database 331 that stores the 3D point cloud data stored in the storage unit 18 of the aerial imaging apparatus 10 transferred through a portable recording medium such as a USB memory or an SD card. A parameter setting area 332 that stores setting values of various parameters set by the user, a parameter learning area 333 that stores recommended values of parameters obtained by the execution of the learning module 325, and the like.

<Flowchart showing a point cloud processing method using the point cloud processing system 1>
FIG. 2 is a flowchart showing a point cloud processing method using the point cloud processing system 1. The processing executed by each hardware and the software module described above will be described.

[Step S10: Acquisition of Captured 3D Point Cloud Data]
First, the control unit 14 of the aerial imaging apparatus 10 of the point cloud processing system 1 executes the imaging module 141 and controls the camera 17 to image the imaging target. Then, the control unit 14 converts the image data captured by the camera 17 into the three-dimensional coordinate data detected by the position detection unit 15 (the latitude of each point constituting the three-dimensional point cloud data of the image captured by the camera 17, The longitude and height (for example, the height of a tree) are stored in a three-dimensional point cloud data storage area (not shown) of the storage unit 18.

  The information stored in the 3D point cloud data storage area is stored in the 3D point cloud database 331 of the point cloud processing device 30 via the recording medium after the aerial imaging device 10 has landed. In this way, the control unit 32 of the point cloud processing device 30 executes the acquisition module 321 to acquire 3D point cloud data including the color and 3D coordinates of the point to be processed.

[Step S11: Image Display of 3D Point Cloud Data]
Subsequently, the control unit 32 of the point group processing apparatus 30 executes the image display module 322, and based on the three-dimensional coordinates included in the three-dimensional point group data, the three-dimensional point group data is displayed in a predetermined dimension. 34.

  The dimension of the image to be displayed is not particularly limited. For example, the image display unit 34 may display a two-dimensional image as a two-dimensional image or three-dimensionally as a three-dimensional image.

  FIG. 3 is an example when a two-dimensional image of three-dimensional point cloud data is displayed on the image display unit 34. The three-dimensional point cloud data includes color information. In the example of FIG. 3, the two-dimensional image includes a region 101 indicating a forest and a region 102 indicating a flat ground.

[Step S12: Accept parameter change]
Returning to FIG. Subsequently, the control unit 32 of the point cloud processing device 30 executes the receiving module 323 to receive a parameter change related to the analysis method of the point to be processed.

  FIG. 4 is a display example on the image display unit 34 when a parameter change is received. On the right side of the image display unit 34, a parameter setting operation area 103 (forest detection intensity, forest removal intensity, height resolution, color resolution) is displayed.

  The forest detection intensity is an intensity at which the control unit 32 detects the forest based on the point height (for example, tree height) and color for each point constituting the three-dimensional point cloud data. means. When the intensity is low, the control unit 32 recognizes that the point is the region 101 indicating the forest only when the height of the point (for example, the height of the tree or crop) is high and the color is dark green. . In other cases, the area 102 is recognized as a flat area 102. When the intensity is high, the control unit 32 is a region where the point indicates a forest even when the height of the point (for example, the height of a tree or the height of a crop) is low or the color is yellow-green. If the color is earthy, it is recognized as an area 102 indicating a flat ground.

  Deforestation intensity means the extent to which the region 101 indicating the forest is excluded from the three-dimensional point cloud data. When the intensity is high, the control unit 32 has a high point height (for example, a tree height) and a low point height (for example, a tree height) as well as a region having a dark green color. In some cases, the region whose color is yellowish green is also changed from the display state to the non-display state as the region 101 indicating the forest. When the intensity is low, the control unit 32 determines that only a region having a high point height (for example, a tree height) and a dark green color is the region 101 indicating the forest, from the display state to the non-display state. Change to

  For example, a field or a vacant land with grass is not necessarily an area 101 indicating a forest even if the color is green. The setting of the deforestation intensity is particularly important in order to prevent erroneous recognition of the region 101 indicating the forest in spite of the fact that the region 102 is actually flat.

  In the present embodiment, the user-desired region (here, the region 101 indicating the forest) is deleted from the three-dimensional point cloud data. However, the user desires from the three-dimensional point cloud data. It is good also as a setting which extracts an area.

  The height resolution means a function for adjusting the accuracy of the height of a point (for example, the height of a tree). If the accuracy is high, the height of a point (for example, the height of a tree) is determined in centimeters. If the accuracy is low, the height of the point (eg, the height of the tree) is determined in meters.

  The color resolution means a function for determining how far a color is judged with respect to a color. When the color resolution is high, for example, the RGB value is determined in one unit. When the color resolution is low, for example, the RGB value is determined in units of 10 units.

  On the left side of the image display unit 34, the current image (image before update), that is, the image displayed on the image display unit 34 in the process of step S11 is displayed. On the right side, an updated image, that is, an image when various parameters are set under the conditions shown in the setting operation area 103 and the green area 101 indicating the forest is changed from the display state to the non-display state is shown.

  Below the setting operation area 103, a confirmation area 104 is provided, and the word “OK” is shown in the center. When the cursor 105 is clicked on the confirmation area 104, the changed parameter setting value is confirmed. The changed parameter setting value is stored in the parameter setting area 332 of the storage unit 33.

[Step S13: Redisplay of the point cloud reflecting the change]
Returning to FIG. Subsequently, the control unit 32 of the point cloud processing device 30 redisplays the point cloud reflecting the received parameter change.

  FIG. 5 is an example when the two-dimensional image of the three-dimensional point cloud data is redisplayed on the image display unit 34. In the display example of FIG. 3, the area 101 indicating the forest that has been displayed is not displayed, and only the area 102 indicating the flat ground is displayed. At that time, not only the area where the ground surface is exposed, but also the area where the color is green, such as a vacant area where fields and grass grow, is actually a flat area instead of a forest, as the area 102 indicating a flat area. It is displayed correctly.

  According to the present embodiment, it is possible to change the parameters relating to the analysis method of the point to be processed for the three-dimensional point cloud data, and redisplay the point cloud reflecting the change. Therefore, it is possible to provide the point cloud processing system 1 that can improve the accuracy that the redisplayed image of the point cloud is an image obtained by accurately deleting the region desired by the user.

[Step S14: Learn the correlation between 3D point cloud data and parameters S14]
In the display example of FIG. 4, the setting operation area 103 is manually operated when a parameter change is accepted. However, in order to support an operator who is not familiar with the operation of the point cloud processing system 1, the control unit 32 of the point cloud processing device 30 executes the learning module 325 to execute the three-dimensional point cloud data and the step S12. It is preferable to learn the correlation with the parameters set in the process.

  The learning result is stored in the parameter learning area 333 of the storage unit 33. Then, when executing the process of step S12 again, the control unit 32 of the point cloud processing device 30 executes the recommendation module 326, and based on the content stored in the parameter learning area 333, new three-dimensional point cloud data. It is preferable to recommend the parameter to be set to the operator.

  According to this embodiment, even an operator who is not familiar with the operation of the point cloud processing system 1 can operate the point cloud processing system 1 with peace of mind. In addition, it is possible to provide the point cloud processing system 1 that can further increase the accuracy that the redisplayed image of the point cloud is an image obtained by accurately deleting the region desired by the user.

  The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.), for example. In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

1 point group processing system 10 aerial imaging device 11 battery 12 motor 13 rotor 14 control unit 141 imaging module 15 position detection unit 16 driver circuit 17 camera 18 storage unit 19 wireless communication unit 20 controller 30 point group processing device 31 input unit 32 control unit 321 Acquisition module 322 Image display module 323 Reception module 324 Redisplay module 325 Learning module 326 Recommendation module 33 Storage unit 331 Three-dimensional point cloud database 332 Parameter setting area 333 Parameter learning area 34 Image display section

Claims (3)

A point cloud processing system for processing 3D point cloud data,
Obtaining means for obtaining three-dimensional point cloud data including the color and three-dimensional coordinates of a point to be processed;
Image display means for displaying the three-dimensional point cloud data in a predetermined dimension based on the three-dimensional coordinates;
Accepting means for accepting a change in parameters relating to the analysis method of the point to be processed;
Re-display means for re-displaying the point cloud reflecting the received parameter change;
Learning means for learning a correlation between the three-dimensional point cloud data and the parameter;
Recommending means for recommending parameters to be set in the new three-dimensional point cloud data based on the result learned by the learning means;
A point cloud processing system comprising: A point cloud processing method executed by a point cloud processing system for processing 3D point cloud data,
Obtaining 3D point cloud data including the color and 3D coordinates of a point to be processed;
Displaying the 3D point cloud data in a predetermined dimension based on the 3D coordinates;
A step of accepting a parameter change relating to an analysis method of a point to be processed;
Redisplaying the point cloud reflecting the received parameter change;
Learning a correlation between the three-dimensional point cloud data and the parameter;
A step based on the result learned by step, recommending to be set in a new three-dimensional point group data parameters the learning,
A point cloud processing method comprising: In the point cloud processing system,
Obtaining 3D point cloud data including the color and 3D coordinates of a point to be processed;
Displaying the three-dimensional point cloud data in a predetermined dimension based on the three-dimensional coordinates;
A step of accepting a parameter change relating to an analysis method of a point to be processed;
Redisplaying the point cloud reflecting the change in the accepted parameter;
Learning a correlation between the three-dimensional point cloud data and the parameter;
On the basis of the learning that results learned by step, recommending the parameters to be set in a new three-dimensional point group data step,
A computer-readable program for executing the program.

Images