JPWO2018179424A1 - Point cloud processing system - Google Patents

Abstract

A system capable of accurately extracting or deleting a desired region from point cloud data is provided.
A point group processing system according to the present invention includes an aerial imaging apparatus, a controller, and a point group processing apparatus. The control unit 32 of the point cloud processing device 30 executes the acquisition module 321 to acquire 3D point cloud data including the color of the point and the 3D coordinates in the processing target captured by the aerial imaging device 10. Store in the three-dimensional point cloud database 331. Subsequently, the control unit 32 executes the candidate image display module 322, changes parameters for a plurality of candidates stored in the parameter storage area 332, and displays an image for parameters related to the analysis method of the point to be processed. Displayed on the unit 34. Then, the control unit 32 executes the reception module 323 to receive selection of a desired image from among the plurality of candidate images, and subsequently executes the selection result display module 324 to display the image that has received the selection.
[Selection] Figure 1

Description

  The present invention relates to a point cloud processing system, method, and program.

  A technique for photographing the sky with an unmanned air vehicle such as a drone and measuring the distance to the ground with a radar is known. An attempt has been made to acquire three-dimensional point cloud data of a field such as a construction site using this technique (see, for example, Patent Document 1). The point cloud data includes three-dimensional coordinate data and color data. Then, using these three-dimensional coordinate data and color data, a soil volume calculation for making the designed land is performed from an aerial image taken by an unmanned air vehicle.

JP 2006-194515 A

  By the way, after acquiring point cloud data, there is a need to extract or delete only a desired region. For example, when a certain vacant area is photographed, there is a need to extract only the forest part from the photographed vacant area image or to delete it. At that time, only the desired forest part is selected by image analysis. For example, it is difficult to select a predetermined region such as a forest appropriately even if artificial intelligence is selected by distinguishing the color or altitude of a point. There are many cases.

  The present invention has been made in view of such a demand, and an object of the present invention is to provide a system that makes it possible to accurately extract or delete a desired region from point cloud data.

  The present invention provides the following solutions.

The invention according to the first feature is a point cloud processing system for processing three-dimensional point cloud data,
Obtaining means for obtaining three-dimensional point cloud data including the color and three-dimensional coordinates of a point to be processed;
Parameter storage means for storing a plurality of candidates in advance for parameters relating to the analysis method of the point to be processed;
With respect to the three-dimensional point cloud data, parameters are changed according to each of a plurality of candidates stored in the parameter storage means, and the point cloud reflecting the change is imaged in a predetermined dimension for each of the plurality of candidates. Candidate image display means for displaying;
Receiving means for receiving selection of a desired image among a plurality of types of candidate images displayed on the candidate image display means;
There is provided a point group processing system comprising: a selection result display unit that displays an image received by the reception unit.

  According to the invention relating to the first feature, with respect to the three-dimensional point cloud data, parameters relating to the analysis method of the point to be processed are changed to display a plurality of candidates, and a desired image is displayed from the plurality of candidates. It can be displayed as a selection result. Therefore, it is possible to provide a system that can accurately extract or delete an area desired by a user from a point cloud image.

The invention according to the second feature is the invention according to the first feature,
Learning means for learning a correlation between the three-dimensional point cloud data and an image parameter received by the receiving means;
There is provided a point cloud processing system, further comprising: an updating unit that updates a candidate stored in the parameter storage unit based on a result learned by the learning unit.

  According to the second aspect of the invention, the correlation between the three-dimensional point cloud data and the parameter is learned, and the candidate stored in the parameter storage unit is updated based on the learning result. Therefore, it is possible to further increase the accuracy of accurately extracting or deleting a region desired by the user from the point cloud image.

  According to the present invention, it is possible to provide a system that can accurately extract or delete a region desired by a user from a point cloud image.

FIG. 1 is a block diagram showing a hardware configuration and software functions of a point cloud processing system 1 in the present embodiment. FIG. 2 is a flowchart showing a point cloud processing method according to this embodiment. FIG. 3 is an example of the parameter storage area 333. FIG. 4 is an example when a candidate image of 3D point cloud data is displayed on the image display unit 34. FIG. 5 shows an example when one image is selected from the plurality of candidate images shown in FIG.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

<Configuration of Point Cloud Processing System 1>
FIG. 1 is a block diagram for explaining the hardware configuration and software functions of a point cloud processing system 1 according to this embodiment. The point cloud processing system 1 is a system for processing 3D point cloud data.

  The point cloud processing system 1 is connected to an aerial imaging apparatus 10 that captures an imaging target, wirelessly communicates with the aerial imaging apparatus 10, a controller 20 that controls the aerial imaging apparatus 10, and the aerial imaging apparatus 10. And a point cloud processing device 30 for processing an image (three-dimensional point cloud data).

[Aerial Camera 10]
The aerial imaging device 10 is not particularly limited as long as it is a device that can shoot a subject to be photographed from the sky. For example, the aerial imaging apparatus 10 may be a radio controlled airplane or an unmanned aerial vehicle called a drone. In the following description, it is assumed that the aerial imaging apparatus 10 is a drone.

  The aerial imaging apparatus 10 is powered by a battery 11 that functions as a power source for the aerial imaging apparatus 10, a motor 12 that operates with electric power supplied from the battery 11, and the motor 12. And a rotor 13.

  The aerial imaging device 10 also includes a control unit 14 that controls the operation of the aerial imaging device 10, a position detection unit 15 that transmits position information of the aerial imaging device 10 to the control unit 14, and a control signal from the control unit 14. A driver circuit 16 for driving the motor 12, a camera 17 for taking an aerial image of an object to be photographed in accordance with a control signal from the control unit 14, a control program executed by the microcomputer of the control unit 14, and the like are stored in advance. 17 includes a storage unit 18 for storing an image taken by the camera.

  The aerial imaging apparatus 10 includes a wireless communication unit 19 that performs wireless communication with the controller 20.

  These components are mounted on a main body structure (frame or the like) having a predetermined shape. As for a main body structure (frame or the like) having a predetermined shape, a similar one to a known drone may be adopted.

[Battery 11]
The battery 11 is a primary battery or a secondary battery, and supplies power to each component in the aerial imaging apparatus 10. The battery 11 may be fixed to the aerial imaging apparatus 100 or may be detachable.

[Motor 12, rotor 13]
The motor 12 functions as a drive source for rotating the rotor 13 with electric power supplied from the battery 11. By rotating the rotor 13, the aerial imaging device 10 can be levitated and flying.

[Control unit 14]
The controller 14 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like.

  Further, the control unit 14 implements the photographing module 141 by reading a predetermined program.

  The control unit 14 controls the motor 12 to perform flight control (control of ascending, descending, horizontal movement, etc.) of the aerial imaging apparatus 10. The control unit 14 controls the attitude of the aerial imaging apparatus 10 by controlling the motor 12 using a gyro (not shown) mounted on the aerial imaging apparatus 10.

[Position detection unit 15]
The position detection unit 15 includes a LIDAR (Laser Imaging Detection and Ranging) technique and a GPS (Global Positioning System) technique. By using these LIDAR technology and GPS technology together, the latitude, longitude, and height (for example, the height of a tree) of each point constituting the three-dimensional point cloud data of the image captured by the camera 17 are detected.

[Driver circuit 16]
The driver circuit 16 has a function of applying a voltage designated by a control signal from the control unit 14 to the motor 12. As a result, the driver circuit 16 can drive the motor 12 in accordance with the control signal from the control unit 14.

[Camera 17]
The camera 17 has a function of converting (imaging) an optical image captured by a lens into an image signal by an imaging element such as a CCD or a CMOS. The type of the camera 17 may be selected as appropriate according to the image analysis method to be photographed.

[Storage unit 18]
The storage unit 18 is a device that stores data and files, and includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card. The storage unit 18 stores a control program storage area (not shown) for storing in advance a control program executed by the microcomputer of the control unit 14, and image data captured by the camera 17. 3D points to be stored together with the 3D coordinate data detected in (the data of the latitude, longitude, and height (eg, tree height) of each point constituting the 3D point cloud data of the image taken by the camera 17) A group data storage area (not shown) and the like are provided.

  The data stored in the three-dimensional point cloud data storage area can be transferred to the point cloud processing device 30 through a portable recording medium such as a USB memory or an SD card.

[Wireless communication unit 19]
The wireless communication unit 19 is configured to be able to perform wireless communication with the controller 20 and receives a remote control signal from the controller 20.

[Controller 20]
The controller 20 has a function of operating the aerial imaging apparatus 10. The controller 20 includes an operation unit 21 that is used by a user to steer the aerial imaging apparatus 10, a control unit 22 that controls the operation of the controller 20, a control program that is executed by a microcomputer of the control unit 22, and the like. A storage unit 23 to be stored, a wireless communication unit 24 that wirelessly communicates with the aerial imaging apparatus 10, and an image display unit 25 that displays a predetermined image to the user are provided.

  The wireless communication unit 24 is configured to be able to wirelessly communicate with the aerial imaging apparatus 10 and receives a remote control signal toward the aerial imaging apparatus 10.

  The image display unit 25 may be integrated with a control device that controls the aerial imaging apparatus 10, or may be a separate body from the control device. If integrated with the control device, the number of devices used by the user can be reduced, and convenience is enhanced. When the control device is separate from the control device, examples of the image display unit 25 include portable terminal devices such as smartphones and tablet terminals that can be wirelessly connected to the wireless communication unit 19 of the aerial imaging device 10. When the control device is separate from the control device, an existing control device that does not have the image display unit 25 can be applied.

[Point Cloud Processing Device 30]
The point cloud processing device 30 has a function of processing three-dimensional point cloud data of a photographed image photographed using the camera of the aerial imaging device 10. The point cloud processing device 30 includes an input unit 31 for a user to input command information, a control unit 32 for controlling the operation of the point cloud processing device 30, a control program executed by a microcomputer of the control unit 32, and the like. Is stored in advance, and an image display unit 34 for displaying a predetermined image to the user.

  The control unit 32 implements an acquisition module 321, a candidate image display module 322, a reception module 323, a selection result display module 324, a learning module 325, and an update module 326 by reading a predetermined program.

  The storage unit 33 stores a 3D point cloud database 331 that stores the 3D point cloud data stored in the storage unit 18 of the aerial imaging apparatus 10 transferred through a portable recording medium such as a USB memory or an SD card. And a parameter storage area 332 in which a plurality of candidates are stored in advance and parameters of the three-dimensional point cloud data are changed according to each of the plurality of candidates for parameters relating to the analysis method of the point to be processed. A candidate image storage area 333 for temporarily storing candidate images to be stored, a selected image storage area 334 for storing images selected from the candidate images, and the like are provided.

<Flowchart showing a point cloud processing method using the point cloud processing system 1>
FIG. 2 is a flowchart showing a point cloud processing method using the point cloud processing system 1. The processing executed by each hardware and the software module described above will be described.

[Step S10: Acquisition of Captured 3D Point Cloud Data]
First, the control unit 14 of the aerial imaging apparatus 10 of the point cloud processing system 1 executes the imaging module 141 and controls the camera 17 to image the imaging target. Then, the control unit 14 converts the image data captured by the camera 17 into the three-dimensional coordinate data detected by the position detection unit 15 (the latitude of each point constituting the three-dimensional point cloud data of the image captured by the camera 17, The longitude and height (for example, the height of a tree) are stored in a three-dimensional point cloud data storage area (not shown) of the storage unit 18.

  The information stored in the 3D point cloud data storage area is stored in the 3D point cloud database 331 of the point cloud processing device 30 via the recording medium after the aerial imaging device 10 has landed. In this way, the control unit 32 of the point cloud processing device 30 executes the acquisition module 321 to acquire 3D point cloud data including the color and 3D coordinates of the point to be processed.

[Step S11: Parameter change of 3D point cloud data]
Subsequently, the control unit 32 of the point cloud processing device 30 executes the candidate image display module 322, and a plurality of parameters related to the analysis method of the point to be processed are stored in advance in the parameter storage area 332 of the storage unit 33. In accordance with each of the candidates, the parameter change for the three-dimensional point cloud data is performed. The candidate image data obtained by the change is temporarily set in the candidate image storage area 333.

  FIG. 3 shows an example of the parameter storage area 332. In the parameter storage area, three candidates are set for four types of parameters (forest detection intensity, forest removal intensity, height resolution, and color resolution). In the present embodiment, the parameter can be set in the range of 0 to 100.

  The forest detection intensity is an intensity at which the control unit 32 detects the forest based on the point height (for example, tree height) and color for each point constituting the three-dimensional point cloud data. means. When the intensity is low, the control unit 32 has a region 101 where the point indicates a forest only when the point height (for example, the height of a tree or crop) is high and the color is dark green (see FIG. 4). Recognize that In other cases, the area 102 is recognized as a flat area 102 (see FIG. 4). When the intensity is high, the control unit 32 is a region where the point indicates a forest even when the height of the point (for example, the height of a tree or the height of a crop) is low or the color is yellow-green. 101 (see FIG. 4), and when the color is earthy, it is recognized as an area 102 (see FIG. 4) indicating a flat ground.

  Deforestation intensity means the extent of excluding the region 101 (see FIG. 4) indicating the forest from the three-dimensional point cloud data. When the intensity is high, the control unit 32 has a high point height (for example, a tree height) and a low point height (for example, a tree height) as well as a region having a dark green color. In this case, the region whose color is yellowish green is also changed from the display state to the non-display state as the region 101 (see FIG. 4) indicating the forest. When the intensity is low, the control unit 32 displays the region 101 (see FIG. 4) indicating the forest only for the region where the height of the point (for example, the height of the tree) is high and the color is dark green. Change from state to hidden state.

  For example, a field or a vacant land with grass is not necessarily a region 101 (see FIG. 4) indicating a forest even if the color is green. In order to prevent the region 102 (see FIG. 4) showing the forest from being mistakenly recognized even though it is actually the region 102 (see FIG. 4) showing the flat ground, Of particular importance.

  In the present embodiment, the area desired by the user (here, the area 101 indicating the forest (see FIG. 4)) is described as being deleted from the 3D point cloud data. It is also possible to set to extract a region desired by the user.

  The height resolution means a function for adjusting the accuracy of the height of a point (for example, the height of a tree). If the accuracy is high, the height of a point (for example, the height of a tree) is determined in centimeters. If the accuracy is low, the height of the point (eg, the height of the tree) is determined in meters.

  The color resolution means a function for determining how far a color is judged with respect to a color. When the color resolution is high, for example, the RGB value is determined in one unit. When the color resolution is low, for example, the RGB value is determined in units of 10 units.

[Step S12: Candidate Image Display of 3D Point Cloud Data]
Returning to FIG. When the parameter change is performed on the three-dimensional point cloud data, the control unit 32 of the point cloud processing device 30 subsequently executes the candidate image display module 322, and sets the point cloud reflecting the parameter change as a plurality of candidates. Are displayed on the image display unit 34 in a predetermined dimension.

  The dimension of the image to be displayed is not particularly limited. For example, the image display unit 34 may display a two-dimensional image as a two-dimensional image or three-dimensionally as a three-dimensional image.

  FIG. 4 is an example when the point group reflecting the parameter change is displayed on the image display unit 34 two-dimensionally for each of the three candidates. The three-dimensional point cloud data includes color information. In the example of FIG. 4, the two-dimensional image includes a region 101 indicating a forest and a region 102 indicating a flat ground.

  On the left side of the image display unit 34, an original image (an image before changing parameters) is displayed. On the right, various parameters are changed under the conditions stored in the three candidate images after the parameter change, that is, the parameter storage area 332, and the green area 101 indicating the forest is changed from the display state to the non-display state. The image is shown.

  Candidate 1 has four types of parameters (forest detection intensity, forest removal intensity, height resolution, and color resolution) set relatively low. For this reason, the ratio of the area to be changed from the display state to the non-display state is small. Here, description will be made assuming that even the region 101 that actually indicates a forest is left as a display state.

  Candidate 2 has four types of parameters set on average. Therefore, the ratio of the area to be changed from the display state to the non-display state is larger than that of candidate 1. Here, description will be made assuming that the region 101 indicating the forest is actually in a non-display state.

  Candidate 3 has four types of parameters set relatively high. For this reason, the ratio of the area to be changed from the display state to the non-display state is large. Here, it is assumed that not only the region 101 indicating the forest but also the region 102 actually indicating the flat ground is partially hidden.

[Step S13: Accept selection of desired image]

  Returning to FIG. Subsequently, the control unit 32 executes the reception module 323 and receives selection of a desired image among a plurality of types of candidate images displayed on the image display unit 34.

  Referring to FIG. 4, the cursor 103 is also displayed on the image display unit 34. A user can select a desired image by placing the cursor 103 on one of candidates 1 to 3 and clicking. Here, it is assumed that the user places the cursor 103 on candidate 2 and clicks.

[Step S14: Display the selected image]

  Returning to FIG. Subsequently, the control unit 32 executes the selection result display module 324 and displays the image selected in the process of step S13 on the image display unit 34.

  The control unit 32 moves the data of the image selected in the process of step S13 among the plurality of candidate images set in the candidate image storage area 333 to the selection result storage area 334, and stores the data of the unselected image. delete. Then, the control unit 32 displays the image set in the selection result storage area 334 on the image display unit 34.

  FIG. 5 shows a display example at that time. Since the user places the cursor 103 on the candidate 2 and clicks, the image of the candidate 2 is displayed large on the image display unit 34. In the display example of FIG. 4, the area 101 indicating the forest that has been displayed is not displayed, and only the area 102 indicating the flat ground is displayed. At that time, not only the area where the ground surface is exposed, but also the area where the color is green, such as a vacant area where fields and grass grow, is actually a flat area instead of a forest, as the area 102 indicating a flat area. It is displayed correctly.

  According to the present embodiment, for the three-dimensional point cloud data, a parameter regarding the analysis method of the point to be processed is changed and a plurality of candidates are displayed as images, and a desired image is displayed as a selection result from the plurality of candidates. can do. Therefore, it is possible to provide the point cloud processing system 1 capable of accurately extracting or deleting a region desired by the user from the point cloud image.

[Step S15: Learn the correlation between 3D point cloud data and parameters]
Although not essential, in order to improve the accuracy of the candidate image, the control unit 32 of the point cloud processing device 30 executes the learning module 325, and sets the three-dimensional point cloud data and the parameters of the candidate image selected in the process of step S13. It is preferable to learn the correlation with.

[Step S16: Parameter update]
And the control part 32 performs the update module 326, and updates the candidate memorize | stored in the parameter storage area | region 332 based on the result learned by the process of step S15. And when performing the process of step S11 again, the control part 32 of the point cloud processing apparatus 30 performs the change process of three-dimensional point cloud data based on the parameter memorize | stored in the parameter storage area | region 332 after an update. .

  According to the present embodiment, the correlation between the three-dimensional point cloud data and the parameters is learned, and the candidates stored in the parameter storage area 332 are updated based on the learning result. Therefore, it is possible to further increase the accuracy of accurately extracting or deleting a region desired by the user from the point cloud image.

  The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.), for example. In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

1 point group processing system 10 aerial imaging device 11 battery 12 motor 13 rotor 14 control unit 141 imaging module 15 position detection unit 16 driver circuit 17 camera 18 storage unit 19 wireless communication unit 20 controller 30 point group processing device 31 input unit 32 control unit 321 Acquisition module 322 Candidate image display module 323 Acceptance module 324 Selection result display module 325 Learning module 326 Update module 33 Storage unit 331 Three-dimensional point cloud database 332 Parameter setting area 333 Candidate image storage area 333 Selection result storage area 34 Image display section

Claims (4)

A point cloud processing system for processing 3D point cloud data,
Obtaining means for obtaining three-dimensional point cloud data including the color and three-dimensional coordinates of a point to be processed;
Parameter storage means for storing a plurality of candidates in advance for parameters relating to the analysis method of the point to be processed;
With respect to the three-dimensional point cloud data, parameters are changed according to each of a plurality of candidates stored in the parameter storage means, and the point cloud reflecting the change is imaged in a predetermined dimension for each of the plurality of candidates. Candidate image display means for displaying;
Receiving means for receiving selection of a desired image among a plurality of types of candidate images displayed on the candidate image display means;
A point group processing system comprising: a selection result display unit that displays an image received by the reception unit. Learning means for learning a correlation between the three-dimensional point cloud data and an image parameter received by the receiving means;
The point cloud processing system according to claim 1, further comprising: an updating unit that updates a candidate stored in the parameter storage unit based on a result learned by the learning unit. A point cloud processing method for processing 3D point cloud data,
Obtaining 3D point cloud data including the color and 3D coordinates of a point to be processed;
Regarding the parameters related to the analysis method of the point to be processed, the parameters of the three-dimensional point cloud data are changed according to each of a plurality of candidates stored in advance, and a plurality of point clouds reflecting the change are obtained. Displaying an image in a predetermined dimension for each of the candidates,
Receiving a selection of a desired image among a plurality of types of candidate images displayed as images;
A point cloud processing method comprising: displaying an image that has received the selection. In the point cloud processing system,
Obtaining 3D point cloud data including the color and 3D coordinates of a point to be processed;
Regarding the parameters related to the analysis method of the point to be processed, the parameters of the three-dimensional point cloud data are changed according to each of a plurality of candidates stored in advance, and a plurality of point clouds reflecting the change are obtained. Displaying an image in a predetermined dimension for each of the candidates,
Receiving a selection of a desired image among a plurality of types of candidate images displayed as images;
Displaying an image that has received the selection;
A program for running

Images