JP2024021143A - 3D data generation system and 3D data generation method - Google Patents

Abstract

The present invention provides a three-dimensional data generation system and a three-dimensional data generation method that can reduce the time required to generate three-dimensional model data based on image data. [Solution] A moving object capable of photographing a target area with a photographing device and generating video data including shooting time information, and a position data generator capable of detecting the position of the moving object and generating position data including detection time information. a system, and an information processing device that receives video data and position data, and the information processing device performs a dividing process that divides the received video data into predetermined time lengths to generate a plurality of short-time video data. , a generation process of generating three-dimensional point cloud data of at least a part of the target area corresponding to the short-time video data, based on the short-time video data and the position data of the moving object corresponding to the shooting time of the short-time video data; and transmitting at least some of the generated three-dimensional point group data to a display device. [Selection diagram] Figure 1

Description

The present disclosure relates to a three-dimensional data generation system and a three-dimensional data generation method.

In recent years, a method of generating three-dimensional model data using images taken by an unmanned flying vehicle such as a drone has been known. Further, Patent Document 1 discloses a method of generating three-dimensional model data by SfM (Structure from Motion) processing or the like based on a plurality of image data.

JP 2021-177134 Publication

By the way, since it takes time to generate three-dimensional model data based on a plurality of image data taken with a camera, there is a problem that there is a large time lag from the time the images are taken to the time when the three-dimensional model data is generated and checked.

Therefore, the present disclosure has been made in view of the above problems, and its purpose is to provide a three-dimensional data generation system and three-dimensional data generation system that can reduce the time required to generate three-dimensional model data based on image data. The object of the present invention is to provide a dimensional data generation method.

According to this disclosure:
a moving object capable of photographing a target area with a photographing device and generating video data including photographing time information;
a position data generation system capable of detecting the position of the mobile object and generating position data including detection time information;
an information processing device that receives the video data from the mobile object and the position data from the position data generation system,
The information processing device includes:
a dividing process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data;
Generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data based on the short-time video data and the position data of the moving object corresponding to the shooting time of the short-time video data. generation processing,
A three-dimensional data generation system is provided that executes a process of transmitting at least a portion of the generated three-dimensional point group data to a display device.

Further, according to the present disclosure, a mobile object capable of photographing a target area with a photographing device and generating video data including photographing time information;
a position data generation system capable of detecting the position of the mobile object and generating position data including detection time information;
A method of generating three-dimensional point group data using an information processing device that receives the video data from the moving object and the position data from the position data generation system, the method comprising:
The information processing device includes:
a dividing process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data;
Generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data based on the short-time video data and the position data of the moving object corresponding to the shooting time of the short-time video data. generation processing,
A three-dimensional data generation method is provided, which executes a process of transmitting at least part of the generated three-dimensional point group data to a display device.

According to the present disclosure, it is possible to provide a three-dimensional data generation system and a three-dimensional data generation method that can reduce the time required to generate three-dimensional model data based on image data.

1 is a schematic diagram showing an example of a three-dimensional data generation system according to an embodiment of the present disclosure. It is a diagram showing an example of the hardware configuration of an unmanned flying vehicle according to the same embodiment. FIG. 2 is a block diagram illustrating a configuration example of an information processing apparatus according to the embodiment. It is a flowchart which shows an example of the flow of the three-dimensional data generation method based on the same embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals and redundant explanation will be omitted.

<Summary>
FIG. 1 is a schematic diagram illustrating an example of a system according to an embodiment of the present disclosure. This system 100 includes an unmanned flying vehicle (drone) 1 equipped with a photographing device for photographing a target area, an information processing device 20, a motion capture device 30 (position data generation system), and a display device 40. The unmanned aerial vehicle 1 is capable of communicating with at least the information processing device 20 . The unmanned aerial vehicle 1 may be configured to be able to communicate with the motion capture device 30 and the display device 40, or may be configured to be unable to communicate with the motion capture device 30 and the display device 40. The motion capture device 30 can communicate with at least the information processing device 20, and may or may not be able to communicate with the unmanned aerial vehicle 1 and the display device 40. Note that in the diagnostic method of this embodiment, the imaging device used to acquire images is not limited to a camera mounted on an unmanned flying vehicle, but may also be an imaging device mounted on another moving object, or a single camera. Any image acquisition device may be used. The position data generation system capable of detecting the position of a moving object and generating position data including detection time information is not limited to the motion capture device 30, and other position estimation systems may be used. For example, a system that uses V-SLAM (Visual Simultaneous Localization and Mapping) to estimate the position based on images taken by a camera on a moving object, a system that receives a signal emitted from a beacon and estimates the position based on the intensity of the signal emitted by the moving object, or , a system using a laser tracker that estimates the position by irradiating a laser onto a reflector provided on a moving body and receiving the reflected light, etc. can be used. Furthermore, such a position estimation system may be partially or entirely provided in the unmanned flying vehicle 1, or may be provided outside the unmanned flying vehicle 1.

<Unmanned aerial vehicle>
The unmanned aerial vehicle 1 photographs a target area with a photographing device and generates video data including photographing time information. The unmanned flying object 1 flies over an area to be photographed for the purpose of inspection, work progress management, and the like. The subject to be photographed is not particularly limited, and may be an artificial building, factory, equipment, etc., or an internal space or underground space of a structure such as a cave that exists in the natural world. This system can be used in both outdoor and indoor spaces.

The unmanned flying object (drone) 1 according to the present embodiment is a rotary-wing aircraft that obtains lift and thrust using a plurality of rotary wings 3, and captures image data by photographing an area to be photographed with a camera while flying. get. The unmanned flying object 1 may fly autonomously based on a preset program, or may be controlled in flight by a user operating from inside or outside the space to be photographed. Alternatively, the flight may be controlled by a combination of these.

FIG. 2 is a diagram showing an example of the hardware configuration of the unmanned flying vehicle 1 according to the present embodiment. As shown in FIG. 2, the unmanned flying vehicle 1 according to the present embodiment includes a main body 2, a rotor 3, a motor 4, a camera 5 (photographing device), and a sensor 6. The unmanned flying vehicle 1 also includes a flight controller 11 , a battery 14 , an ESC (Electric Speed Controller) 15 , and a transmitting/receiving section 16 in the main body section 2 . Note that the configuration of the unmanned flying object 1 shown in FIG. 2 is an example, and even a rotary wing aircraft having a configuration different from that of the main body 2 shown in FIG. 2 may be included in the scope of the present invention.

The main body portion 2 is formed by a frame and the like that constitute the unmanned aerial vehicle 1 . The material constituting the main body portion 2 is not particularly limited, and may be, for example, carbon fiber resin, glass fiber resin, magnesium, magnesium alloy, aluminum, aluminum alloy, steel, titanium, or other materials. The rotor 3 is attached to a motor 4. The rotor blade 3 generates lift (thrust) to the unmanned flying vehicle 1 by rotating itself due to the rotation of the motor 4 . The rotary blade 3 and the motor 4 are an example of a thrust generating section. In this embodiment, the rotary blades 3 are provided at four locations: front, rear, left, and right, but the present invention is not limited to this example. The number of rotary wings 3 provided may be changed as appropriate depending on the structure, shape, equipment, size, etc. of the unmanned flying vehicle 1.

The flight controller 11 can have one or more processors 13, such as a central processing unit (CPU) or a programmable processor such as a field-programmable gate array (FPGA). The flight controller 11 has a memory 12 and can access the memory 12. Memory 12 stores logic, code, and/or program instructions executable by flight controller 11 to perform one or more steps. Flight controller 11 is an example of a control device. The control device generates video data including photographing time information by photographing the target area with the photographing device. Video data can be data in which a large number of image data (still image data) are continuously connected by continuous shooting over a predetermined period of time. The control device can perform image processing on image data such as video data captured by the camera 5. Image processing is, for example, processing to reduce data volume by lowering image quality, reducing processing time, etc., but is not limited to this, and may also include adding other information. Moreover, such a control device may be provided in the control terminal 17 or may be provided in a relay device (control device) that communicates with the unmanned aerial vehicle 1 and the information processing device 20. For example, when a relay device is provided, the relay device receives image data taken by a drone, reduces the amount of data by performing image processing such as image quality reduction, and then transmits it to the information processing device 20. You can.

Memory 12 may include, for example, a separable medium or external storage such as an SD card or random access memory (RAM). Data acquired from camera 5 and sensor 6 may be directly communicated to and stored in memory 12. For example, image data (still image data/video data) captured by the camera 5 is recorded in the built-in memory or external memory. Further, image data captured by the camera 5 is transmitted to the control terminal 17, the information processing device 20, the display device 40, etc. via the transmitting/receiving section 16.

Flight controller 11 includes a control module configured to control the state of unmanned air vehicle 1 . For example, the control module may be configured to adjust the spatial position, velocity, and/or acceleration of the unmanned air vehicle 1 with six degrees of freedom (translational movements x, y, and z, and rotational movements θx, θy, and θz). The motor 4, which is the propulsion mechanism of the unmanned flying vehicle 1, is controlled via the ESC 15. Rotating the rotary blade 3 by the motor 4 generates lift of the unmanned flying vehicle 1. The flight controller 11 can control the number of rotations of the motor 4 (the number of rotations also means the number of rotations per predetermined time) to adjust the thrust by the rotary blade 3.

The flight controller 11 is capable of communicating with a transceiver 16 configured to transmit and/or receive data from one or more external devices (e.g., a pilot terminal 17, an information processing device 20, a motion capture device 30). It is. The transmitting/receiving unit 16 can use any suitable communication means such as wired communication or wireless communication. The transmitter/receiver 16 connects the control device such as the flight controller 11 to a network such as the Internet, and includes, for example, a local area network (LAN), wide area network (WAN), infrared, wireless, WiFi, point-to-point network, etc. One or more of arbitrary communication methods such as point-to-point (P2P) network, telecommunications network, cloud communication, LTE, Bluetooth (registered trademark), and BLE (Bluetooth Low Energy) can be used.

The transmitter/receiver 16 transmits one or more of data captured by the camera 5, data acquired by the sensor 6, processed data generated by the flight controller 11, predetermined control data, user commands from a terminal or remote controller, etc. can be sent and/or received. The information obtained by the camera 5, the sensor 6, etc. may be output to the server (information processing device 20), the control terminal 17, the display device 40, etc. via the transmitter/receiver 16. The transmitter/receiver 16 can receive information from the motion capture device 30.

The control terminal 17 is a device for controlling the flight of the unmanned flying object 1 . Note that the flight of the unmanned flying object 1 may be controlled by an operator on the ground or by automatic piloting based on an autonomous flight program (for example, GCS (Ground Control Station)) based on flight path information and sensing. , or may be partially controlled by manual operation. The control terminal 17 may be, for example, a transmitter/receiver (propo), a smartphone, a tablet, or other terminal. The pilot terminal 17 can send flight control instruction information to the flight controller 11.

The sensor 6 according to the present embodiment is, for example, an inertial sensor, an acceleration sensor, a gyro sensor, a GPS sensor, a wind sensor, a temperature sensor, a humidity sensor, a barometric pressure sensor, an altitude sensor, a LiDAR (Laser Imaging Detection and Ranging), etc. or a vision/image sensor other than a camera. Further, the sensor 6 may be mounted on the flight controller 11 or may be provided outside the flight controller 11. Further, the camera may be any camera as long as it can acquire image data. For example, the camera 5 may be an infrared camera, a stereo camera, etc. in addition to a general camera, and may be provided integrally with any sensor.

<Information processing device>
As shown in FIG. 1, the information processing device 20 can communicate with the unmanned flying vehicle 1, the control terminal 17, the motion capture device 30, and the display device 40, and can send and receive various information. The information processing device 20 is a server device (for example, a cloud server) that acquires image data photographed by the unmanned aerial vehicle 1 and generates three-dimensional data of the object to be photographed. The information processing device 20 can output the generated three-dimensional data to the display device 40 for display. The display device 40 may be configured with a monitor, touch panel, etc. connected to the information processing device 20 by wire or wirelessly, or may be a general-purpose computer, smartphone, or tablet connected to the information processing device 20 via a network such as the Internet. It may also be a mobile terminal such as a PC.

FIG. 3 is a block diagram showing a configuration example of the information processing device 20 according to the present embodiment. As illustrated, the information processing device 20 includes a control section 21. The information processing device 20 may be, for example, a general-purpose computer such as a workstation or a personal computer, or a mobile terminal such as a smartphone or a tablet PC.

The processor 21a is an arithmetic device that controls the operation of the control unit 21, controls the transmission and reception of data between each element, and performs processing necessary for executing a program. In this embodiment, the processor 21a is, for example, a CPU (Central Processing Unit), and performs various processes by executing a program stored in a storage 21c and developed in a memory 21b, which will be described later.

The memory 21b includes a main storage device made up of a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary storage device made up of a nonvolatile storage device such as a flash memory or an HDD (Hard Disc Drive). . This memory 21b is used as a work area for the processor 21a, and also stores a boot loader executed when the control unit 21 is started, various setting information, and the like.

The storage 21c stores programs and information used in various processes. For example, the storage 21c may store a program for displaying the flight status, inspection status, etc. of the unmanned flying vehicle. The storage 21c also contains information regarding image data taken of the target area, information regarding the generated three-dimensional data, information regarding the position and attitude of the flying object acquired from the unmanned aerial vehicle, motion capture device, etc., and deformation of the target area. Information regarding the amount, information regarding the transition pattern of change over time, etc. may be stored.

The transmitting/receiving unit 22 connects the control unit 21 to a network such as the Internet, and includes a local area network (LAN), wide area network (WAN), infrared, wireless, WiFi, point-to-point (P2P) network, It may also include a communication interface such as a telecommunications network, cloud communication, LTE, Bluetooth (registered trademark), or BLE (Bluetooth Low Energy). Further, the transmitter/receiver 22 may include a communication interface for wired communication.

The input/output unit 23 is an interface to which an input/output device is connected, and in this embodiment, for example, a display device may be connected.

The bus 24 transmits, for example, address signals, data signals, and various control signals between the connected processor 21a, memory 21b, storage 21c, transmitting/receiving section 22, and input/output section 23.

The control unit 21 of the information processing device 20 according to this embodiment functions as an image acquisition unit, a position information acquisition unit, a three-dimensional data generation unit, and a data distribution unit.

The image acquisition unit executes a process of acquiring image data of a region to be photographed. For example, an image acquisition process is executed to acquire image data (video data, still image data) of a predetermined range of areas including the equipment to be inspected. Specifically, the image data captured by the camera 5 of the unmanned aerial vehicle 1 is transmitted to the information processing device 20 via the transmitting/receiving section 16 (and the control terminal 17 as necessary), so that the image acquiring section acquires image data. The image acquisition unit may store the image data in the storage 21c. The image data may include information regarding the time when the image data was photographed, and information regarding the position and orientation (photographing direction) of the camera at the time of the photographing.

The position information acquisition unit acquires position data of the unmanned aerial vehicle generated by the motion capture device 30 (position data generation system). The position data includes the position (three-dimensional coordinates) of the unmanned flying object detected by the motion capture device and information on the time when the position was detected (detection time information).

The three-dimensional data generation unit performs a division process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data, and corresponds to the short-time video data and the shooting time of the short-time video data. A generation process of generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data is executed based on the position data of the moving object. The data distribution unit executes a process of transmitting at least part of the generated three-dimensional point cloud data to a display device. Such division processing, three-dimensional point cloud data generation processing, and transmission processing to a display device are repeatedly executed. Thereby, the three-dimensional point group data based on the image data photographed by the unmanned flying vehicle 1 can be displayed on the display device in a substantially real-time manner. When generating three-dimensional model data, coordinate information included in the image data may be associated with coordinate information based on a known marker or the like. The three-dimensional model data is, for example, three-dimensional point group data, but is not limited to this. The three-dimensional model data can be three-dimensional point group data of the object included in the target area. The three-dimensional data generation process may include, for example, SfM processing and MVS (Multi-view Stereo) processing, or may generate three-dimensional data using other processing methods. The three-dimensional data generation unit can generate three-dimensional model data of the target area based on SfM processing using a plurality of image data. SfM processing is a process that generates a three-dimensional shape of an object on virtual data using multiple image data of a common object represented by multiple feature points taken from multiple camera positions. Then, the position data of the unmanned aerial vehicle at the time when each image data acquired from the motion capture device was captured is referred to. The position data includes information on the position of the camera (photographing position (three-dimensional coordinates)) and the orientation of the camera (photographing direction). The three-dimensional data generation unit may store all or part of the information regarding the generated three-dimensional data in the storage 21c. In this way, by using the position data of the unmanned aerial vehicle acquired by the motion capture device, the object to be photographed can be placed at the appropriate position (coordinates) and in the appropriate orientation in the generated 3D data. Become.

<Motion capture device>
The motion capture device 30 includes a plurality of motion capture cameras 31 arranged around the flight space of the unmanned aerial vehicle. The plurality of motion capture cameras 31 can be connected to each other by wire or wirelessly and operate in cooperation. The motion capture device 30 detects and detects the position of the unmanned aerial vehicle 1 by, for example, monitoring the position and movement of one or more retroreflective markers placed on the outer surface of the unmanned aerial vehicle 1 with a motion capture camera 31. Location data including time information can be generated. The position data includes three-dimensional coordinate information on the coordinate system of the motion capture device 30 (relative position and orientation to the real space coordinate system are known) and time information corresponding to the coordinate information. The position data may include information on the orientation (azimuth) of the unmanned aerial vehicle. The orientation information calculates the orientation of the unmanned aerial vehicle based on the positional relationship of multiple retroreflective markers on the unmanned aerial vehicle (for example, the direction of a vector connecting the three-dimensional coordinates of two specific retroreflective markers). can do. The motion capture device 30 has a control unit and a transmitting/receiving unit that are built into the motion capture camera 31 or connected to the outside, and repeatedly performs a process of generating position data including detection time information and a process of transmitting the position data. be able to. The configurations of the control unit and the transmitting/receiving unit are substantially the same as those of the control unit 21 and the transmitting/receiving unit 22 of the information processing device 20, so the description thereof will be omitted. By repeatedly generating and transmitting the position data of the unmanned air vehicle, the motion capture device 30 can transmit the position data of the unmanned air vehicle to the information processing device 20 and the like in substantially real time. The motion capture device 30 can also transmit position data to an unmanned flying vehicle. When estimating the unmanned aerial vehicle's self-position, by referring to the position data from the motion capture device 30 (without referring to the position data, self-positioning based only on the unmanned aerial vehicle's sensor information and GNSS signals). The accuracy of self-position estimation can be improved (compared to the case where self-position is estimated).

FIG. 4 shows the flow when the information processing device 20 generates three-dimensional point cloud data and displays it on the display device 40 based on the video data from the unmanned aerial vehicle 1 and the position data from the motion capture device 30. show.

The unmanned aerial vehicle 1 flies by manual control using the control terminal 17, or autonomously flies along a preset route according to an autonomous flight program, and photographs the target area from above while in flight. Video data including information is generated and transmitted to the information processing device (S101). At the same time, the motion capture device 30 detects (measures) the position of the unmanned flying object 1 in flight, generates position data including detection time information, and transmits it to the information processing device 20 (S102). The generation and transmission processing of video data by the unmanned aerial vehicle 1 and the generation and transmission processing of position data by the motion capture device 30 are continuously and repeatedly executed while the unmanned aerial vehicle 1 is photographing. Note that before the unmanned aerial vehicle 1 starts photographing, the motion capture camera 31 is installed in advance around the region to be photographed.

The information processing device 20 executes a division process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data (S103). For example, if the received video data is shooting data with a length of 10 seconds, it is possible to generate 10 pieces of short-time video data of 1 second, or 2 pieces of short-time video data of 5 seconds. . Basically, short-time video data of the same length is generated by equal division, but short-time video data of different time lengths may be generated. Each short-time video data includes shooting time information.

Next, the information processing device 20 determines the target area corresponding to the short-time video data based on the generated short-time video data and the position data of the unmanned aerial vehicle 1 (mobile object) corresponding to the shooting time of the short-time video data. (S104). For example, for each generated short-time video data, three-dimensional point cloud data of the object to be photographed is generated by SfM processing using a plurality of image data included in the short-time video data. In addition, at that time, the position data of the unmanned aerial vehicle at the detection time that coincides with the shooting time of each short video data is used to estimate the position of the unmanned aerial vehicle (that is, the position and orientation of the camera), and Based on the position information of the flying object, the position of the object in the virtual model space of the three-dimensional model data is determined (corrected). Note that when determining the position of the object on the virtual model space, the origin of the coordinate system of the motion capture device 30 and the direction of each axis are positioned relative to the coordinate system of the virtual model space corresponding to the real space. Match.

The three-dimensional data generated by the information processing device 20 is transmitted to the display device 40 via a network such as the Internet, and is displayed on the screen of the display device 40 (S105). The user can check the target area and the three-dimensional shape of the target object via the display device 40 from any location connected to the network. Such video data division (time reduction) processing, three-dimensional point cloud data generation processing, and transmission processing to a display device are continuously and repeatedly executed.

As described above, the three-dimensional data generation system of this embodiment detects a moving object capable of photographing a target area with a photographing device and generating video data including photographing time information, and detecting the position of the mobile object. The information processing device includes a motion capture device that can generate position data including detection time information, and an information processing device that receives video data from a moving object and position data from the motion capture device. A dividing process that divides data into predetermined time lengths to generate multiple short-time video data, and a short-time video data based on the short-time video data and the position data of the moving object corresponding to the shooting time of the short-time video data. A generation process of generating three-dimensional point cloud data of at least a portion of a target area corresponding to video data, and a process of transmitting at least a portion of the generated three-dimensional point cloud data to a display device are executed. In this way, by repeating the process of dividing video data into multiple pieces of short-time video data and generating 3D data, it is possible to generate 3D point cloud data in a shorter time than when creating 3D point cloud data without dividing video data. It becomes possible to generate three-dimensional data in time and display it on a display device. Therefore, the user can confirm the three-dimensional shape of the object photographed by the unmanned flying vehicle 1 substantially in real time.

Further, the three-dimensional data generation system of this embodiment may further include a relay device that receives video data sent from the unmanned flying object 1 (mobile object) and transmits the video data to the server. . In this case, since the video data can be transmitted to a relay device that is closer to the unmanned aerial vehicle 1 than the information processing device 20, the video data can be directly transmitted from the unmanned aerial vehicle 1 to the information processing device 20. Compared to the case of transmission, the processing load on the unmanned aerial vehicle 1 can be reduced, and the communication capacity can be kept low.

Furthermore, in the three-dimensional data generation system of this embodiment, the moving object may lower the image quality of the video data shot by the imaging device before transmitting it to the server device. In this way, by lowering the image quality of the video data, the amount of data to be transmitted can be reduced, so the load associated with communication can be reduced and the communication time can be shortened.

Although preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. It is understood that these also naturally fall within the technical scope of the present disclosure.

In addition, in the said embodiment, although the autonomous flight control was demonstrated as being performed by the flight controller 11 of the unmanned flying object 1, this technique is not limited to such an example. That is, such an autonomous flight control method is not limited to an example in which processing is performed at the edge of an unmanned flying vehicle, but also involves performing the above-described correction processing remotely by another autonomous flight control device, transmitting the processing result to the unmanned flying vehicle, The drive unit may be controlled based on such results. In other words, the hardware that executes this autonomous flight control method is not particularly limited, and the above-described functional units may be executed by a plurality of pieces of hardware.

Further, the effects described in this specification are merely explanatory or illustrative, and are not limiting. In other words, the technology according to the present disclosure can have other effects that are obvious to those skilled in the art from the description of this specification, in addition to or in place of the above effects.

Note that the following configurations also belong to the technical scope of the present disclosure.
(Item 1)
A flying object capable of photographing a target area with a photographing device and generating video data including photographing time information;
a position data generation system capable of detecting the position of an aircraft and generating position data including detection time information;
an information processing device that receives the video data from the flying object and the position data from the position data generation system,
The information processing device includes:
a dividing process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data;
Generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data based on the short-time video data and the position data of the flying object corresponding to the shooting time of the short-time video data. generation processing,
A three-dimensional data generation system that executes a process of transmitting at least part of the generated three-dimensional point group data to a display device.
(Item 2)
The three-dimensional data generation system according to item 1, further comprising a relay device that receives the video data sent from the flying object and transmits the video data to the server.
(Item 3)
The three-dimensional data generation system according to claim 1 or 2, wherein the flying object lowers the image quality of the video data before transmitting it to the server device.
(Item 4)
The three-dimensional data generation system according to any one of claims 1 to 3, wherein the position data generation system is a motion capture device.
(Item 5)
A flying object capable of photographing a target area with a photographing device and generating video data including photographing time information;
a position data generation system capable of detecting the position of an aircraft and generating position data including detection time information;
A method of generating three-dimensional point group data using an information processing device that receives the video data from the flying object and the position data from the position data generation system, the method comprising:
The information processing device includes:
a dividing process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data;
Generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data based on the short-time video data and the position data of the flying object corresponding to the shooting time of the short-time video data. generation processing,
A three-dimensional data generation method, comprising: transmitting at least part of the generated three-dimensional point group data to a display device.

1 Unmanned aerial vehicle (mobile object)
20 Information processing device 30 Motion capture device (position data generation system)
100 3D data generation system

Claims (5)

A flying object capable of photographing a target area with a photographing device and generating video data including photographing time information;
a position data generation system capable of detecting the position of the flying object and generating position data including detection time information;
an information processing device that receives the video data from the flying object and the position data from the position data generation system,
The information processing device includes:
a dividing process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data;
Generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data based on the short-time video data and the position data of the flying object corresponding to the shooting time of the short-time video data. generation processing,
A three-dimensional data generation system that executes a process of transmitting at least part of the generated three-dimensional point group data to a display device. The three-dimensional data generation system according to claim 1, further comprising a relay device that receives the video data sent from the flying object and transmits the video data to the server. The three-dimensional data generation system according to claim 1 or 2, wherein the flying object lowers the image quality of the video data before transmitting it to the server device. The three-dimensional data generation system according to claim 1 or 2, wherein the position data generation system is a motion capture device. A flying object capable of photographing a target area with a photographing device and generating video data including photographing time information;
a position data generation system capable of detecting the position of an aircraft and generating position data including detection time information;
A method of generating three-dimensional point group data using an information processing device that receives the video data from the flying object and the position data from the position data generation system, the method comprising:
The information processing device includes:
a dividing process of dividing the received video data into predetermined time lengths to generate a plurality of short-time video data;
Generating three-dimensional point cloud data of at least a portion of the target area corresponding to the short-time video data based on the short-time video data and the position data of the flying object corresponding to the shooting time of the short-time video data. generation processing,
A three-dimensional data generation method, comprising: transmitting at least part of the generated three-dimensional point group data to a display device.

Images