JP7017998B2 - Information processing equipment, flight path generation methods, programs, and recording media - Google Patents

Description

The present disclosure relates to an information processing device for generating a flight path for an air vehicle to fly, a flight path generation method, a program, and a recording medium.

Conventionally, a platform (unmanned aerial vehicle) that performs imaging while passing through a preset fixed path is known (see Patent Document 1). This platform receives imaging instructions from ground bases and captures imaging targets. When an image pickup target is imaged, the platform tilts the image pickup device of the platform according to the positional relationship between the platform and the image pickup target while flying on a fixed path.

Japanese Unexamined Patent Publication No. 2010-61216

In Patent Document 1, the imaging angle for imaging the imaging target from the platform is determined so that the imaging target falls within the imaging range. However, the imaging angle is not determined in order to image the front of the terrain, and there may be places where it is difficult to image the front of the terrain, so that the amount of information at each point in the terrain may decrease. On the other hand, if the terrain is imaged at various imaging angles in order to sufficiently obtain the amount of information of each point in the terrain, the imaging of the image at an unnecessary imaging angle may be included, and the imaging efficiency may decrease. Therefore, it is desirable that the flying object can acquire a large amount of information on the front of the terrain by suppressing the decrease in the imaging efficiency due to the flying object.

In one aspect, the information processing device is an information processing device that generates a flight path for the flying object to fly, and includes a processing unit, and the processing unit acquires topographical information of the flight range in which the flying object flies. , Based on the terrain information of the flight range, generate a flight path including the imaging position in the three-dimensional space for imaging the terrain of the flight range, and image the flight path based on the terrain information of the flight range and the flight path. It is an information processing device that derives the imaging angle for each position.

The processing unit acquires a candidate angle that is a candidate for an imaging angle for imaging the terrain in the flight range, and sets the first imaging cost, which is the imaging cost when imaging at the candidate angle at the imaging position, for each candidate angle. A candidate angle for which the first imaging cost at the imaging position is calculated and equal to or greater than the first threshold value may be determined as the imaging angle at the imaging position.

The processing unit samples the terrain of the flight range, acquires a plurality of sampling positions to be imaged by the flying object, and obtains a second imaging cost, which is an imaging cost when the sampling position is imaged at a candidate angle at the imaging position. , The first imaging cost may be calculated by calculating for each sampling position and summing up the second imaging costs at each sampling position.

The shorter the distance between the imaging position and the sampling position, the higher the second imaging cost may be.

The larger the internal product value of the normal vector with respect to the ground surface at the sampling position and the imaging vector which is a vector along the imaging direction indicated by the candidate angle, the larger the second imaging cost may be.

The processing unit may exclude the second imaging cost whose inner product value is a negative value from the calculation target of the first imaging cost.

The processing unit may acquire an area of interest included in the flight range and include a position to be imaged, and derive an imaging angle for each imaged position of the flight path based on the topographical information of the area of interest and the flight path.

The processing unit acquires a candidate angle that is a candidate for an imaging angle for imaging the terrain of the flight range, and obtains a third imaging cost, which is an imaging cost when the region of interest is imaged at the candidate angle at the imaging position. A candidate angle calculated for each angle and at which the third imaging cost at the imaging position is equal to or greater than the second threshold value may be determined as the imaging angle at the imaging position.

When the third imaging cost when the third imaging cost is captured at the imaging angle at the first imaging position among the plurality of imaging positions is equal to or less than the third threshold value, the processing unit obtains the first imaging position from the plurality of imaging positions. It may be excluded to generate a flight path.

The processing unit may classify a plurality of imaging positions for each imaging target to be imaged from the imaging position to generate a plurality of imaging position groups, and connect the plurality of imaging position groups to generate a flight path.

The information processing device is a terminal provided with a communication unit, and the processing unit may transmit information on an imaging position, a flight path, and an imaging angle to an air vehicle via the communication unit.

The information processing apparatus is a flying object including an image pickup unit, and the processing unit may control flight according to a flight path and capture an image at an imaging position at an imaging position of the flight path via the imaging unit.

In one aspect, the flight path generation method is a flight path generation method in an information processing device that generates a flight path for a flight object to fly, and is a step of acquiring topographical information of a flight range in which the flight object flies. Based on the terrain information of the flight range, the step of generating the flight path including the imaging position in the three-dimensional space for imaging the terrain of the flight range, and the terrain information of the flight range and the flight path of the flight path. This is a flight path generation method including a step of deriving an imaging angle for imaging the terrain of the flight range for each imaging position.

The steps for deriving the imaging angle are a step of acquiring a candidate angle that is a candidate for an imaging angle for imaging the terrain in the flight range, and a first imaging cost that is an imaging cost when imaging at the candidate angle at the imaging position. May include a step of calculating each candidate angle and a step of determining a candidate angle at which the first imaging cost at the imaging position is equal to or greater than the first threshold value as the imaging angle at the imaging position.

The first step of calculating the imaging cost is a step of sampling the terrain of the flight range to acquire a plurality of sampling positions imaged by the flying object, and imaging when the sampling position is imaged at a candidate angle at the imaging position. It may include a step of calculating the second imaging cost, which is a cost, for each sampling position, and a step of summing up the second imaging costs at each sampling position to calculate the first imaging cost.

The shorter the distance between the imaging position and the sampling position, the higher the second imaging cost may be.

The larger the internal product value of the normal vector with respect to the ground surface at the sampling position and the imaging vector which is a vector along the imaging direction indicated by the candidate angle, the larger the second imaging cost may be.

The step of calculating the first imaging cost may include a step of excluding the second imaging cost having a negative inner product value from the calculation target of the first imaging cost.

The step of deriving the imaging angle is included in the flight range, and the imaging angle is taken for each imaging position of the flight path based on the step of acquiring the region of interest including the position of the imaging target and the topographical information and the flight path of the region of interest. May be derived.

The step of deriving the imaging angle is a step of acquiring a candidate angle that is a candidate for an imaging angle for imaging the terrain of the flight range, and a third imaging cost when the region of interest is imaged at the candidate angle at the imaging position. It may include a step of calculating the imaging cost for each candidate angle and a step of determining a candidate angle at which the third imaging cost at the imaging position is equal to or higher than the second threshold value as the imaging angle at the imaging position.

The step of generating the flight path is the first from the plurality of imaging positions when the third imaging cost when the image is captured at the imaging angle at the first imaging position among the plurality of imaging positions is equal to or less than the third threshold value. It may include a step of generating a flight path, excluding the imaging position of.

The step of generating a flight path is a step of classifying a plurality of imaging positions for each imaging target to be imaged from the imaging position to generate a plurality of imaging position groups, and a step of connecting a plurality of imaging position groups to generate a flight path. And may include.

The information processing apparatus is a terminal and may further include a step of transmitting information of an imaging position, a flight path, and an imaging angle to the flying object.

The information processing apparatus is a flying object, and may further include a step of controlling flight according to a flight path and a step of capturing an image at an imaging angle at an imaging position of the flight path.

In one aspect, the program flies based on a step of acquiring topographical information of the flight range in which the flying object flies and the topographical information of the flight range in an information processing device that generates a flight path for the flying object to fly. Based on the step of generating a flight path including the imaged position in the three-dimensional space for imaging the terrain of the range, and the terrain information of the flight range and the flight path, the terrain of the flight range is determined for each imaged position of the flight path. It is a program for executing a step of deriving an imaging angle for imaging.

In one embodiment, the recording medium is based on a step of acquiring topographical information of the flight range in which the flying object flies and the topographical information of the flight range in an information processing device that generates a flight path for the flying object to fly. The terrain of the flight range for each imaged position of the flight path based on the step of generating the flight path including the imaging position in the three-dimensional space for imaging the terrain of the flight range, and the terrain information of the flight range and the flight path. It is a computer-readable recording medium in which a step for deriving an imaging angle for imaging a flight and a program for executing the flight are recorded.

The outline of the above invention does not list all the features of the present disclosure. A subcombination of these feature groups can also be an invention.

Schematic diagram showing a first configuration example of the flying object system in the first embodiment Schematic diagram showing a second configuration example of the flying object system in the first embodiment A diagram showing an example of the concrete appearance of an unmanned aerial vehicle Block diagram showing an example of the hardware configuration of an unmanned aerial vehicle Block diagram showing an example of terminal hardware configuration Figure showing an example of flight range, flight path and imaging position The figure which shows an example of the table which shows the candidate of the imaging angle. A diagram showing an example of sampling positions set on an undulating ground surface A diagram illustrating an example of calculating the imaging cost according to the candidate angle. A sequence diagram showing an example of a flight path generation procedure in the flight object system of the first embodiment. A sequence diagram showing an example of a flight path generation procedure in the flight object system of the second embodiment. Diagram showing an example of specifying the region of interest A diagram illustrating a procedure for regenerating a flight path for imaging a region of interest. A sequence diagram showing an example of a flight path generation procedure in the flight object system of the third embodiment. A sequence diagram showing an example of a flight path generation procedure in the flight object system of the fourth embodiment. A diagram showing the situation where an unmanned aerial vehicle flies along the terrain and images the terrain beneath it. A diagram showing a situation where an unmanned aerial vehicle flies along the terrain and images the terrain at a certain angle. A diagram showing the situation where an unmanned aerial vehicle flies along the terrain and images the terrain from various angles.

Hereinafter, the present disclosure will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Not all combinations of features described in the embodiments are essential to the solution of the invention.

The claims, description, drawings, and abstracts include matters subject to copyright protection. The copyright holder will not object to any reproduction of these documents by any person, as shown in the Patent Office files or records. However, in other cases, all copyrights are reserved.

(Background to Obtaining One Form of the Present Invention)
When imaging the terrain (state on the ground), it is conceivable to image from an unmanned aerial vehicle in a direction directly below or at a certain angle. In this case, since there may be places where it is difficult to image the front of the terrain in the undulating terrain, the amount of information at each point in the terrain may decrease (see FIGS. 16 and 17). FIG. 16 is a diagram showing a situation in which an unmanned aerial vehicle 100R as an air vehicle captures an image of the terrain ms directly below while flying along the terrain ms. When the terrain ms directly below is the mountain slope ms1, it may not be possible to sufficiently capture the information in front of the terrain. FIG. 17 is a diagram showing a situation in which the unmanned aerial vehicle 100R captures the terrain ms at a constant angle while flying along the terrain ms. When the unmanned aerial vehicle 100R images at a constant angle, it may be difficult to image the back surface ms2 of the mountain. That is, it is difficult to obtain information on the front of the terrain.

Further, it is conceivable to image the terrain at a plurality of angles at each imaging position (Waypoint) (see FIG. 18). FIG. 18 is a diagram showing a situation in which the unmanned aerial vehicle 100R captures the terrain ms from various angles while flying along the terrain ms. In this case, for example, an unnecessary image other than the image necessary for the restoration of the terrain shape (three-dimensional restoration) and the generation of the ortho image can be captured. That is, the imaging efficiency when imaging the topographical shape may decrease.

Further, in Patent Document 1, the imaging angle for imaging the imaging target from the platform is determined so that the imaging target falls within the imaging range. That is, the imaging angle is not determined in order to image the front of the terrain, and there may be places where it is difficult to image the front of the terrain, so that the amount of information at each point in the terrain may decrease.

Therefore, it is desirable that the unmanned aerial vehicle can acquire a large amount of information on the front of the terrain by suppressing the deterioration of the imaging efficiency due to the unmanned aerial vehicle.

In the following embodiment, an unmanned aerial vehicle (UAV) is exemplified as an air vehicle. Unmanned aerial vehicles include aircraft that move in the air. In the drawings attached to this specification, the unmanned aerial vehicle is also referred to as "UAV". As the information processing device, for example, a terminal is exemplified, but other devices (for example, a transmitter, a PC (Personal Computer), an unmanned aerial vehicle, and other information processing devices) may be used. The flight path generation method defines the operation of the information processing device. Further, the recording medium is one in which a program (for example, a program for causing an information processing apparatus to execute various processes) is recorded.

(First Embodiment)
FIG. 1 is a diagram showing a first configuration example of the flying object system 10 according to the first embodiment. The aircraft system 10 includes an unmanned aerial vehicle 100 and a terminal 80. The unmanned aerial vehicle 100 and the terminal 80 can communicate with each other by wired communication or wireless communication (for example, wireless LAN (Local Area Network)). FIG. 1 illustrates that the terminal 80 is a mobile terminal (for example, a smartphone or a tablet terminal). The terminal 80 is an example of an information processing device.

The flying object system 10 may be configured to include an unmanned aerial vehicle 100, a transmitter (propo), and a terminal 80. If the transmitter is provided, the user can instruct the control of the flight of the unmanned aerial vehicle by using the left and right control rods arranged in front of the transmitter. Further, in this case, the unmanned aerial vehicle 100, the transmitter, and the terminal 80 can communicate with each other by wire communication or wireless communication.

FIG. 2 is a schematic diagram showing a second configuration example of the flying object system 10 in the first embodiment. FIG. 2 illustrates that the terminal 80 is a PC. In any of FIGS. 1 and 2, the function of the terminal 80 may be the same.

FIG. 3 is a diagram showing an example of a specific appearance of the unmanned aerial vehicle 100. FIG. 3 shows a perspective view of the unmanned aerial vehicle 100 flying in the moving direction STV0. The unmanned aerial vehicle 100 is an example of a moving body.

As shown in FIG. 3, the roll axis (see x-axis) may be set in a direction parallel to the ground and along the moving direction STV0. In this case, the pitch axis (see y-axis) is set in a direction parallel to the ground and perpendicular to the roll axis, and further, a yaw axis (z-axis) is set in a direction perpendicular to the ground and perpendicular to the roll axis and the pitch axis. See) may be set.

The unmanned aerial vehicle 100 includes a UAV main body 102, a gimbal 200, an image pickup unit 220, and a plurality of image pickup units 230.

The UAV main body 102 includes a plurality of rotary wings (propellers). The UAV main body 102 flies the unmanned aerial vehicle 100 by controlling the rotation of a plurality of rotor blades. The UAV body 102 flies the unmanned aerial vehicle 100 using, for example, four rotor blades. The number of rotor blades is not limited to four. Further, the unmanned aerial vehicle 100 may be a fixed-wing aircraft having no rotary wings.

The imaging unit 220 is a camera for imaging that captures a subject included in a desired imaging range (for example, a state of the sky to be imaged, a landscape such as a mountain or a river, a building on the ground).

The plurality of image pickup units 230 are sensing cameras that image the surroundings of the unmanned aerial vehicle 100 in order to control the flight of the unmanned aerial vehicle 100. Two image pickup units 230 may be provided in front of the nose of the unmanned aerial vehicle 100. Further, two other imaging units 230 may be provided on the bottom surface of the unmanned aerial vehicle 100. The two image pickup units 230 on the front side are paired and may function as a so-called stereo camera. The two image pickup units 230 on the bottom surface side may also be paired and function as a stereo camera. Three-dimensional spatial data around the unmanned aerial vehicle 100 may be generated based on the images captured by the plurality of imaging units 230. The number of image pickup units 230 included in the unmanned aerial vehicle 100 is not limited to four. The unmanned aerial vehicle 100 may include at least one image pickup unit 230. The unmanned aerial vehicle 100 may include at least one image pickup unit 230 on each of the nose, aft, side surface, bottom surface, and ceiling surface of the unmanned aerial vehicle 100. The angle of view that can be set by the image pickup unit 230 may be wider than the angle of view that can be set by the image pickup unit 220. The image pickup unit 230 may have a single focus lens or a fisheye lens.

FIG. 4 is a block diagram showing an example of the hardware configuration of the unmanned aerial vehicle 100. The unmanned aerial vehicle 100 includes a UAV control unit 110, a communication interface 150, a memory 160, a storage 170, a gimbal 200, a rotary wing mechanism 210, an image pickup unit 220, an image pickup unit 230, a GPS receiver 240, and the like. The configuration includes an inertial measurement unit (IMU) 250, a magnetic compass 260, a barometric altimeter 270, an ultrasonic sensor 280, and a laser measuring instrument 290.

The UAV control unit 110 is configured by using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). The UAV control unit 110 performs signal processing for controlling the operation of each unit of the unmanned aerial vehicle 100, data input / output processing with and from other units, data calculation processing, and data storage processing.

The UAV control unit 110 controls the flight of the unmanned aerial vehicle 100 according to the program stored in the memory 160. The UAV control unit 110 may control the flight. The UAV control unit 110 may capture an image (for example, aerial photography).

The UAV control unit 110 acquires position information indicating the position of the unmanned aerial vehicle 100. The UAV control unit 110 may acquire position information indicating the latitude, longitude, and altitude in which the unmanned aerial vehicle 100 exists from the GPS receiver 240. The UAV control unit 110 acquires latitude / longitude information indicating the latitude and longitude of the unmanned aerial vehicle 100 from the GPS receiver 240 and altitude information indicating the altitude at which the unmanned aerial vehicle 100 exists from the barometric altitude meter 270 as position information. good. The UAV control unit 110 may acquire the distance between the emission point of the ultrasonic wave and the reflection point of the ultrasonic wave by the ultrasonic sensor 280 as altitude information.

The UAV control unit 110 may acquire orientation information indicating the orientation of the unmanned aerial vehicle 100 from the magnetic compass 260. The orientation information may be shown, for example, in the orientation corresponding to the orientation of the nose of the unmanned aerial vehicle 100.

The UAV control unit 110 may acquire position information indicating a position where the unmanned aerial vehicle 100 should exist when the image pickup unit 220 captures an image pickup range to be imaged. The UAV control unit 110 may acquire position information indicating the position where the unmanned aerial vehicle 100 should exist from the memory 160. The UAV control unit 110 may acquire position information indicating the position where the unmanned aerial vehicle 100 should exist from another device via the communication interface 150. The UAV control unit 110 may refer to the three-dimensional map database to specify a position where the unmanned aerial vehicle 100 can exist, and acquire the position as position information indicating the position where the unmanned aerial vehicle 100 should exist.

The UAV control unit 110 may acquire image pickup range information indicating the respective image pickup ranges of the image pickup unit 220 and the image pickup unit 230. The UAV control unit 110 may acquire the angle of view information indicating the angle of view of the image pickup unit 220 and the image pickup unit 230 from the image pickup unit 220 and the image pickup unit 230 as a parameter for specifying the image pickup range. The UAV control unit 110 may acquire information indicating the imaging direction of the imaging unit 220 and the imaging unit 230 as a parameter for specifying the imaging range. The UAV control unit 110 may acquire posture information indicating the posture state of the imaging unit 220 from the gimbal 200, for example, as information indicating the imaging direction of the imaging unit 220. The posture information of the image pickup unit 220 may indicate the rotation angle of the gimbal 200 from the reference rotation angle of the pitch axis and the yaw axis.

The UAV control unit 110 may acquire position information indicating the position where the unmanned aerial vehicle 100 exists as a parameter for specifying the imaging range. The UAV control unit 110 defines an image pickup range indicating a geographical range imaged by the image pickup unit 220 based on the image angle and image pickup direction of the image pickup unit 220 and the image pickup unit 230, and the position where the unmanned aerial vehicle 100 exists. The imaging range information may be acquired by generating the imaging range information.

The UAV control unit 110 may acquire image pickup range information from the memory 160. The UAV control unit 110 may acquire image pickup range information via the communication interface 150.

The UAV control unit 110 controls the gimbal 200, the rotor blade mechanism 210, the image pickup unit 220, and the image pickup section 230. The UAV control unit 110 may control the image pickup range of the image pickup unit 220 by changing the image pickup direction or the angle of view of the image pickup unit 220. The UAV control unit 110 may control the imaging range of the imaging unit 220 supported by the gimbal 200 by controlling the rotation mechanism of the gimbal 200.

The image pickup range refers to a geographical range imaged by the image pickup unit 220 or the image pickup unit 230. The imaging range is defined by latitude, longitude, and altitude. The imaging range may be a range in 3D spatial data defined by latitude, longitude, and altitude. The imaging range may be a range in two-dimensional spatial data defined by latitude and longitude. The imaging range may be specified based on the angle of view and imaging direction of the imaging unit 220 or the imaging unit 230, and the position where the unmanned aerial vehicle 100 is present. The imaging direction of the image pickup unit 220 and the image pickup unit 230 may be defined from the direction and depression angle at which the front surface of the image pickup unit 220 and the image pickup unit 230 provided with the image pickup lens faces. The imaging direction of the imaging unit 220 may be a direction specified from the orientation of the nose of the unmanned aerial vehicle 100 and the posture of the imaging unit 220 with respect to the gimbal 200. The imaging direction of the imaging unit 230 may be a direction specified from the orientation of the nose of the unmanned aerial vehicle 100 and the position where the imaging unit 230 is provided.

The UAV control unit 110 may identify the environment around the unmanned aerial vehicle 100 by analyzing a plurality of images captured by the plurality of image pickup units 230. The UAV control unit 110 may control the flight, for example, avoiding obstacles, based on the environment around the unmanned aerial vehicle 100.

The UAV control unit 110 may acquire three-dimensional information (three-dimensional information) indicating the three-dimensional shape (three-dimensional shape) of an object existing around the unmanned aerial vehicle 100. The object may be, for example, a part of a landscape such as a building, a road, a car, a tree, or the like. The three-dimensional information is, for example, three-dimensional spatial data. The UAV control unit 110 may acquire the three-dimensional information by generating the three-dimensional information indicating the three-dimensional shape of the object existing around the unmanned aerial vehicle 100 from each image obtained from the plurality of image pickup units 230. The UAV control unit 110 may acquire three-dimensional information indicating the three-dimensional shape of an object existing around the unmanned aerial vehicle 100 by referring to the three-dimensional map database stored in the memory 160 or the storage 170. The UAV control unit 110 may acquire three-dimensional information regarding the three-dimensional shape of an object existing around the unmanned aerial vehicle 100 by referring to a three-dimensional map database managed by a server existing on the network.

The UAV control unit 110 controls the flight of the unmanned aerial vehicle 100 by controlling the rotary wing mechanism 210. That is, the UAV control unit 110 controls the position including the latitude, longitude, and altitude of the unmanned aerial vehicle 100 by controlling the rotary wing mechanism 210. The UAV control unit 110 may control the image pickup range of the image pickup unit 220 by controlling the flight of the unmanned aerial vehicle 100. The UAV control unit 110 may control the angle of view of the image pickup unit 220 by controlling the zoom lens included in the image pickup unit 220. The UAV control unit 110 may control the angle of view of the image pickup unit 220 by digital zoom using the digital zoom function of the image pickup unit 220.

When the image pickup unit 220 is fixed to the unmanned aerial vehicle 100 and the image pickup unit 220 cannot be moved, the UAV control unit 110 moves the unmanned aerial vehicle 100 to a specific position at a specific date and time to obtain a desired image in a desired environment. The range may be imaged by the image pickup unit 220. Alternatively, even if the image pickup unit 220 does not have a zoom function and the angle of view of the image pickup unit 220 cannot be changed, the UAV control unit 110 desired by moving the unmanned aerial vehicle 100 to a specific position at a specified date and time. The image pickup unit 220 may image a desired imaging range under the above environment.

The communication interface 150 communicates with the terminal 80. The communication interface 150 may perform wireless communication by any wireless communication method. The communication interface 150 may perform wired communication by any wired communication method. The communication interface 150 may transmit the captured image and additional information (metadata) related to the captured image to the terminal 80.

The UAV control unit 110 has a gimbal 200, a rotary wing mechanism 210, an image pickup unit 220, an image pickup unit 230, a GPS receiver 240, an inertial measurement unit 250, a magnetic compass 260, a barometric altimeter 270, an ultrasonic sensor 280, and a laser. Stores programs and the like required to control the measuring instrument 290. The memory 160 may be a computer-readable recording medium, and may be SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), and It may include at least one of flash memories such as USB (Universal Serial Bus) memory. The memory 160 may be removable from the unmanned aerial vehicle 100. The memory 160 may operate as a working memory.

The storage 170 may include at least one of an HDD (Hard Disk Drive), an SSD (Solid State Drive), an SD card, a USB memory, and other storage. The storage 170 may hold various information and various data. The storage 170 may be removable from the unmanned aerial vehicle 100. The storage 170 may record a captured image.

The memory 160 or the storage 170 may hold information on the imaging position and imaging path generated by the terminal 80 or the unmanned aerial vehicle 100. Information on the imaging position and the imaging path is set by the UAV control unit 110 as one of the imaging parameters related to the imaging scheduled by the unmanned aerial vehicle 100 or the flight parameters related to the flight scheduled by the unmanned aerial vehicle 100. good. This setting information may be held in the memory 160 or the storage 170. Further, the imaging parameter may include information on the imaging angle by the imaging unit 220.

The gimbal 200 may rotatably support the imaging unit 220 about the yaw axis, the pitch axis, and the roll axis. The gimbal 200 may change the imaging direction of the imaging unit 220 by rotating the imaging unit 220 around at least one of the yaw axis, the pitch axis, and the roll axis.

The rotary blade mechanism 210 includes a plurality of rotary blades and a plurality of drive motors for rotating the plurality of rotary blades. The rotary wing mechanism 210 flies the unmanned aerial vehicle 100 by controlling the rotation by the UAV control unit 110. The number of rotary blades 211 may be, for example, four or any other number. Further, the unmanned aerial vehicle 100 may be a fixed-wing aircraft having no rotary wings.

The image pickup unit 220 captures a subject in a desired imaging range and generates data of the captured image. The image data (for example, an aerial image) obtained by the image pickup of the image pickup unit 220 may be stored in the memory of the image pickup unit 220 or the storage 170.

The image pickup unit 230 captures the surroundings of the unmanned aerial vehicle 100 and generates data of the captured image. The image data of the image pickup unit 230 may be stored in the storage 170.

The GPS receiver 240 receives a plurality of signals indicating the time and the position (coordinates) of each GPS satellite transmitted from the plurality of navigation satellites (that is, GPS satellites). The GPS receiver 240 calculates the position of the GPS receiver 240 (that is, the position of the unmanned aerial vehicle 100) based on the plurality of received signals. The GPS receiver 240 outputs the position information of the unmanned aerial vehicle 100 to the UAV control unit 110. The position information of the GPS receiver 240 may be calculated by the UAV control unit 110 instead of the GPS receiver 240. In this case, information indicating the time included in the plurality of signals received by the GPS receiver 240 and the position of each GPS satellite is input to the UAV control unit 110.

The inertial measurement unit 250 detects the attitude of the unmanned aerial vehicle 100 and outputs the detection result to the UAV control unit 110. The inertial measurement unit 250 detects, as the posture of the unmanned aerial vehicle 100, the acceleration in the three axial directions of the front-back, left-right, and up-down of the unmanned aerial vehicle 100, and the angular velocity in the three-axis directions of the pitch axis, the roll axis, and the yaw axis. It's okay.

The magnetic compass 260 detects the direction of the nose of the unmanned aerial vehicle 100 and outputs the detection result to the UAV control unit 110.

The barometric altimeter 270 detects the altitude at which the unmanned aerial vehicle 100 flies, and outputs the detection result to the UAV control unit 110.

The ultrasonic sensor 280 emits ultrasonic waves, detects ultrasonic waves reflected by the ground or an object, and outputs the detection result to the UAV control unit 110. The detection result may indicate the distance or altitude from the unmanned aerial vehicle 100 to the ground. The detection result may indicate the distance from the unmanned aerial vehicle 100 to the object (subject).

The laser measuring instrument 290 irradiates an object with laser light, receives the reflected light reflected by the object, and measures the distance between the unmanned aircraft 100 and the object (subject) by the reflected light. As an example, the time-of-flight method may be used as the method for measuring the distance by the laser beam.

FIG. 5 is a block diagram showing an example of the hardware configuration of the terminal 80. The terminal 80 includes a terminal control unit 81, an operation unit 83, a communication unit 85, a memory 87, a display unit 88, and a storage 89. The terminal 80 may be possessed by a user who desires flight control of the unmanned aerial vehicle 100.

The terminal control unit 81 is configured by using, for example, a CPU, an MPU, or a DSP. The terminal control unit 81 performs signal processing for controlling the operation of each unit of the terminal 80 in a unified manner, data input / output processing with each other unit, data calculation processing, and data storage processing.

The terminal control unit 81 may acquire data and information (various measurement data, captured images, additional information thereof, etc.) from the unmanned aerial vehicle 100 via the communication unit 85. The terminal control unit 81 may acquire data and information (for example, various parameters) input via the operation unit 83. The terminal control unit 81 may acquire data and information held in the memory 87. The terminal control unit 81 may transmit data and information (for example, image pickup position, image pickup angle, flight path information) to the unmanned aerial vehicle 100 via the communication unit 85. The terminal control unit 81 may send data or information to the display unit 88 and display the display information based on the data or information on the display unit 88.

The terminal control unit 81 may execute an application for performing flight control on the unmanned aerial vehicle 100. The terminal control unit 81 may generate various data used in the application.

The operation unit 83 receives and acquires data and information input by the user of the terminal 80. The operation unit 83 may include an input device such as a button, a key, a touch screen, a microphone, and the like. Here, it is illustrated that the operation unit 83 and the display unit 88 are mainly composed of a touch panel. In this case, the operation unit 83 can receive a touch operation, a tap operation, a drag operation, and the like. The operation unit 83 may receive information on various parameters. The information input by the operation unit 83 may be transmitted to the unmanned aerial vehicle 100. Various parameters may include parameters related to flight control.

The communication unit 85 wirelessly communicates with the unmanned aerial vehicle 100 by various wireless communication methods. The wireless communication method of this wireless communication may include communication via, for example, a wireless LAN, Bluetooth®, or a public wireless line. The communication unit 85 may perform wired communication by any wired communication method.

The memory 87 has, for example, a ROM in which data of a program or set value that defines the operation of the terminal 80 is stored, and a RAM in which various information and data used during processing of the terminal control unit 81 are temporarily stored. It's okay. The memory 87 may include a memory other than the ROM and the RAM. The memory 87 may be provided inside the terminal 80. The memory 87 may be provided so as to be removable from the terminal 80. The program may include an application program.

The display unit 88 is configured by using, for example, an LCD (Liquid Crystal Display), and displays various information and data output from the terminal control unit 81. The display unit 88 may display various data and information related to the execution of the application.

The storage 89 stores and holds various data and information. The storage 89 may be an HDD, SSD, SD card, USB memory, or the like. The storage 89 may be provided inside the terminal 80. The storage 89 may be provided so as to be removable from the terminal 80. The storage 89 may hold the captured image and additional information acquired from the unmanned aerial vehicle 100. The additional information may be held in the memory 87.

When the flying object system 10 includes a transmitter (propo), the transmitter may execute the process executed by the terminal 80. Since the transmitter has the same components as the terminal 80, detailed description thereof will be omitted. The transmitter has a control unit, an operation unit, a communication unit, a display unit, a memory, and the like. If the flying object system 10 has a transmitter, the terminal 80 may not be provided.

FIG. 6 is a diagram showing a flight range AR, a flight path rt, and an imaging position wp. The flight range AR indicates the range in which the unmanned aerial vehicle 100 flies. The flight range AR may coincide with the imaging range captured by the imaging unit 220 of the unmanned aerial vehicle 100. The flight path rt indicates the route when the unmanned aerial vehicle 100 flies. The image pickup position wp is a position where the image pickup unit 220 of the unmanned aerial vehicle 100 captures an image. The flight path rt is generated so as to pass through the imaging position wp. The terminal control unit 81 acquires the flight range AR, generates the flight path rt, and determines the imaging position wp.

The unmanned aerial vehicle 100 performs aerial photography at the imaging position wp while flying along the flight path rt within the flight range AR. In FIG. 6, the flight path rt is a route set to enter from the lower left corner, move in a rectangular wave shape, and exit from the upper right corner. The flight path rt in this case is a flight path according to a scanning method for uniformly imaging the flight range AR. Further, the flight path rt may be a route set in a zigzag shape or a spiral shape in addition to a square wavy route, or may be a flight path having another shape.

FIG. 7 is a diagram showing a table showing candidates for imaging angles (also referred to as candidate angles). The image pickup angle is an image pickup angle for the image pickup unit 220 of the unmanned aerial vehicle 100 to image the terrain of the flight range AR. The candidate for the imaging angle is a candidate for the imaging angle that is actually adopted at the time of imaging among various imaging angles. The table showing the candidate angles may be registered in the memory 87 or the storage 89 of the terminal 80. The table showing the candidate angles may be held in the external server.

The imaging angle may be defined by the pitch angle and yaw angle of the gimbal 200 that supports the imaging unit 220. Therefore, the candidate angle may also be defined by the pitch angle and yaw angle of the gimbal 200 that supports the imaging unit 220. In the table shown in FIG. 7, combinations (points) of nine pitch angles and yaw angles are shown as candidate angles. For example, the nine points include pitch angle: 0 °, yaw angle: 0 °, pitch angle: 0 °, yaw angle: 270 °, and pitch angle: -45 °, yaw angle: 270 °. Points are included.

Note that FIG. 7 is an example, and the pitch angle and the yaw angle may be defined in more detail. Further, in FIG. 7, the pitch angle and the yaw angle of the candidate angles are defined at uniform intervals, but may be defined at non-uniform intervals. For example, many candidate angles are defined in the range of angles that are easily selected (easily assumed) as the imaging angle by the imaging unit 220, and candidates are determined in the range of angles that are difficult to be selected (difficult to assume) as the imaging angle by the imaging unit 220. The angle may be small.

In FIG. 7, assuming a case where the unmanned aerial vehicle 100 takes an aerial photograph from the sky, the pitch angle is set to be a minus angle, that is, the imaging direction is set to be downward from the horizontal plane with respect to the horizontal direction. The direction along the horizontal direction is the pitch angle: 0 °, and the direction directly below is the pitch angle: 90 °. When the altitude of the unmanned aerial vehicle 100 is low and aerial photography is performed from below so that the image pickup unit 220 looks up, the pitch angle may be set to a plus angle. This makes it possible to take an image suitable for the situation of the subject.

The terminal control unit 81 may acquire a candidate angle for capturing the terrain of the flight range AR. The terminal control unit 81 may acquire the candidate angle from the memory 87 or the storage 89. The terminal control unit 81 may acquire the candidate angle from the external server via the communication unit 85.

FIG. 8 is a diagram showing an example of the sampling position k set on the undulating ground surface hm. The sampling position k may be a sampling position obtained by extracting the terrain of the flight range AR, which is imaged by the unmanned aerial vehicle 100. The sampling position may be defined by a three-dimensional position (latitude, longitude, altitude). Topographical information may be determined based on multiple sampling positions.

In FIG. 8, the sampling position k is set in one direction within the flight range AR, but the sampling position k may be set in the two-dimensional direction within the flight range AR. That is, the sampling position k may be set in a grid pattern (grid shape) within the flight range AR, that is, at predetermined intervals. Further, the sampling positions k may be arranged at irregular intervals instead of equal intervals.

Further, the arrow from the sampling position k on the ground surface hm toward the image pickup unit 220 of the unmanned aerial vehicle 100 indicates a normal vector (normal direction) with respect to the ground surface hm. When the imaging unit 220 is present on the normal vector at the sampling position k, when the sampling position is imaged from the imaging unit 220, a lot of information around the sampling position k can be obtained from the front of the sampling position k. On the other hand, the imaging range by the imaging unit 220 may include not only the sampling position k existing in the front surface but also other sampling positions k not in the front surface. Therefore, it is desirable that each sampling position k is imaged from the direction as close to the front as possible to the entire sampling position k included in the imaging range. The degree to which it is preferable to image each sampling position k by the image pickup unit 220 may be quantified as an image pickup cost.

In FIG. 8, the vertical axis of the graph represents the altitude in the three-dimensional space (for example, the image pickup unit 220 of the unmanned aerial vehicle 100, the altitude of the ground surface hm), and the horizontal axis represents the position (latitude, mild) in the three-dimensional space. For example, the sampling position k, the position of the unmanned aerial vehicle 100 provided with the image pickup unit 220) is represented.

The sampling position information may be held in the memory 87, the storage 89, or an external server. The terminal control unit 81 may acquire information on the sampling position k from the memory 87 or the storage 89. The terminal control unit 81 may acquire information on the sampling position k via the communication unit 85. The terminal control unit 81 itself may sample the topographical information and determine the sampling position k.

FIG. 9 is a diagram illustrating an example of calculating an imaging cost according to a candidate angle. The imaging cost is a numerical value of whether or not the imaging unit 220 of the unmanned aerial vehicle 100 is suitable for imaging. The imaging cost is calculated for each candidate angle, for example.

The terminal control unit 81 performs the following specific example processing (for example, calculation of a mathematical formula) to calculate the imaging cost according to the candidate angle. The terminal control unit 81 determines the imaging angle θ based on the imaging cost. Since the imaging angle θ is calculated for each imaging position i, it is also referred to as an imaging angle θi.

For example, the position of the ground surface hm to be imaged is represented by the sampling position k (k = 1, 2, ..., K). The image pickup position wp imaged by the image pickup unit 220 of the unmanned aerial vehicle 100 is represented by the image pickup position i (i = 1, 2, ..., I). The candidate angle of the image pickup unit 220 is represented by the candidate angle j (j = 1, 2, ..., J).

In this case, the terminal control unit 81 may calculate the imaging cost Cij of the candidate angle j at the imaging position i according to the equation (1).

Further, for example, the distance from the image pickup unit 220 to the ground surface hm, that is, the distance between the image pickup position i and the sampling position k is represented by the distance d. The imaging direction corresponding to the candidate angle j of the imaging unit 220 is represented by the imaging vector n. The normal direction of the ground surface hm at the sampling position k is represented by the normal vector l. The terminal control unit 81 may calculate the imaging cost Cijk for the sampling position k of the imaging target according to the equation (2).

From the equation (2), the shorter the distance d, the larger the imaging cost Cijk of the sampling position. Further, n * (−l) represents the inner product (inner product value) of the image pickup vector n and the normal vector l. From the equation (2), the larger the inner product n * (−l), the larger the imaging cost Cijk of the sampling position.

Further, max (n * (−l), 0) represents the larger value of the inner product n * (−l) and the value 0. This means that the terminal control unit 81 excludes the imaging cost Cijk at the sampling position k such that the inner product n * (−l) is negative from the calculation target of the imaging cost Cij at the imaging position i.

なお、ａｒｇｍａｘ（Ｃｉｊ）は、撮像コストＣｉｊが最大（ｍａｘ）となる場合の候補角度ｊであり、この角度が最適撮像角度θｉｍとなる。 As shown in the equations (1) and (2), the terminal control unit 81 obtains the imaging cost Cij by summing the imaging cost Cijk at each sampling position k. The candidate angle j at which the imaging cost Cij at the imaging position i is maximized is the optimum imaging angle θim. The terminal control unit 81 may calculate the optimum imaging angle θim according to the equation (3).

The argmax (Cij) is a candidate angle j when the imaging cost Cij is the maximum (max), and this angle is the optimum imaging angle θim.

The optimum imaging angle θim is an example of the imaging angle θi. That is, the imaging angle θi is not limited to the angle at which the imaging cost Cij is maximized, and may be an angle that satisfies a predetermined reference. For example, the imaging angle θi determined (selected) from the candidate angles may be an angle having the second or third largest imaging cost Cij among the angles having the threshold value th1 or more. Further, the imaging angle θi may be an imaging angle corresponding to the imaging cost Cij, which is equal to or higher than the average value obtained by averaging each imaging cost Cij.

Further, when the imaging cost Cij at the imaging position i is less than the threshold value th1, the imaging angle θi is not determined from the candidate angle j, and imaging at this imaging position may be omitted. In this case, for example, by not setting the imaging position i at which the image quality of the captured image is equal to or less than a predetermined reference regardless of the imaging angle θi, the terminal 80 can omit unnecessary imaging and can improve the imaging efficiency. Further, the terminal 80 can also fly the unmanned aerial vehicle 100 according to the flight path rt that does not pass through the imaging position i.

In this way, the terminal control unit 81 may sample the terrain of the flight range AR and acquire a plurality of sampling positions k imaged by the unmanned aerial vehicle 100. The terminal control unit 81 may calculate the imaging cost Cijk (an example of the second imaging cost) when the sampling position k is imaged at the candidate angle j at the imaging position i for each sampling position k. The terminal control unit 81 may total the imaging cost Cijk of each sampling position k and calculate the imaging cost Cij (an example of the first imaging cost) of the imaging position i.

As a result, the terminal 80 can determine the imaging angle θi at each imaging position i in consideration of the imaging cost Cijk of each sampling position k. For example, in the terminal 80, even if the imaging cost Cijk at one sampling position k is small, if the imaging cost Cijk at another sampling position k is large, the imaging cost Cij of the entire plurality of sampling positions k becomes large. The candidate angle j in the case can be adopted as the imaging angle θi. Therefore, the terminal 80 can determine the imaging angle θi by comprehensively considering the goodness of imaging at a plurality of sampling positions k.

Further, as in the equation (2), the shorter the distance d, the larger the imaging cost Cijk of the sampling position may be.

As a result, the shorter the distance d between the imaging position i and the sampling position k, the larger the imaging cost Cijk at the sampling position k. Therefore, the closer the sampling position k is to the unmanned aerial vehicle 100, the higher the imaging cost at the sampling position k. The Cijk becomes large, and the imaging cost Cij at the imaging position i tends to increase. Therefore, the terminal 80 can increase the influence of the imaging cost Cijk at the sampling position k close to the unmanned aerial vehicle 100 when determining the imaging angle θi. Further, when the sampling position k is close to the unmanned aerial vehicle 100, the imaging range (range included in the captured image) imaged by the unmanned aerial vehicle 100 is narrowed, and the image information of the sampling position k in the imaging range is relatively increased. Therefore, since the information on the ground surface for the ortho image and the three-dimensional restoration is increased, the terminal 80 can improve the generation accuracy and the three-dimensional restoration accuracy of the ortho image.

Further, as in the equation (2), the larger the inner product n * (−l), the larger the imaging cost Cijk at the sampling position k may be.

As a result, the larger the internal product value of the normal vector l with respect to the ground surface hm at the sampling position k and the imaging vector n which is a vector along the imaging direction indicated by the candidate angle, the larger the imaging cost Cijk at the sampling position. Therefore, the smaller the angle formed by the normal vector l and the image pickup vector n, the larger the image pickup cost Cijk at the sampling position k, and the larger the image pickup cost Cij at the image pickup position i. Therefore, when determining the image pickup angle θi, the terminal 80 can increase the degree of influence of the image pickup vector n, which makes the angle formed with the normal vector l smaller. Further, when the angle formed by the normal vector l and the image pickup vector n is small, the sampling position k can be imaged from a position close to the front, and the image information of the sampling position k increases. Therefore, since the information on the ground surface for the ortho image and the three-dimensional restoration is increased, the terminal 80 can improve the generation accuracy and the three-dimensional restoration accuracy of the ortho image.

Further, the terminal control unit 81 excludes the imaging cost Cijk at the sampling position k where the value (inner product value) of the inner product n * (−l) is a negative value from the calculation target of the imaging cost Cij at the imaging position i. You can do it.

As a result, the terminal 80 sets the imaging cost Cijk of the sampling position such that the value of the inner product n * (−l) becomes a negative value to 0, so that the terminal 80 has one extreme value in which the inner product value becomes negative. Therefore, it is possible to prevent the imaging cost Cij at the imaging position i from being significantly affected.

Next, an operation example of the flying object system 10 will be described.

FIG. 10 is a sequence diagram showing an example of a flight path generation procedure in the flight body system 10. In FIG. 10, it is illustrated that the flight path generation process is mainly performed by the terminal 80.

The terminal control unit 81 acquires the information of the flight range AR (T1). The terminal control unit 81 may receive user input via the operation unit 83 and acquire the flight range AR. In this case, the terminal control unit 81 may acquire map information from an external server via the communication unit 85. The flight range AR information may be obtained, for example, when the flight range AR is set to a rectangular range, by the user inputting the positions (latitude, longitude) of the four corners of the rectangle in the map information. Further, the information of the flight range AR may be obtained by the user inputting the radius of the circle centered on the flight position when the flight range AR is set to the circular range. Further, the flight range AR information may be obtained based on the map information by the user inputting information such as a region or a specific place name (for example, Tokyo). Further, the terminal control unit 81 may acquire the flight range AR held in the memory 87 or the storage 89 from the memory 87 or the storage 89. The terminal control unit 81 may acquire the flight range AR from an external server via the communication unit 85.

The terminal control unit 81 acquires various parameters (T2). The parameters may be parameters related to imaging by the imaging unit 220 and flight of the unmanned aerial vehicle 100. This parameter may include, for example, an imaging position, an imaging date and time, a distance to a subject, an imaging angle of view, an imaging condition, and camera parameters (shutter speed, exposure value, imaging mode, etc.). The terminal control unit 81 may acquire parameters input by the user via the operation unit 83. The terminal control unit 81
Various parameters held in the memory 87 and the storage 89 may be acquired from the memory 87 and the storage 89. The terminal control unit 81 may acquire various parameters from the unmanned aerial vehicle 100 or an external server via the communication unit 85.

The terminal control unit 81 acquires terrain information based on the information of the flight range AR (T3). For example, the terminal control unit 81 may interlock with a map server on the network connected via the communication unit 85 to acquire topographical information of the flight range AR. The terrain information may include position information (latitude, longitude, altitude) of each position in the flight range AR. By aggregating the position information of each position, the three-dimensional shape of the flight range AR may be shown. In addition, the topographical information may include information on the shape of the ground surface such as buildings, mountains, forests, and steel towers, and information on objects.

The terminal control unit 81 calculates the flight altitude based on the topographical information of the flight range AR and the information such as the distance to the subject included in the acquired parameters (T4). For example, the terminal control unit 81 may calculate the flight altitude of the unmanned aerial vehicle 100 so that the distance to the subject is secured according to the undulations of the ground surface hm indicated by the topographical information.

The terminal control unit 81 generates a flight path rt (T5). In this case, the terminal control unit 81 may generate a flight path rt based on the flight range AR, topographical information, and flight altitude. The generated flight path rt passes through the imaging position wp in three-dimensional space for maintaining the derived flight altitude and imaging the terrain within the flight range AR at each position in the flight range AR. The terminal control unit 81 is known about which position in the two-dimensional plane (latitude, longitude) in the flight range AR the flight path passes through, and which sampling position k (imaging position wp) the flight path passes through. It may be decided according to the method of.

The terminal control unit 81 derives (for example, calculates) an imaging angle θi for each imaging position i along the flight path rt based on the topographical information and the flight altitude (T6). In deriving the imaging angle θi, the terminal control unit 81 calculates the imaging cost Cij at the imaging position i for each candidate angle j. The terminal control unit 81 determines a candidate angle (for example, the optimum imaging angle θim) at which the imaging cost Cij of the imaging position i becomes the threshold value th1 or more (for example, the maximum) as the imaging angle θi at the imaging position i. In the derivation of the imaging angle θi, the optimum imaging angle θim may be calculated based on the topographical information and the flight path rt information.

The terminal control unit 81 transmits the notification parameters including the image pickup position wp, the flight path rt, and the image pickup angle θi to the unmanned aerial vehicle 100 via the communication unit 85 (T7). The notification parameter may include an imaging parameter relating to the camera (imaging unit 220) at the time of imaging and a flight parameter relating to flight at the time of imaging.

In the unmanned aerial vehicle 100, the UAV control unit 110 receives the notification parameter from the terminal 80 via the communication interface 150 (T8). The UAV control unit 110 sets each parameter used by the unmanned aerial vehicle 100 by holding the received notification parameter in the memory 160 (T9). The UAV control unit 110 drives the image pickup unit 220 while flying along the flight path rt based on the set parameters, and performs aerial photography at the image pickup angle θi (T10).

In this way, the terminal 80 may generate a flight path rt for the unmanned aerial vehicle 100 to fly. The terminal control unit 81 may acquire topographical information of the flight range AR in which the unmanned aerial vehicle 100 flies. The terminal control unit 81 may generate a flight path rt including an imaging position wp in a three-dimensional space for imaging the terrain of the flight range AR based on the terrain information of the flight range AR. The terminal control unit 81 may derive (for example, calculate) an imaging angle θi for each imaging position of the flight path rt based on the topographical information of the flight range AR and the flight path rt.

As a result, the terminal 80 determines the imaging angle θi in consideration of the undulations of the terrain, so that it is possible to reduce the portion of the ground surface hm that is difficult to image due to the undulations of the terrain. Therefore, the unmanned aerial vehicle 100 can take an image from the front side as much as possible when taking an image of each point on the ground surface hm for each image pickup position i. Therefore, the terminal 80 can improve the generation accuracy of the ortho image and the three-dimensional restoration accuracy (accuracy of three-dimensional shape estimation) by using the captured image captured at the determined imaging angle θi. Further, the terminal 80 can eliminate the need to capture an image from various angles at each imaging position i in order to improve the ortho image generation accuracy and the three-dimensional restoration accuracy, and can improve the imaging efficiency by the unmanned aerial vehicle 100. .. Therefore, the terminal 80 can suppress the deterioration of the imaging efficiency by the unmanned aerial vehicle 100 and allow the unmanned aerial vehicle 100 to acquire as much information as possible at each point of the undulating terrain. The flight path generation process mainly executed by the terminal 80 may be performed during the flight of the unmanned aerial vehicle 100 or before the start of the flight.

Further, the terminal control unit 81 may acquire a candidate angle j which is a candidate for an imaging angle θi for imaging the terrain of the flight range AR. The terminal control unit 81 may calculate the imaging cost Cij at the imaging position i (an example of the first imaging cost which is the imaging cost when imaging at the candidate angle j at the imaging position) for each candidate angle j. The terminal control unit 81 may determine the imaging angle θi at the imaging position based on the imaging cost Cij at the imaging position i. In this case, the terminal control unit 81 may determine the candidate angle j at which the imaging cost Cij at the imaging position i is the threshold value th1 or more as the imaging angle θi at the imaging position.

Thereby, the terminal 80 can quantify whether or not it is suitable for imaging as an imaging cost, and can easily determine how suitable the imaging at the imaging position i and the candidate angle j is. Here, among the imaging cost Cij of the imaging position i having the threshold value th1 or more, the candidate angle j of the nth largest (for example, the maximum) imaging cost Cij may be determined as the imaging angle θi.

Further, the terminal control unit 81 may transmit information on the image pickup position i, the flight path rt, and the image pickup angle θi to the unmanned aerial vehicle 100 via the communication unit 85.

As a result, many processes for generating the flight path rt can be performed on the terminal 80 side, and the terminal 80 suppresses a decrease in imaging efficiency due to the unmanned aerial vehicle 100 while reducing the processing load of the unmanned aerial vehicle 100. Then, the unmanned aerial vehicle 100 can acquire a lot of information on the front of the terrain.

(Second embodiment)
In the second embodiment, it is illustrated that the processing of flight path generation is mainly performed by the unmanned aerial vehicle 100. The flying object system of the second embodiment has substantially the same configuration as that of the first embodiment. For the same components as those in the first embodiment, the same reference numerals are used to omit or simplify the description.

FIG. 11 is a sequence diagram showing a flight path generation procedure in the flight body system 10 of the second embodiment. In FIG. 11, it is illustrated that this flight path generation process is mainly performed by the unmanned aerial vehicle 100.

The processing of the procedures T21 to T23 is the same as the procedures T1 to T3 of the first embodiment. The terminal control unit 81 transmits the flight range AR, parameters, and terrain information acquired in the procedures T21 to T23 to the unmanned aerial vehicle 100 via the communication unit 85 (T24).

In the unmanned aerial vehicle 100, the UAV control unit 110 receives flight range AR, parameters and terrain information via the communication interface 150 (T25). The UAV control unit 110 stores the received flight range AR, parameters, and terrain information in the memory 160.

The UAV control unit 110 calculates the flight altitude (T26). The method of calculating the flight altitude may be the same as that of T4. The UAV control unit 110 generates a flight path rt (T27). The method of generating the flight path rt may be the same as that of T5. The UAV control unit 110 derives an imaging angle θi for each imaging position along the flight path rt (T28). The method for deriving the imaging angle θi may be the same as that for T6. The UAV control unit 110 holds and sets the parameters including the image pickup position, the flight path rt, and the image pickup angle θi in the memory 160 (T29). The UAV control unit 110 drives the image pickup unit 220 at the image pickup position and performs aerial photography at the image pickup angle θi while flying along the flight path rt based on the set parameters (T30).

The processing of T23 may be performed by the unmanned aerial vehicle 100. In this case, the terminal control unit 81 may transmit the flight range AR and parameters acquired in the procedures T21 and T22 to the unmanned aerial vehicle 100 via the communication unit 85. The UAV control unit 110 may receive the flight range AR and parameters, and calculate the terrain information and the flight altitude.

In this way, the unmanned aerial vehicle 100 may generate a flight path rt for the unmanned aerial vehicle 100 to fly. The UAV control unit 110 may acquire topographical information of the flight range AR in which the unmanned aerial vehicle 100 flies. The UAV control unit 110 may generate a flight path rt including an imaging position wp in a three-dimensional space for imaging the terrain of the flight range AR based on the terrain information of the flight range AR. The UAV control unit 110 may derive (for example, calculate) an imaging angle θi for each imaging position of the flight path rt based on the topographical information of the flight range AR and the flight path rt.

As a result, the unmanned aerial vehicle 100 determines the imaging angle θ in consideration of the undulations of the terrain, so that it is possible to reduce the portion of the ground surface hm that is difficult to image due to the undulations of the terrain. Therefore, the unmanned aerial vehicle 100 can take an image from the front side as much as possible when taking an image of each point of the ground surface hm at each image pickup position. Therefore, the unmanned aerial vehicle 100 can improve the generation accuracy of the ortho image and the three-dimensional restoration accuracy (accuracy of three-dimensional shape estimation) by using the captured image captured at the determined imaging angle θi. Further, the unmanned aerial vehicle 100 can improve the imaging efficiency by eliminating the need to capture images from various angles at each imaging position in order to improve the ortho image generation accuracy and the three-dimensional restoration accuracy. Therefore, the unmanned aerial vehicle 100 can suppress a decrease in imaging efficiency and acquire as much information as possible at each point of the undulating terrain. The flight path generation process mainly executed by the unmanned aerial vehicle 100 may be performed during the flight of the unmanned aerial vehicle 100 or before the start of the flight.

Further, the UAV control unit 110 controls the flight according to the flight path rt, and aerial photographs (an example of image capture) of the terrain surface at the image pickup position i of the flight path rt at the image pickup angle θi via the image pickup unit 220. good. As a result, many processes for generating the flight path rt can be performed on the unmanned aerial vehicle 100 side, and the unmanned aerial vehicle 100 reduces the processing load of the terminal 80 while reducing the imaging efficiency by the image pickup unit 220. You can suppress it and get a lot of information on the front of the terrain. Further, the unmanned aerial vehicle 100 can intensively perform from the generation of the flight path rt capable of acquiring a large amount of information on the front surface of the terrain by suppressing the decrease in the imaging efficiency by the imaging unit 220 to the imaging along the generated flight path rt. ..

(Third embodiment)
In the first and second embodiments, the case where the entire flight range AR is aerial photographed is shown. The third embodiment shows a case where the area of interest RI is mainly aerial photographed in the flight range AR. The region of interest RI may be an region including an region of interest to the user or a position where an object of interest to the user exists. The region of interest RI may be set by inputting and operating the region or target of interest by the user, for example, via the operation unit 83.

The flying object system 10 of the third embodiment has substantially the same configuration as that of the first and second embodiments. By using the same reference numerals for the same components as those of the first and second embodiments, the description thereof will be omitted or simplified.

FIG. 12 is a diagram showing an example of designation of the region of interest RI. Here, the case where the region of interest RI is an region including a building is shown.

The terminal control unit 81 displays, for example, the acquired flight range AR on the display unit 88. The display unit 88 and the operation unit 83 may be composed of, for example, a touch panel. When the user touch-operates the buildings 501 and 502 in the flight range AR displayed on the display unit 88 with a finger, the terminal control unit 81 receives the positions of the buildings 501 and 502 via the operation unit 83. The terminal control unit 81 sets the area including the positions of the two buildings 501 and 502 as the area of interest RI.

Further, the terminal control unit 81 sets a plurality of initial imaging points gp in the region of interest RI. The initial image pickup point gp is an image pickup position wp set as an initial setting. The terminal control unit 81 may acquire the information of the initial imaging point gp from, for example, the memory 87 or the storage 89, may be acquired by user operation via the operation unit 83, or may be acquired by user operation via the communication unit 85. It may be obtained from an external server. In FIG. 12, the initial imaging points gp are arranged in a grid pattern on a two-dimensional plane. Further, the adjacent initial imaging points gp are arranged at equal intervals. The initial imaging points gp may be arranged in a shape other than a grid pattern, or adjacent initial imaging points gp may not be arranged at equal intervals.

FIG. 13 is a diagram illustrating a procedure for regenerating the flight path rt for the unmanned aerial vehicle 100 to image the region of interest RI. In this case, as in the first embodiment, the terminal control unit 81 derives the imaging cost Cij at the candidate angles j for the plurality of initial imaging points gp. Of the plurality of initial imaging points gp, the terminal control unit 81 deletes the imaging point gpl having the imaging cost Cij having the imaging cost Cij at the candidate angle j of the threshold value th3 or less and having a low imaging cost Cij from the imaging position wp. That is, the terminal control unit 81 does not capture an image at the imaging position (imaging point) where the imaging cost Cij is equal to or less than the threshold value th3. Then, the terminal control unit 81 leaves the imaging point gp having a high imaging cost Cij whose imaging cost Cij at the candidate angle j exceeds the threshold value th3 among the plurality of initial imaging points gp as the imaging position wp.

The terminal control unit 81 may perform clustering (classification) for each region of interest RI (here, for each building of interest) for a plurality of imaging points gp having a high imaging cost Cij at the imaging angle θi. For clustering, well-known methods such as K-means (k-means method) and DBSCAN (Density-based spatial clustering of applications with noise) may be used.

In FIG. 13, the terminal control unit 81 calculates the imaging position groups sg1 and sg2 including the four imaging point gps whose imaging cost Cij is the threshold value th3 or more by clustering. The terminal control unit 81 connects each image pickup point gp included in the image pickup position groups sg1 and sg2 as the image pickup position wp, and regenerates the flight path rtn.

In this way, the terminal control unit 81 classifies a plurality of imaging points gp (imaging positions) having a high imaging cost Cij at the imaging angle θi into each of the region of interest RI to be imaged from the imaging point gp, and a plurality of imaging positions. Groups sg1 and sg2 may be generated. The terminal control unit 81 may connect (connect) a plurality of image pickup position groups sg1 and sg2 to regenerate the flight path rtn (an example of flight path generation).

As a result, the unmanned aerial vehicle 100 can collectively image each of a plurality of imaging points gp (imaging position) included in the imaging position groups sg1 and sg2. Therefore, since the flight distance of the unmanned aerial vehicle 100 for imaging each region of interest RI is shortened, the unmanned aerial vehicle 100 can improve the imaging efficiency.

Next, an operation example of the flying object system 10 will be described.

FIG. 14 is a sequence diagram showing an example of a flight path generation procedure in the flight body system 10 of the third embodiment. FIG. 14 illustrates that this flight path generation process is mainly performed by the terminal 80.

Procedures T41 to T43 are the same as procedures T1 to T3 in the first embodiment.

The terminal control unit 81 acquires the information of the region of interest RI. The terminal control unit 81 may receive user input via the operation unit 83 and acquire the region of interest RI (T44). For example, the user may directly specify the area of interest RI by inputting a place name via the operation unit 83, or may specify the area of interest RI by surrounding a part of the area in the map information. In this case, the terminal control unit 81 may acquire the map information via the communication unit 85. Further, the terminal control unit 81 may receive user input via the operation unit 83 and acquire information on the type of image pickup target. The terminal control unit 81 may detect the region of interest RI (for example, the region including a building) corresponding to the type of the imaging target included in the flight range AR by using the image processing technique based on the type of the imaging target. ..

Procedures T45 and T46 are the same as procedures T4 and T5 in the first embodiment. The terminal control unit 81 derives an imaging angle θi for imaging the region of interest RI at each initial imaging point gp (imaging position) along the flight path rt based on the topographical information of the region of interest RI and the flight altitude ( For example, calculate) (T47).

The terminal control unit 81 may calculate the imaging cost Cij at the candidate angle j according to the equations (4), (5), and (6) in deriving the imaging angle θi for imaging the region of interest RI. The imaging cost Cij at the candidate angle j is calculated for each candidate angle j.

Here, (Pk In ROI) in the equation (5) means that the sampling position k for which the imaging cost Cijk is calculated is limited to the position included in the region of interest RI. That is, the terminal control unit 81 may calculate the imaging cost Cijk when imaging the sampling position k in the region of interest RI at the candidate angle j from the initial imaging point gp (imaging position) according to the equation (5). In other respects, equations (4), (5), and (6) may be the same as equations (1), (2), and (3).

The terminal control unit 81 calculates the imaging angle θi for imaging the region of interest based on, for example, the imaging cost Cij calculated according to the equations (4), (5), and (6). The imaging angle θi may be an imaging angle θim for imaging the region of interest.

When the imaging cost Cij at the imaging angle θi at the plurality of initial imaging points gp is equal to or less than the threshold value th3, the terminal control unit 81 deletes the corresponding imaging point gpl (low imaging cost Cij) (T48). On the other hand, when the imaging cost Cij at the imaging angle θi at the plurality of initial imaging points gp exceeds the threshold value th3, the terminal control unit 81 does not delete the corresponding imaging point gp (high imaging cost Cij), and the imaging position wp. Maintain as.

The terminal control unit 81 performs clustering (grouping of imaging points) for a plurality of imaging points gp having a high imaging cost Cij, and calculates imaging position groups sg1 and sg2 (T49). The terminal control unit 81 flies so as to connect each imaging point gp included in the imaging position groups sg1 and sg2 as an imaging position, that is, excluding the imaging position wp in which the imaging cost Cij of the imaging angle is the threshold value th3 or less. Regenerate the path rtn (T50).

It should be noted that the generation of the imaging position group is not essential, and it is not essential to pass through all the imaging positions wp included in the imaging position group sg2 after passing through all the imaging positions wp included in the imaging position group sg1. For example, an imaging position for imaging different regions of interest RI so as to fly in the order of an imaging position included in the imaging position group sg1, an imaging position included in the imaging position group sg2, and an imaging position included in the imaging position group sg1. Each flight path rtn may be regenerated in random order of wp.

The terminal control unit 81 unmanned the notification parameters including the image pickup position wp (corresponding to a plurality of image pickup points gp having a high image pickup cost Cij), the regenerated flight path rtn, and the image pickup angle θi via the communication unit 85. It is transmitted to the aircraft 100 (T51).

In the unmanned aerial vehicle 100, the UAV control unit 110 receives the notification parameter via the communication interface 50 (T52). The UAV control unit 110 sets each parameter used by the unmanned aerial vehicle 100 by holding the received notification parameter in the memory 160 (T53). The UAV control unit 110 drives the image pickup unit 220 while flying along the flight path rtn based on the set parameters, and performs aerial photography of the region of interest RI at the image pickup angle θi (T54).

In this way, the terminal control unit 81 may acquire the region of interest RI including the positions of, for example, two buildings 501 and 502 included in the flight range AR. The terminal control unit 81 may derive an imaging angle θi for each imaging position based on the topographical information of the region of interest RI and the flight path rt.

As a result, when the unmanned aerial vehicle 100 aerial photographs of the region of interest RI, the terminal 80 can suppress a decrease in imaging efficiency and acquire as much information as possible at each point of the undulating terrain. Further, the terminal 80 can derive an imaging angle θi that can improve the generation accuracy and the three-dimensional restoration accuracy of the ortho image of the region of interest RI.

Further, the terminal control unit 81 may acquire a candidate angle j which is a candidate for an imaging angle θi for imaging the terrain of the flight range AR. The terminal control unit 81 may calculate the imaging cost Cij (an example of the third imaging cost) when the region of interest RI is imaged at the candidate angle j at the imaging position i for each candidate angle j. The terminal control unit 81 may determine the imaging angle θi at the imaging position i based on the imaging cost Cij at the imaging position i. In this case, the terminal control unit 81 may determine the candidate angle j at which the imaging cost Cj at the imaging position i is equal to or higher than the threshold value th2 (an example of the second threshold value) as the imaging angle θi at the imaging position i.

As a result, the terminal 80 can quantify whether or not it is suitable for imaging the region of interest RI as an imaging cost, and can easily determine to what extent it is suitable for imaging. Here, the candidate angle of the nth largest (for example, the maximum) imaging cost Cij among the imaging cost Cij of the imaging position i having the threshold value th2 or more may be determined as the optimum imaging angle θim.

Further, the terminal control unit 81 sets an imaging point gpl (an example of the first imaging position) in which the imaging cost Cij is equal to or less than the threshold th3 when the imaging is performed at the imaging position θi among the plurality of imaging positions i. , The flight path rtn may be regenerated by excluding it from a plurality of imaging positions wp. That is, the terminal control unit 81 may regenerate the flight path rtn by removing the imaging point gpl from the generated flight path rtn.

As a result, since the imaging position having little influence on the imaging cost Cij at the imaging angle θi is not included in the generation (regeneration) of the flight path rtn, the terminal 80 can generate the ortho image of the region of interest RI and three-dimensionally. It is possible to suppress a decrease in restoration accuracy and increase the imaging efficiency when imaging the region of interest RI by the unmanned aerial vehicle 100.

(Fourth Embodiment)
In the fourth embodiment, it is illustrated that the process of generating the flight path in consideration of the region of interest is mainly performed by the unmanned aerial vehicle 100. The flying object system of the fourth embodiment has substantially the same configuration as that of the first to third embodiments. For the same components as those of the first to third embodiments, the same reference numerals are used to omit or simplify the description.

FIG. 15 is a sequence diagram showing an example of a flight path generation procedure in the flight body system 10 of the fourth embodiment. In FIG. 15, it is illustrated that this flight path generation process is mainly performed by the unmanned aerial vehicle 100.

The processing of procedures T61 to T64 is the same as that of procedures T41 to T44 of the third embodiment. The terminal control unit 81 transmits the notification parameters including the flight range AR, parameters, terrain information, and region of interest RI acquired in the procedures T61 to T64 to the unmanned aerial vehicle 100 via the communication unit 85 (T65).

In the unmanned aerial vehicle 100, the UAV control unit 110 receives the notification parameter via the communication interface 150 (T66). The UAV control unit 110 calculates the flight altitude (T67). The method for calculating the flight altitude may be the same as in procedure T4 and procedure T45. The UAV control unit 110 generates a flight path (T68). The method of generating the flight path rt may be the same as in steps T5 and T46.

The UAV control unit 110 derives an imaging angle θi for imaging the region of interest RI (T69). The method for deriving the imaging angle θi may be the same as in the procedure T47. The UAV control unit 110 deletes unnecessary imaging point gpl (T70). The method of deleting the unnecessary imaging point gpl may be the same as in the procedure T48. The UAV control unit 110 performs clustering and calculates the imaging position groups sg1 and sg2 (T71). The clustering method and the calculation method of the imaging position groups sg1 and sg2 may be the same as in the procedure T49. The UAV control unit 110 regenerates the flight path rtn (T72). The method for regenerating the flight path rtn may be the same as in the procedure T50.

The UAV control unit 110 holds parameters including the imaging position wp (corresponding to the imaging point gp having a high imaging cost Cij), the regenerated flight path rtn, and the imaging angle θi in the memory 160, so that the unmanned aerial vehicle 100 can use the unmanned aerial vehicle 100. Each parameter to be used is set (T73). The UAV control unit 110 drives the image pickup unit 220 while flying along the flight path rtn based on the set parameters, and performs aerial photography of the region of interest RI at the image pickup angle θi (T74).

As a result, the unmanned aerial vehicle 100 can perform many processes for generating the flight path rtn in consideration of the region of interest RI on the unmanned aerial vehicle 100 side, and can reduce the processing load of the terminal 80.

Although the present disclosure has been described above using the embodiments, the technical scope of the present disclosure is not limited to the scope described in the above-described embodiments. It will be apparent to those skilled in the art to make various changes or improvements to the embodiments described above. It is clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present disclosure.

The order of execution of each process such as operation, procedure, step, and step in the apparatus, system, program, and method shown in the claims, the specification, and the drawing is particularly "before" and "prior to". , Etc., and can be realized in any order unless the output of the previous process is used in the subsequent process. Even if the scope of claims, the specification, and the operation flow in the drawings are explained using "first", "next", etc. for convenience, it means that it is essential to carry out in this order. is not it.

10 Aircraft system 80 Terminal 81 Terminal control unit 83 Operation unit 85 Communication unit 87 Memory 88 Display unit 89 Storage 100, 100R Unmanned aerial vehicle (UAV)
110 UAV control unit 150 Communication interface 160 Memory 170 Storage 200 Gimbal 210 Rotating wing mechanism 220, 230 Imaging unit 240 GPS receiver 250 Inertial measurement unit 260 Magnetic compass 270 Atmospheric altimeter 280 Ultrasonic sensor 290 Laser measuring instrument 501,502 Building AR flight Range gp Initial imaging point gp, gpl Imaging point hm Ground surface k Sampling position ms Topography ms1 Slope ms2 Back surface rt Flight path sg1, sg2 Imaging position group wp Imaging position

Claims (24)

An information processing device that generates a flight path for an aircraft to fly.
Equipped with a processing unit
The processing unit
Obtain the terrain information of the flight range in which the flying object flies,
Based on the terrain information of the flight range, a flight path including the imaging position in the three-dimensional space for imaging the terrain of the flight range is generated.
Based on the topographical information of the flight range and the flight path, an imaging angle is derived for each imaging position of the flight path.
The processing unit
Obtaining a candidate angle that is a candidate for the imaging angle for imaging the terrain of the flight range,
The first imaging cost, which is the imaging cost when imaging at the candidate angle at the imaging position, is calculated for each candidate angle.
A candidate angle at which the first imaging cost at the imaging position is equal to or greater than the first threshold value is determined by the imaging angle at the imaging position.
Information processing equipment. The processing unit
The terrain of the flight range is sampled to obtain a plurality of sampling positions imaged by the flying object.
A second imaging cost, which is an imaging cost when the sampling position is imaged at the candidate angle at the imaging position, is calculated for each sampling position.
The first imaging cost is calculated by summing up the second imaging costs at each sampling position.
The information processing apparatus according to claim 1 . The shorter the distance between the imaging position and the sampling position, the higher the second imaging cost.
The information processing apparatus according to claim 2 . The larger the internal product value of the normal vector with respect to the ground surface at the sampling position and the imaging vector which is a vector along the imaging direction indicated by the candidate angle, the larger the second imaging cost.
The information processing apparatus according to claim 2 . The processing unit excludes the second imaging cost whose inner product value is a negative value from the calculation target of the first imaging cost.
The information processing apparatus according to claim 4 . The processing unit
Acquire the region of interest included in the flight range and including the position of the image target,
Based on the topographical information of the region of interest and the flight path, an imaging angle is derived for each imaging position of the flight path.
The information processing apparatus according to any one of claims 1 to 5 . The processing unit
Obtaining a candidate angle that is a candidate for the imaging angle for imaging the terrain of the flight range,
A third imaging cost, which is an imaging cost when the region of interest is imaged at the candidate angle at the imaging position, is calculated for each candidate angle.
A candidate angle at which the third imaging cost at the imaging position is equal to or greater than the second threshold value is determined by the imaging angle at the imaging position.
The information processing apparatus according to claim 6 . When the third imaging cost when the third imaging cost is captured at the imaging angle at the first imaging position among the plurality of imaging positions, the processing unit can use the plurality of imaging positions from the plurality of imaging positions. The flight path is generated by excluding the imaging position of 1.
The information processing apparatus according to claim 7 . The processing unit
A plurality of the imaging positions are classified for each imaging target to be imaged from the imaging position to generate a plurality of imaging position groups.
The plurality of imaging position groups are connected to generate the flight path.
The information processing apparatus according to any one of claims 6 to 8 . The information processing device is a terminal provided with a communication unit, and is a terminal.
The processing unit transmits information on the imaging position, the flight path, and the imaging angle to the flying object via the communication unit.
The information processing apparatus according to any one of claims 1 to 9 . The information processing device is a flying object including an image pickup unit.
The processing unit
Control the flight according to the flight path,
An image is imaged at the image pickup angle at the image pickup position of the flight path via the image pickup unit.
The information processing apparatus according to any one of claims 1 to 9 . It is a flight path generation method in an information processing device that generates a flight path for an air vehicle to fly.
The step of acquiring the terrain information of the flight range in which the flying object flies, and
Based on the terrain information of the flight range, a step of generating a flight path including an imaging position in a three-dimensional space for imaging the terrain of the flight range, and
A step of deriving an imaging angle for imaging the terrain of the flight range for each imaging position of the flight path based on the terrain information of the flight range and the flight path.
Including
The step of deriving the imaging angle is
A step of acquiring a candidate angle that is a candidate for the imaging angle for imaging the terrain of the flight range, and
A step of calculating a first imaging cost, which is an imaging cost when imaging at the candidate angle at the imaging position, for each candidate angle, and
The step includes a step of determining a candidate angle at which the first imaging cost at the imaging position is equal to or higher than the first threshold value to the imaging angle at the imaging position.
Flight route generation method. The step of calculating the first imaging cost is
A step of sampling the terrain of the flight range and acquiring a plurality of sampling positions imaged by the flying object.
A step of calculating a second imaging cost, which is an imaging cost when the sampling position is imaged at the candidate angle at the imaging position, for each sampling position.
Includes a step of summing up the second imaging costs at each sampling position to calculate the first imaging costs.
The flight path generation method according to claim 12 . The shorter the distance between the imaging position and the sampling position, the higher the second imaging cost.
The flight path generation method according to claim 13 . The larger the internal product value of the normal vector with respect to the ground surface at the sampling position and the imaging vector which is a vector along the imaging direction indicated by the candidate angle, the larger the second imaging cost.
The flight path generation method according to claim 14 . The step of calculating the first imaging cost includes a step of excluding the second imaging cost whose inner product value is a negative value from the calculation target of the first imaging cost.
The flight path generation method according to claim 15 . The step of deriving the imaging angle is
The step of acquiring the region of interest included in the flight range and including the position of the image target,
Based on the topographical information of the region of interest and the flight path, the imaging angle is derived for each imaging position of the flight path.
The flight path generation method according to any one of claims 12 to 16 . The step of deriving the imaging angle is
A step of acquiring a candidate angle that is a candidate for the imaging angle for imaging the terrain of the flight range, and
A step of calculating a third imaging cost, which is an imaging cost when the region of interest is imaged at the candidate angle at the imaging position, for each candidate angle.
The step includes a step of determining a candidate angle at which the third imaging cost at the imaging position is equal to or greater than the second threshold value to the imaging angle at the imaging position.
The flight path generation method according to claim 17 . The step of generating the flight path is a plurality of imaging when the third imaging cost when imaging at the imaging angle at the first imaging position among the plurality of imaging positions is equal to or less than the third threshold value. A step of excluding the first imaging position from the position to generate the flight path, comprising.
The flight path generation method according to claim 18 . The step of generating the flight path is
A step of classifying a plurality of the imaging positions for each imaging target to be imaged from the imaging position to generate a plurality of imaging position groups, and a step of generating a plurality of imaging position groups.
A step of connecting the plurality of imaging position groups to generate the flight path, and the like.
The flight path generation method according to any one of claims 17 to 19 . The information processing device is a terminal.
Further including a step of transmitting information of the imaging position, the flight path, and the imaging angle to the flying object.
The flight path generation method according to any one of claims 12 to 20 . The information processing device is a flying object and is
Steps to control flight according to the flight path,
A step of capturing an image at the imaging angle at the imaging position of the flight path further comprises.
The flight path generation method according to any one of claims 12 to 20 . For information processing devices that generate flight paths for flying objects to fly,
The step of acquiring the terrain information of the flight range in which the flying object flies, and
Based on the terrain information of the flight range, a step of generating a flight path including an imaging position in a three-dimensional space for imaging the terrain of the flight range, and
A step of deriving an imaging angle for imaging the terrain of the flight range for each imaging position of the flight path based on the terrain information of the flight range and the flight path.
It is a program to execute
The step of deriving the imaging angle is
A step of acquiring a candidate angle that is a candidate for the imaging angle for imaging the terrain of the flight range, and
A step of calculating a first imaging cost, which is an imaging cost when imaging at the candidate angle at the imaging position, for each candidate angle, and
The step includes a step of determining a candidate angle at which the first imaging cost at the imaging position is equal to or higher than the first threshold value to the imaging angle at the imaging position.
Program . For information processing devices that generate flight paths for flying objects to fly,
The step of acquiring the terrain information of the flight range in which the flying object flies, and
Based on the terrain information of the flight range, a step of generating a flight path including an imaging position in a three-dimensional space for imaging the terrain of the flight range, and
A step of deriving an imaging angle for imaging the terrain of the flight range for each imaging position of the flight path based on the terrain information of the flight range and the flight path.
A computer-readable recording medium on which a program for executing a program is recorded.
The step of deriving the imaging angle is
A step of acquiring a candidate angle that is a candidate for the imaging angle for imaging the terrain of the flight range, and
A step of calculating a first imaging cost, which is an imaging cost when imaging at the candidate angle at the imaging position, for each candidate angle, and
The step includes a step of determining a candidate angle at which the first imaging cost at the imaging position is equal to or higher than the first threshold value to the imaging angle at the imaging position.
Recording medium .

Images