JP2019039892A - Optical information processor, method for processing optical information, and program for processing optical information - Google Patents

Abstract

To monitor a flying UAV efficiently.SOLUTION: A TS (total station) 100 includes: a target position calculation unit 109 for an unmanned aircraft for measuring a position of the unmanned aircraft by measuring a distance to an UAV while tracking the unmanned aircraft; a flying height calculation unit 116 and a flying rate calculation unit 117 for calculating a parameter of a flying state of the unmanned aircraft based on the position of the unmanned aircraft measured by the target position calculation unit 109; and an image generation unit 115 for generating an image which shows the parameter of a flying state on a display.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for monitoring an aircraft such as a UAV.

  A technique using an unmanned aerial vehicle (UAV (Unmanned aerial vehicle)) for imaging, surveying, or the like is known. This technique requires visual monitoring of the flying UAV. As a method of monitoring a flying UAV, there is a method of tracking a UAV flying in a TS (total station) (see, for example, Patent Document 1). In this method, the UAV is tracked by using the target tracking function provided by the TS, and since the TS has a telescope and an image capturing function, the UAV can be visually tracked by using this optical system. . Further, it is possible to specify the position of the flying UAV using the laser ranging function provided in the TS.

US2014 / 0210663 Publication

  In the method of tracking UAV with the above-described TS, an automatic tracking function of a target included in the TS is used. In this technique, a UAV is captured and tracked with a search laser beam. The UAV includes a reflecting prism that reflects search laser light in the incident direction, and the UAV is tracked by the TS by detecting reflected light from the reflecting prism on the TS side.

  By the way, in tracking UAV by TS, there is a problem that TS loses sight of UAV. For example, when a bird, electric wire, utility pole, steel tower, tall tree, building, etc. enters between TS and UAV, the search laser light (tracking light) from TS is blocked and TS loses sight of UAV There is.

  When the TS loses sight of the UAV, an operation for searching for the UAV using the TS telescope is performed by the operator or the supervisor, but it is difficult to place the UAV once lost in the field of view of the telescope. Although the field of view can be widened by lowering the magnification of the telescope, it is difficult to find a UAV flying at a distance of several hundred meters at most from a wide field of view.

  If the driver loses sight of the UAV, there is a possibility that an accident such as a collision with a building or a crash may occur, and it is desirable to prevent such an occurrence. In such a background, an object of the present invention is to provide a technique for effectively monitoring a flying UAV.

  The invention according to claim 1 is an unmanned aircraft position measuring unit that measures the position of the unmanned aircraft by measuring a distance to the UAV while optically tracking the unmanned aircraft, and a camera that photographs the unmanned aircraft. A flight state calculation unit that calculates a parameter indicating the flight state of the unmanned aircraft based on the position of the unmanned aircraft measured by the position measurement unit, the parameter indicating the flight state, and the unmanned aircraft captured by the camera This is an optical information processing apparatus that includes an image creation unit that creates an image for displaying the captured image on a display.

  The invention according to claim 2 is the invention according to claim 1, wherein the parameter indicating the flight state is a flight direction of the unmanned aircraft. The invention according to claim 3 is the invention according to claim 1 or 2, wherein the parameter indicating the flight state is one or both of altitude and speed from the ground surface of the unmanned aircraft. .

  The invention according to claim 4 is the invention according to any one of claims 1 to 3, wherein the position of the unmanned aircraft measured by the position measurement unit and the three-dimensional landform information in the flying airspace of the unmanned aircraft An altitude calculation unit that calculates the altitude from the ground surface based on

  According to a fifth aspect of the present invention, in the invention according to any one of the first to fourth aspects, the position of the unmanned aircraft measured by the position measurement unit and an obstacle that obstructs the flight of the unmanned aircraft; It is characterized by comprising a distance calculation unit for calculating the distance between the two.

  According to a sixth aspect of the present invention, in the invention according to any one of the first to fourth aspects of the present invention, the image creation unit is configured to determine whether the unmanned aircraft is based on a distance between the unmanned aircraft and the obstacle. An image displaying a relative positional relationship with the obstacle is created.

  According to a seventh aspect of the present invention, in the invention according to the fifth or sixth aspect, the image creating unit determines whether the unmanned aircraft and the obstacle are based on a distance between the unmanned aircraft and the obstacle. An image that warns of a collision is created.

  According to an eighth aspect of the present invention, in the invention according to any one of the fifth to seventh aspects, the image creating unit is configured to control the unmanned aircraft based on a distance between the unmanned aircraft and the obstacle. An image showing the direction of retraction is created.

  The invention according to claim 9 is the invention according to any one of claims 5 to 8, wherein a distance between the unmanned aircraft and an obstacle that obstructs a flight of the unmanned aircraft is equal to or less than a specified value. The display magnification adjustment unit adjusts the display magnification on the display so that the unmanned aircraft and the obstacle are displayed on the display at the same time.

  The invention according to claim 10 is the invention according to any one of claims 5 to 9, further comprising a laser scanner for obtaining point cloud data, wherein the three-dimensional data of the obstacle is obtained by the laser scanner. It is characterized by being able to.

  The invention according to claim 11 is the invention according to any one of claims 5 to 9, wherein the display is a transmissive head-mounted display.

  The invention according to claim 12 is the invention according to claim 11, further comprising an external orientation element calculation unit that calculates an external orientation element of the head-mounted display, wherein the position of the unmanned aircraft and the head-mounted type are calculated. A position calculating unit that calculates the position of the unmanned aircraft on the display screen of the head-mounted display based on an external orientation element of the display is provided.

  According to a thirteenth aspect of the present invention, in the invention according to the eleventh or twelfth aspect, from a person wearing the head-mounted display based on the position of the unmanned aircraft and an external orientation element of the head-mounted display. A direction calculation unit that calculates the direction of the unmanned aircraft as seen is provided.

  The invention according to claim 14 is the invention according to any one of claims 11 to 13, wherein the external orientation element of the head-mounted display is taken by a camera fixed to the head-mounted display. The calculated position is calculated by a backward dating method based on a plurality of specified points.

  The invention according to claim 15 is a position measurement step of an unmanned aircraft that measures the position of the unmanned aircraft by measuring a distance to the UAV while optically tracking the unmanned aircraft, and photographing for photographing the unmanned aircraft A flight state calculation step for calculating a parameter indicating a flight state of the unmanned aircraft based on the position of the unmanned aircraft measured in the position measurement step, a parameter indicating the flight state, and the unmanned image taken by the camera An optical information processing method comprising: an image creation step of creating an image for displaying a photographed image of an aircraft on a display.

  The invention according to claim 16 is a program that is read and executed by a computer, and measures the position of the unmanned aircraft by measuring the distance to the UAV while the computer optically tracks the unmanned aircraft. Photographed by an aircraft position measurement unit, a flight state calculation unit for calculating a parameter indicating the flight state of the unmanned aircraft based on the position of the unmanned aircraft measured by the position measurement unit, a parameter indicating the flight state, and a camera This is an optical information processing program that causes an image creation unit to create an image for displaying a photographed image of the unmanned aircraft on a display.

  According to the present invention, a technique for effectively monitoring a flying UAV can be obtained.

It is a conceptual diagram of embodiment. It is a block diagram of TS (total station) using invention. It is a flowchart which shows an example of the procedure of a process. It is a flowchart which shows an example of the procedure of a process. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a block diagram of an embodiment. It is a principle figure which shows the principle which calculates | requires the position on the screen of a reticle. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a principle figure which shows the principle of a backward intersection method.

1. First embodiment (configuration)
FIG. 1 shows a block diagram of a TS (total station) which is an example of a position measuring apparatus using the invention. The TS 100 measures the three-dimensional position of the UAV 200 using ranging light while tracking the flying UAV 200. The tracking of the UAV 200 is performed by searching for the reflecting prism 202 provided in the UAV 200 using the search light, and the position of the UAV 200 is measured by irradiating the reflecting prism 202 with the distance measuring light and detecting the reflected light. Done in The TS 100 functions as an optical information processing apparatus using the present invention. With this function, various types of image information are displayed on the smart glasses worn by the operator 500.

  The UAV 200 is steered by a pilot 500 by remote control. The operator 500 uses the controller 501 to perform wireless operation of the UAV 200. Of course, it is also possible to fly the UAV 200 autonomously according to a predetermined flight plan. The UAV 200 is a commercially available model and is not special. The UAV 200 includes a GNSS position measurement device, an IMU, a wireless communication device, a storage device that stores a flight plan and a flight log, and an altimeter.

  At the lower part of the UAV 200, a reflecting prism 202 that is a target for surveying by the TS 100 is attached. Further, the UAV 200 is equipped with a camera 201, which can shoot from the air. The operator 500 wears a smart glass 503, and various information described later processed by the TS 100 is displayed on the smart glass 503.

  TS100 uses its own position measurement device using GNSS, a camera that acquires an image of a survey target (UAV200), a search laser scan function that searches for a target (reflection prism 202 of UAV200), and a distance measuring laser beam. Laser distance measurement function for measuring the distance to the target (reflecting prism 202), function for measuring the direction of the target (horizontal angle and vertical angle (elevation angle or depression angle)), and distance and direction to the target It has a function to calculate the three-dimensional position of the target, a function to communicate with an external device, and a laser scan function to obtain point cloud data.

  By measuring the distance and direction to the target, the position of the target with respect to TS100 can be measured. Here, if the position of the TS 100 is known, the position (latitude / longitude / altitude or XYZ coordinates on the orthogonal coordinate system) of the target (in this case, the UAV 200) in the map coordinate system can be known. This function is a function of a commercially available TS and is not special. Techniques relating to TS are described in, for example, Japanese Unexamined Patent Application Publication Nos. 2009-229192 and 2012-202821. Note that the map coordinate system is a coordinate system that handles map information (for example, latitude, longitude, altitude (elevation)). For example, position information obtained by GNSS is usually described in a map coordinate system.

  Hereinafter, an example of the TS (total station) 100 used in the present embodiment will be described. FIG. 2 shows a block diagram of TS100. The TS 100 includes a camera 101, a target searching unit 102, a distance measuring unit 103, a horizontal / vertical direction detecting unit 104, a horizontal / vertical direction driving unit 105, a data storage unit 106, a position measuring unit 107, a communication device 108, and a target position calculating unit. 109, UAV tracking control unit 111, laser scanner 112, control microcomputer 113, image acquisition unit 114, image creation unit 115, flight altitude calculation unit 116, flight speed calculation unit 117, distance to obstacle calculation unit 119, display magnification An adjustment unit 120, a three-dimensional model creation unit 121, an external orientation element calculation unit 122, a reticle display position calculation unit 123, and a UAV direction calculation unit 124 are provided.

  Each functional unit shown in FIG. 2 may be configured by dedicated hardware, or a component that can be configured by software using a microcomputer may be configured by software. As hardware used to realize the configuration of FIG. 2, various electronic devices (for example, a camera module configuring the camera 101, a wireless module configuring the communication device 108, and the like), various driving using a motor, etc. Examples thereof include a mechanism, a sensor mechanism, an optical component, various electronic circuits, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array). The above hardware configuration is the same for the UAV 200.

  The camera 101 captures a moving image or a still image of a survey target such as the UAV 200 or a target. The camera 101 is configured using a camera module using a CCD or a CMOS sensor, and captures a positioning target (for example, UAV200) via a telescope to obtain image data. Usually, an operator performs an approximate collimation of a target to be surveyed using an image captured by the camera 101 through a telescope, and then performs an accurate collimation of the target by an autonomous operation by an automatic tracking function. .

  The optical axis of the camera 101 coincides with the optical axis of the distance measuring unit 103 described later (the optical axis of the distance measuring laser beam), and the target for specifying the position is captured at the center of the captured image of the camera 101. . The TS 100 includes a small display (for example, a liquid crystal display), and an image captured by the camera 101 can be displayed there. Data of images taken by the camera 101 can be output to the outside and displayed on an appropriate display. In this example, an image captured by the camera 101 is displayed on a smart glass 502 (glasses having a head-mounted display function) worn by a pilot 500 who controls the UAV 200.

  The smart glass 502 is a transmissive head-mounted display that is also called a glasses-type wearable terminal, and displays various information related to the flight of the UAV 200. The smart glass 502 can also see the distance through the screen, and the image output from the TS 100 is displayed over the background visible in the field of view. Smart glasses are glasses-type devices in which a small display is mounted at a position where the glasses lens is located. This display is transmissive, and the other side can be seen through the display.

  A person wearing smart glasses looks at an object through glasses and at the same time virtually forms a screen in front of him (as if the screen is unfolded from several tens of centimeters to several meters ahead of him) ) Displays various images like a normal display. The standard of the image displayed on the smart glass can be the same as the image displayed on the PC or smartphone. Various types of smart glasses are sold by multiple manufacturers.

  The magnification of the image taken by the camera 101 and displayed on the TS100-equipped display and the external display can be varied. The zoom function of the image is realized by a method using an optical method, a method using digital processing, a method combining both, and the like. It is also possible to use a commercially available digital camera as the camera 101 and attach it to the TS 100 externally. Data of an image captured by the camera 101 can be stored in an appropriate storage area in association with data such as a measurement time, a measurement direction, a measurement distance, and a position of the measurement object related to the distance measurement object.

  The target search unit 102 searches for a target (a reflection prism 202 of the UAV 200) using a search laser beam having a triangular pyramid or fan beam. The target search is performed using TS100 as a reference position. The search and tracking of the target (reflecting prism) by the TS is described in, for example, Japanese Patent No. 5124319.

  The distance measuring unit 103 measures the distance to the target using the distance measuring laser beam. The distance measuring unit 103 includes a light emitting element for distance measuring laser light, an irradiation optical system, a light receiving optical system, a light receiving element, a distance calculating unit, and an optical path for reference light. The distance to the object is calculated from the phase difference between the distance measuring light reflected from the object and the reference light. The method for calculating the distance is the same as that for normal laser ranging.

  The horizontal / vertical direction detection unit 104 measures the horizontal direction angle and the vertical direction angle (elevation angle and depression angle) of the target measured by the distance measurement unit 103. The housing portion including the optical system of the target search unit 102, the distance measurement unit 103, and the camera 101 can be controlled in horizontal rotation and elevation angle (decline angle), and the horizontal direction angle and the vertical direction angle are measured by an encoder. . The output of the encoder is detected by the horizontal / vertical direction angle detection unit 104, and the horizontal direction angle and the vertical direction angle (elevation angle and depression angle) are measured.

  The horizontal / vertical direction driving unit 105 includes a motor that performs horizontal rotation and elevation angle control (and depression angle control) of a housing portion including the optical system of the target search unit 102, the distance measurement unit 103, and the camera 101, and a drive circuit for the motor And a control circuit for the drive circuit. Note that a laser scanner 112, which will be described later, is also disposed in this housing portion. The data storage unit 106 stores a control program necessary for the operation of the TS 100, various data, survey results, and the like.

  The position measuring unit 107 measures the position of the TS 100 using GNSS. The position measuring unit 107 has a function of performing both relative positioning and single positioning. In an environment where relative positioning is possible, measurement of the position of TS100 using relative positioning is preferable. However, when relative positioning is difficult, the position of TS100 is measured by independent positioning. In the positioning of the UAV 200 by the TS 100, it is necessary to obtain the position of the TS 100 in the map coordinate system as known data in advance. In actual operation, there are a case where the TS 100 is installed in a position where the coordinates are known in advance, and a case where the position measuring unit 107 measures the position where the TS 100 is installed. In the former case, positioning by the position measuring unit 107 is not necessary.

  The communication device 108 communicates with an external device. The TS 100 can be operated by an external terminal (dedicated terminal, PC, tablet, smartphone, etc.), and communication at this time is performed using the communication device 108. In addition, the communication device 108 receives various data necessary for the operation of the TS 100 and outputs various data acquired by the TS 100 to the outside. For example, the TS 100 can acquire various information related to surveying by accessing a data server that handles map information and terrain information from the TS 100 via the Internet line. In addition, a configuration in which some of the functions illustrated in FIG. 2 are performed by an external device is possible, and in this case, various types of data are exchanged via the communication device 108. Further, the communication device 108 receives a control signal transmitted from the controller 501 to the UAV 200. Based on this control signal, the control stick operation instruction display illustrated in FIG. 5 is performed.

  The target position calculation unit 109 calculates the position (coordinates) of the target with respect to the TS 100 from the distance and direction to the target (in this case, the reflecting prism 202 mounted with the UAV 200). Here, the distance to the target is obtained by the distance measuring unit 103, and the direction of the target is obtained by the horizontal / vertical direction detecting unit 104. Since the position of the TS 100 serving as the reference position is specified by the position measuring unit 107, the position of the target in the map coordinate system can be obtained by obtaining the position of the target with respect to the TS 100.

  The UAV tracking control unit 111 performs control for tracking the supplemented UAV. That is, the direction of the TS is controlled in correspondence with the incident direction of the tracking light (tracking light reflected from the UAV) detected by the target search unit 102, and the movement of the TS100 so that the optical axis of the TS100 always faces the UAV moving in the air. Control is performed by the UAV tracking control unit 111. Specifically, the incident direction of the tracking light reflected from the target (reflecting prism 202) with respect to TS100 is detected, and based on the detected direction, the optical axis of TS100 (the optical axis of the distance measuring light from distance measuring unit 103) is set at the position of reflecting prism 202. ) Is always performed by the UAV tracking control unit 111 so as to output a control signal to the horizontal / vertical direction driving unit 105.

  The laser scanner 112 obtains point cloud data by using the distance measuring laser light as scanning light. The point cloud data is data that captures an object as a set of points whose three-dimensional coordinates are known. In this example, the target search unit 102 and the laser scanner 112 have different configurations, and the laser scanner 112 operates separately from the target search unit 102. Here, the wavelength of the laser light used by the laser scanner is selected to be different from the laser light used by the target search unit 102 so as not to interfere with the laser light used by the target search unit 102. Laser scanners for obtaining point cloud data are described in, for example, Japanese Patent Application Laid-Open Nos. 2010-151682, 2008-268004, US Pat. No. 8,767,190, and US Pat. No. 7,969,558. As the laser scanner 112, an electronic laser scanner that performs scanning electronically (Solid state optical phased array method) may be used. This technique is described in, for example, US Publication No. US2015 / 0293224.

  Hereinafter, the laser scanner 112 will be described. The laser scanner 112 is a distance measuring laser beam irradiation unit, a light receiving unit that receives the distance measuring laser beam reflected from the object, and a distance measuring unit that detects the distance to the object based on the flight time of the distance measuring laser beam. And a distance measuring direction detector for detecting the irradiation direction (ranging direction) of the distance measuring laser beam. Further, the laser scanner 112 includes a position calculation unit for a distance measurement target point that calculates a three-dimensional position of a reflection point of the distance measurement laser light based on its own position, distance measurement distance, and distance measurement direction. And a scan control unit for controlling the light receiving direction of the reflected light (the optical axis of the distance measuring light).

  The distance measuring laser light is output in pulses at a specific repetition frequency, and irradiated to a specific range in a point while being scanned. The distance from the flight time of the laser beam for distance measurement to the reflection point is calculated. Usually, the distance to the object is calculated from the phase difference between the reference light flying in the reference optical path provided in the apparatus and the distance measuring light irradiated to the object and reflected from the object. A three-dimensional position with the origin of the position of the laser scanner 112 at the reflection point is calculated from the distance measured, the irradiation direction of the laser light for distance measurement, and the position of the laser scanner 112. Point cloud data can be obtained by measuring a number of positions of the reflection points. Here, the position and orientation of the laser scanner 112 in the TS 100 are acquired in advance as known information, and the position of the laser scanner 112 in the map coordinate system is calculated based on the positioning data of the position measuring unit 107. Therefore, the three-dimensional coordinates in the map coordinate system of each scanned point (reflection point) can be acquired.

  The distance measuring laser light is scanned and irradiated at a specific oscillation frequency, whereby the three-dimensional coordinates of each of a number of reflection points on the object are acquired. A set of a large number of reflection points on this object becomes point cloud data. In the point cloud data, an object is captured as a set of points whose three-dimensional positions are specified.

  The laser scanner 112 can acquire the RGB intensity of the reflection point. The RGB intensity is obtained by selecting the reflected light with an R filter, a G filter, and a B filter and detecting the light intensity of each color. Therefore, data on the RGB intensity of each point of the obtained point cloud data is also obtained. In addition, it is not limited to RGB, The form which acquires the intensity | strength of one or several specific color information is also possible. In addition, a configuration in which the intensity of reflected light of the scan light is detected separately from (or in addition to) the color information is also possible. The UAV 200 may be distinguished (identified) from other flying objects such as birds by using at least one of the color intensity and the reflection intensity of the scanning light.

  The control microcomputer 113 controls the processing procedures of FIGS. 3 and 4 described later and the operation of the entire TS100. The image acquisition unit 114 acquires image data of an image captured by the camera 101. The image creation unit 115 creates an image to be displayed on the smart glass 502. FIGS. 5 to 7 and FIGS. 10 to 13 show examples of images created by the image creation unit 115 and displayed on the smart glass 502.

  The flight altitude calculation unit 116 calculates the ground altitude of the UAV 200 that the TS 100 is positioning. The ground altitude of the UAV 200 is obtained by subtracting the altitude of the ground surface at the longitude and latitude of the UAV 200 position from the altitude of the position of the UAV 200 measured by the TS 200. The altitude of the ground surface is acquired from the map information, but if the map information is not obtained, the altitude value of the installation position of TS100 is adopted.

  The flight speed calculation unit 117 calculates the speed (speed and direction) of the UAV 200 tracked by the TS 100. Since the position of the TS100 is measured every moment, the velocity vector is obtained by calculating the tangent vector of the movement path, and the movement direction (θx, θy, θz) and the velocity (Vx) of the UAV 200 are obtained from this velocity vector. , Vy, Vz).

  When there is an obstacle that obstructs the flight of the UAV 200, the distance calculation unit 119 to the obstacle calculates a distance from the UAV 200 to the obstacle. Obstacles are buildings, trees, steel towers, birds, and the like. The three-dimensional data related to the obstacle is acquired from existing three-dimensional data (for example, three-dimensional map coordinate data) or point cloud data obtained from the laser scanner 112. Since the bird data is obtained from the existing data, it is obtained from the point cloud data obtained from the laser scanner 112.

  The display magnification adjustment unit 120 adjusts the display magnification of the image displayed on the smart glass 502. For example, when the UAV 200 is likely to interfere with an obstacle, the display magnification is adjusted so that both are within the field of view.

  The three-dimensional model creation unit 121 creates a model that captures the UAV 200 and the obstacle in a three-dimensional manner. This model is used when the positional relationship between the UAV 200 and the obstacle is displayed on the smart glass. This point will be described later.

  The reticle display position calculation unit 123 calculates the display position of a reticle (guide display such as Δ, □, ○, ◎, × indicating the position in the screen) on the display screen of the smart glass 502. The display position of the reticle is the position on the display screen of the UAV 200 (position specifying object) that can be seen through the smart glass 502. Therefore, the reticle display position calculation unit 123 calculates the position on the display screen of the UAV 200 (position specifying object) that can be seen through the smart glass 502. The direction calculation unit 124 calculates the direction of the UAV 200 that is out of the visual field of the smart glass 502. Details of the processing of the reticle display position calculation unit 123 and the direction calculation unit 124 will be described later.

  The operations of the TS 100 and the smart glass 502 are performed using a dedicated terminal and a smartphone or tablet in which scanning application software is installed. For example, various displays displayed on a smart glass 502 to be described later are switched using a smartphone or a tablet. Of course, a dedicated controller may be prepared and the TS 100 and the smart glass 502 may be operated. In addition, a mode in which the operation function of the TS100 is added to the controller 501 of the UAV 200, a mode in which the UAV 200 and the TS 100 are operated with a smartphone, and the like are possible.

(Example of processing)
Hereinafter, an example of processing performed in the TS 100 will be described. 3 and 4 are flowcharts illustrating an example of a processing procedure. The program for executing the processes of FIGS. 3 and 4 is stored in an appropriate storage area such as the data storage unit 106, and is read out and executed from there. A mode in which the program is stored in an appropriate storage medium, a data server, or the like, read from the program, and executed is possible.

  First, a process of capturing the UAV 200 by the TS 100 and specifying the position by laser ranging will be described. FIG. 3 shows an example of this process. In this processing, first, the search for the reflecting prism 202 of the UAV 200 is performed (step S101). This process is performed by the target search unit 102. Further, a laser scan for acquiring point cloud data by the laser scanner 112 is started.

  For example, the TS 100 captures the UAV 200 while it is landing on the ground before the UAV 200 takes off. Then, laser scanning of the captured UAV 200 is started by the laser scanner 112, and acquisition of the UAV 200 and surrounding point cloud data is started. Thereafter, the UAV 200 starts to fly, the UAV 200 is tracked by the TS 100, and at the same time, the laser scanning of the UAV 200 and its surroundings is continuously performed. Further, tracking of the UAV 200 may be performed by the following method. In this case, when the flight of the UAV 200 is started, the UAV 200 first rises from the ground to a specified height and stops there. This stop position is defined in advance in the flight plan, and is also grasped on the TS 100 side. Therefore, the TS 100 captures the UAV 200 while aiming at the position of the UAV 200 that rises and stops. Further, laser scanning with the UAV 200 within the scanning range is started by the laser scanner 112, and acquisition of the UAV 200 and surrounding point cloud data is started. As the range of the laser scan at this stage, for example, a range with a radius of about 20 to 50 m centering on the UAV 200 is selected.

  When the target reflection prism 202 is captured, control is performed to direct the optical axis of the TS 100 (the optical axis of the distance measuring unit 103) toward the reflection prism 202 (step S102). After the reflection prism 202 is captured, the UAV tracking control unit 111 controls the optical axis of the distance measuring unit 103 to be directed to the reflection prism 202.

  Next, ranging light is emitted from the ranging unit 103 toward the reflecting prism 202, and the position of the UAV 200 is measured (step S103). The measurement of the position of the UAV 200 is performed by the target position calculation unit 109.

  The processing of steps S101 → S102 → S103 is repeated, and the moving UAV 200 is continuously tracked. As a result, the position (three-dimensional coordinates) of the UAV 200 is acquired every moment, and the position and movement path of the UAV 200 are grasped on the TS 100 side.

  The UAV 200 is photographed by the camera 101 while the UAV 200 is being tracked by the TS 101. A captured image of the UAV 200 by the camera 101 is displayed on the smart glass 502. Hereinafter, processing related to an image displayed on the smart glass 502 will be described. This process is the same when displaying an image captured by the camera 101 on a display such as a smartphone, a tablet, or a PC.

  FIG. 4 shows an example of processing related to an image displayed on the smart glass 502. The processing of FIG. 4 is performed in a state where the processing of FIG. 3 is performed, the UAV 200 is captured by the TS 100, and the positioning is continuously performed. First, the ground height of the UAV 200 is calculated by the height calculation unit 116 (step S111). Next, the flight speed calculation unit 117 calculates the speed (direction and speed) of the UAV 200 (step S112).

  Next, the UAV 200 image captured by the camera 101 is displayed on the display screen of the smart glass 502 (step S113). Next, information on the flight state of the UAV 200 obtained at this stage is displayed on the display screen of the smart glass 502 (step S114). Examples of the flight state information include information on ground altitude and speed (flight direction and flight speed). Examples of the flight state information include an ascent rate, a descending rate, a turning radius, a deviation from the flight plan route, a distance from the TS 100, and the like. The processing of steps S113 and S114 is performed by the image creation unit 115. FIG. 5 shows an example of the state of the field of view that the operator 500 sees through the smart glass 502 in the step S114.

  Next, the calculation of the distance between the UAV 200 and the obstacle is performed by the “distance calculation unit 119 between the obstacle” (step S115). Next, it is determined whether or not the distance calculated in the staple S115 is equal to or less than a predetermined value (for example, 15 m or less) (step S115). If not, the process from step S111 is repeated. By repeating the processes in and after step S111, the altitude and speed information displayed on the smart glass 502 is continuously updated.

  In step S117, the display magnification on the smart glass 502 is adjusted so that both the UAV 117 and the target obstacle are displayed on the screen. This process is performed by the magnification adjustment unit 120. An example of a display screen obtained as a result of this processing is shown in FIG. By displaying the UAV 200 and the obstacle on the screen simultaneously as shown in FIG. 6, the operator 500 can grasp the positional relationship between the UAV 200 and the obstacle.

  Further, when the distance between the UAV 200 and the obstacle is equal to or less than the distance determined to be dangerous (for example, 10 m or less), a warning display screen is displayed on the smart glass 502 (step S118). This process is performed by the image creation unit 115. In this display, it is displayed that the UAV 200 may collide with an obstacle, and further, the distance to the obstacle, the direction, the expected contact time, the direction suitable for avoidance, and the like are displayed. . FIG. 7 shows an example of the display screen at this time.

2. Second Embodiment When a UAV 200 and an obstacle exist on the optical axis of the TS 100, the UAV 200 and the obstacle may overlap in the image obtained by the camera 101 of the TS 100, and it may be difficult to understand the distance between them. In this case, a viewpoint change display described below is displayed on the smart glass 502. The viewpoint change display is created by the image creation unit 115 based on the 3D model created by the 3D model creation unit 121.

  The three-dimensional model creation unit 121 creates a dummy model of the UAV 200 and a three-dimensional model of an obstacle. The dummy model of the UAV 200 is a dummy model converted into three-dimensional data simulating the UAV 200 arranged at the three-dimensional position calculated by the target position calculation unit 109. The orientation of this dummy model is not accurate, but is not important here.

  There are a method of acquiring a three-dimensional model of an obstacle from a method of acquiring from a database of three-dimensional map information and a method of acquiring from a point cloud data obtained from a laser scanner 112. However, the point cloud data obtained from the laser scanner 112 has an occlusion problem, and the data of the blind spot from the TS 100 is missing. Therefore, the missing part was modeled by complementing the data. Create an image.

  The image creation unit 115 creates an image of the three-dimensional model including the UAV 200 and the obstacle as viewed from a direction perpendicular or horizontal to a line connecting the UAV 200 and the obstacle. In this image, the UAV 200 is modeled, and an obstacle is also modeled in some cases, resulting in an unnatural image. However, since this is an image for grasping the sense of distance between the two, this point is allowed.

  Here, since the three-dimensional positions of the UAV 200 and the obstacle are known, the separation distance between them can be known. Therefore, the distance between them is displayed in the above image. In addition, the moving direction of the UAV and other various information shown in FIGS. In this image, even when the UAV 200 and the obstacle overlap on the optical axis of the TS 100, the positional relationship between the UAV 200 and the obstacle can be visually grasped.

3. Third Embodiment FIG. 5 shows a field of view of the smart glass 502, that is, a state where the UAV is seen small through the smart glass 502. In this embodiment, a reticle serving as a guide display (display of a mark) is displayed at a position on the UAV screen that can be seen through the smart glass 502. By displaying this reticle, the position of the UAV 200 on the screen of the smart glass is shown, and the fact that the TS 100 is capturing the UAV 200 is displayed.

  In FIG. 5, the UAV 200 is visible. However, when the flight position of the UAV 200 is far, the UAV 200 may not be visible (or difficult to see) visually. In addition, it may be difficult to distinguish them from birds. In such a case, it is convenient that a reticle for guiding and displaying the position of the UAV 200 is displayed on the screen of the driver 500 on the smart glass 502, in other words, within the field of view of the driver 500 over the smart glass 502. is there.

  FIG. 10 shows an example in which a reticle for guiding the position of the UAV 200 on the screen of the smart glass 502 is displayed. In the case of FIG. 10, an example in which the UAV 200 looks relatively large is shown. However, when the distance to the UAV 200 is long and it is difficult to visually recognize or difficult to distinguish from a bird, the reticle display is displayed over the smart glass 502. By guiding the position of the UAV 200 in the visible field of view, it becomes easy to visually recognize the UAV 200 directly.

  Hereinafter, a mechanism for causing the smart glass 502 to perform the display of FIG. 10 will be described. The process of causing the smart glass 502 to perform the display of FIG. 10 is performed by the reticle display position calculation unit 123 of FIG.

  The reticle display position calculation unit 123 performs the following processing. First, the external orientation element calculation unit 122 calculates the external orientation elements (direction and position) of the smart glass 502. The calculation of the external orientation element of the smart glass 502 will be described later.

  The direction of the smart glass 502 is defined by the front direction. The position of the smart glass 502 is a virtual viewpoint position of the person wearing the smart glass 502 (in this case, the operator 500). Since this viewpoint position is set when the smart glass 502 is designed, that position is used. This position is described in a coordinate system (for example, a map coordinate system) handled by TS100.

  If the position of the smart glass 502 is known, the position of the UAV 200 is measured by the TS 100 and is known. Therefore, a direction line connecting the smart glass 502 and the UAV 200 is set, and the direction line and the display screen of the smart glass 502 (of the driver 500) The point of intersection with the screen virtually displayed in front of the eyes is obtained as the reticle display position. This reticle display position is a position where the UAV 200 can be seen if it can be visually recognized. In this way, the position of the reticle in the field of view of the smart glass 502 is calculated.

  For example, in FIG. 9, the point P <b> 1 is the three-dimensional position of the UAV 200, and the point O is the position of the viewpoint of the person (operator 502) who is wearing the smart glass 502. Here, an intersection point p1 between the direction line connecting the points P1 and O and the display screen of the smart glass 502 virtually arranged is calculated as the display position of the reticle.

  Based on the calculation result, the image creation unit 115 performs a process of displaying a reticle on the screen of the smart glass 502. In this way, the image illustrated in FIG. 10 is displayed on the smart glass 502.

  Next, calculation of the external orientation element of the smart glass 502 performed by the external orientation element calculation unit 122 will be described. First, when the UAV 200 is locked by the TS 100 at the start of flight of the UAV 200, calibration processing is performed to obtain the initial value of the external orientation element. At this time, the UAV 200 stops at a specified altitude, and the UAV 200 is locked (captured) by the TS 100.

  At this time, the display of the smart glass 502 is set to the calibration mode, and the screen shown in FIG. 11 is displayed. The screen of FIG. 11 shows an example in which a collimation guide display is displayed at a specified position (in this case, the center) in the screen (in the visual field). The pilot 500 moves his / her head so that the UAV 200 can be seen in the center of the collimation guide display, and captures the image of the UAV 200 in the center of the collimation guide display. Calibration processing is executed in this state. This process is performed, for example, by operating a smartphone used as an operation terminal.

  Here, the pilot 500 carries the position specifying device 303 using GNSS (for example, the GPS function of a smartphone is used), and its positioning data is sent to the TS 100. The external orientation element calculation unit 122 performs relative positioning using the positioning data sent from the pilot 500 and the positioning data of the position measurement unit 107, and calculates the three-dimensional position of the pilot 500's viewpoint.

  For example, when using the GPS function of a smartphone, when a smartphone is put in a breast pocket in advance, a correction value such as when it is put in a pants pocket is prepared, and the viewpoint position of the operator 500 is calculated using the correction value. Is calculated.

  In this way, data of the position of the viewpoint of the operator 500 and the position of the UAV 200 viewed from the position of this viewpoint are obtained. Here, the direction from the viewpoint position of the operator 500 toward the position of the UAV 200 is the direction of the smart glass 502. As described above, the initial value of the external orientation elements (position and orientation) of the smart glass 502 is calculated.

  This external orientation element changes as the pilot 500 moves and moves his head. The external orientation element calculation unit 122 corrects the external orientation element of the smart glass 502 corresponding to this change. The correction of the external orientation element of the smart glass 502 is detected based on the outputs of the three-dimensional electronic compass 301 and the three-dimensional acceleration sensor 302 shown in FIG.

  The three-dimensional electronic compass 301 and the three-dimensional acceleration sensor 302 are attached to the smart glass 502 and detect a change in the orientation of the smart glass 502. The three-dimensional electronic compass 301 detects the three-dimensional direction of the smart glass 502. The three-dimensional acceleration sensor 302 detects a three-dimensional component of acceleration applied to the smart glass 502. The three-dimensional electronic compass 301 and the three-dimensional acceleration sensor 302 use components used in smartphones.

  Outputs of the three-dimensional electronic compass 301 and the three-dimensional acceleration sensor 302 are input to the TS 100. For data transmission, a wireless LAN line or the like is used. In response to the outputs of the three-dimensional electronic compass 301 and the three-dimensional acceleration sensor 302, the external orientation element calculation unit 122 corrects the orientation of the smart glass 502 as needed. Further, the position of the smart glass 502 is corrected from time to time based on the change in the positioning data of the GNSS position measuring device 303 carried by the operator 500. Thus, the external orientation element of the smart glass 502 is corrected at any time, and the process related to the reticle display illustrated in FIG. 10 is performed using the external orientation element that is corrected at any time.

  In the correction of the external orientation element of the smart glass 502 described above, errors gradually accumulate, and accuracy is lost in the process of correction. Hereinafter, a process for maintaining the accuracy of the external orientation element of the smart glass 502 will be described. In this process, TS100 is used as a collimation target. That is, as shown in FIG. 12, the operator 500 moves the head so that the TS 100 is positioned at the center of the collimation cross guide. In this state, the external orientation element 122 is recalculated.

At this time, the external orientation element calculation unit 122 performs the following processing. In this process, information on the position of the smart glass (the viewpoint position of the operator 500) and the position of the TS100 is acquired,
A direction from the position of the smart glass 502 toward the TS 100 is calculated. Thus, the external orientation element of the smart glass 502 is recalculated. The external orientation element of the smart glass 502 changes as the operator 502 moves and moves his head, but the correction is made at any time in the manner described above.

  The calculation of the external orientation element of the smart glass 502 by collimating the TS 100 may be used for calculating the initial value in the first stage. When the UAV 100 is in a position where it can be easily collimated, the external orientation element of the smart glass 502 may be recalculated (calibrated) by the method described in relation to FIG. Further, the external orientation element may be recalculated (or initialized) by placing the target whose position is specified in a range that can be seen by the operator 500 and collimating the target.

  By recalculating the external orientation elements of the smart glass 502 as described above, it is possible to suppress a decrease in the accuracy of the reticle display in FIG. 10 due to a decrease in the accuracy of the external orientation elements. A form in which a GNSS position specifying device and an IMU (inertial measurement device) are mounted on the smart glass 502 and the external orientation elements (position and orientation) of the smart glass 502 are measured in real time is also possible. If there is no room for mounting the GNSS position identification device and the IMU on the smart glass 502, the GNSS position identification device and the IMU are attached to the head-mounted helmet or the head-mounted auxiliary device. A form in which the external orientation element of the smart glass 502 is acquired by attaching a part-mounted auxiliary device to the head is also possible.

4). Fourth Embodiment A method in which a camera is fixed to a smart glass and an external orientation element of the smart glass is calculated based on an image taken by the camera is also possible. In this case, a plurality of targets whose positions are specified, a small camera attached to the smart glass, and a target extraction unit that extracts the target from the image are necessary, and the external orientation element calculation unit 122 is smart by the backward intersection method using the target. Calculation of the external orientation element of the glass is performed. The calculation of the external orientation element of the camera using the backward intersection method is described in, for example, Japanese Patent Application Laid-Open No. 2013-186816.

  Hereinafter, calculation of the external orientation element of the smart glass using the captured image of the camera will be described. In this case, a camera that shoots a moving image is attached to the smart glass. The relationship between the position and orientation of the smart glass and the camera is acquired in advance as known data. Further, three or more identifiable targets whose positions (three-dimensional coordinates) are specified in advance are arranged in a range that can be seen by a person wearing a smart glass. At this time, the targets are arranged so that three or more targets are always photographed by the camera even if a human moves and the direction of the head changes. As the target, a form identified by a number or a color code can be used. Examples of the color code target are described in JP 2011-053030 A and JP 2011-053031 A, for example.

  Since the main purpose of the camera is to shoot the target, the camera is set in a direction that allows the target to be shot effectively. Further, if the orientation of the camera in the horizontal direction is such that the target can effectively shoot, it is not necessary to match the orientation of the smart glasses. For example, the camera may be directed to the side or back of the person wearing the smart glasses.

  In calculating the external orientation element, first, three or more targets are photographed with the camera in an initial state. Then, the target is identified and extracted from the captured image by image processing. Since the three-dimensional coordinates of each target are known, the external orientation element of the camera can be calculated by the backward intersection method. When the smart glass (camera) moves, the external orientation element of the camera changes. However, if 3 or more targets are captured, the external orientation element of the camera can be calculated by the backward intersection method. In this way, the calculation of the fluctuating external orientation element of the camera is continuously performed.

  FIG. 14 shows the principle of the backward intersection method. In this case, the point O is the position of the camera (the position of the photographing viewpoint), the points P1, P2, and P3 are the positions of the target, and p1, p2, and p3 are the positions of the target image on the screen of the image photographed by the camera. It is. Here, the points P1, P2, and P3 are specified (measured) in advance and known, and p1, p2, and p3 are obtained from the captured image. Then, a direction line connecting P1 and p1, a direction line connecting P2 and p2, and a direction line connecting P3 and p3 are set, and the intersection point O of these three direction lines is obtained to obtain the camera position O.

  Further, the optical axis of the camera is mathematically obtained by obtaining a direction line connecting the point O and the screen center. By obtaining the optical axis of this camera, the direction of the optical axis from the point O is obtained as the direction of the camera. Thus, the position and orientation of the camera fixed to the smart glass, that is, the external orientation element is calculated. Since the relative positional relationship and orientation relationship between the camera and the smart glass are known, the external orientation element of the smart glass can be obtained by obtaining the external orientation element of the camera.

  By continuously taking images with the camera and continuously performing the above-described processing, the external orientation elements that fluctuate in the smart glass are sequentially calculated and continuously updated.

  For example, a camera for photographing the ground surface around the operator 500 is fixed to the smart glass 502 of FIG. Further, a plurality of targets are arranged around the pilot 500 and their positions are measured. The relationship between the position and orientation of the smart glass 502 and the camera is acquired in advance. Here, in a state where the operator 500 is operating (or monitoring) the UAV 200, the targets are arranged so that three or more targets are always photographed by the camera.

  Even if the operator 502 moves his / her head and the external orientation element of the smart glass 502 changes, the external orientation element of the smart glass 502 using the above target is calculated, and the external orientation of the smart glass 502 at that time is calculated. The element is retrieved. This process is continuously performed, and the external orientation element of the smart glass 502 is constantly updated. For example, the process related to the display in FIG. 10 is performed using the dynamically updated external orientation element.

  In this method, a camera and an image processing unit that extracts targets from captured images are necessary, and positioning of multiple targets is required in advance. Can be acquired with high accuracy.

  A plurality of methods for obtaining the external orientation elements of the smart glass 502 described in the present specification can be used in combination. In order to display the reticle shown in FIG. 10, the position data of the UAV 200 is necessary. As the position data, the positioning data of the GNSS position measuring device mounted on the UAV 200 can be used.

  Using a stereo camera as the camera, the position of the feature point extracted from the stereo image is obtained by the forward intersection method to obtain point cloud data, and this point cloud data is used as the orientation point for the external orientation element of the camera (external orientation element for smart glasses) ) Can also be calculated. Also, using a single camera, a stereo image is obtained by changing the viewpoint to obtain point cloud data in the same manner as described above, and an external orientation element of the camera based on this point cloud data (external orientation element of the smart glass) ) Can also be calculated. In these cases, a target whose position is specified is necessary for obtaining the initial value, but since the point cloud extracted from the image is also used for the orientation, the number can be reduced as compared with the case where only the above-described target is used. . Japanese Unexamined Patent Application Publication No. 2013-186816 discloses a technique for obtaining point cloud data from a stereo image and a technique for obtaining a stereo image from a single camera and obtaining point cloud data therefrom.

5. Fifth Embodiment Here, an example will be described in which, when the operator 500 loses sight of the UAV 100, a display for guiding the direction of the UAV 100 viewed from the operator 500 is displayed on the display screen of the smart glass 502. Although TS100 is tracking UAV 200, pilot 500 may miss UAV 100. In this case, the function of the UAV direction calculation unit 124 performs display for guiding the direction of the UAV 100 on the display screen of the smart glass 502 (the field of view that the operator 500 is viewing through the smart glass).

  FIG. 13 shows an example of the display screen at that time. In this case, the UAV 100 can be captured when the line of sight is directed in the upper left direction. Hereinafter, the function of the UAV direction calculation unit 124 will be described. First, TS100 captures. The position of the UAV 200 calculated by the target position calculation unit 109 is acquired. Next, the external orientation element of the smart glass 502 calculated by the external orientation element calculation unit 122 is acquired.

  Next, a vector orthogonal to the optical axis (front direction) of the smart glass 502 and directed to the UAV 200 is obtained, and this vector is orthogonally projected on the display surface of the smart glass 502 and displayed on the smart glass 502 as an arrow. Thus, an arrow display (guide display) indicating the direction of the UAV 200 shown in FIG. 13 is displayed on the smart glass 502. Looking at the image in FIG. 13, the operator 502 can capture the UAV 200 by looking at the direction of the UAV 200 by turning his face in the direction of the arrow. At this time, since the reticle illustrated in FIG. 10 is displayed when the UAV 200 enters the field of view, even if the UAV 200 is far and difficult to see, the UAV 200 can be easily viewed.

(Other)
The smart glass 502 may be worn by a person other than the operator 500 of the UAV 200. As a method for guiding the position of the UAV on the display screen, it is possible to display an arrow, or to highlight which divided screen in the four-divided or nine-divided screen is displayed. . It is also possible to select a point or an object whose position other than the UAV is specified as an object to be subjected to the reticle display of FIG. The technique illustrated in FIG. 10 can be used for monitoring, searching, confirming, etc., an object whose position is specified. The invention disclosed in this specification can be used in a technique for effectively visualizing an object.

  100 ... TS (total station), 200 ... UAV, 201 ... camera, 200 ... reflecting prism, 500 ... UAV driver, 501 ... controller, 502 ... smart glass.

Claims (16)

A position measuring unit for the unmanned aircraft that measures the position of the unmanned aircraft by measuring the distance to the UAV while optically tracking the unmanned aircraft;
A camera for photographing the unmanned aerial vehicle;
Based on the position of the unmanned aircraft measured by the position measurement unit, a flight state calculation unit that calculates a parameter indicating the flight state of the unmanned aircraft;
An optical information processing apparatus comprising: an image creating unit that creates an image for displaying a parameter indicating the flight state and an image of the unmanned aircraft taken by the camera on a display.   The optical information processing apparatus according to claim 1, wherein the parameter indicating the flight state is a flight direction of the unmanned aircraft.   The optical information processing apparatus according to claim 1, wherein the parameter indicating the flight state is one or both of an altitude and a speed from a ground surface of the unmanned aircraft. The altitude calculation part which calculates the altitude from the said ground surface based on the position of the said unmanned aircraft which the said position measurement part measured, and the three-dimensional terrain information in the flight space of the said unmanned airplane is provided. The optical information processing apparatus according to one item.   The distance calculation part provided with the distance calculation part which calculates the distance between the position of the unmanned aircraft which the position measurement part measured, and the obstruction which becomes the obstacle of the flight of the unmanned airplane is given. Optical information processing device.   The optical information processing apparatus according to claim 5, wherein the image creation unit creates an image that displays a relative positional relationship between the unmanned aircraft and the obstacle based on a distance between the unmanned aircraft and the obstacle. .   The optical information processing apparatus according to claim 5 or 6, wherein the image creation unit creates an image that warns of a collision between the unmanned aircraft and the obstacle based on a distance between the unmanned aircraft and the obstacle. .   The optical information processing according to any one of claims 5 to 7, wherein the image creation unit creates an image indicating a direction in which the unmanned aircraft is retracted based on a distance between the unmanned aircraft and the obstacle. apparatus.   The unmanned aircraft and the obstacle are displayed on the display at the same time when the distance between the unmanned aircraft and the obstacle that obstructs the flight of the unmanned aircraft is equal to or less than a predetermined value. The optical information processing apparatus according to claim 5, further comprising a display magnification adjustment unit that adjusts a display magnification of the display. Equipped with a laser scanner to obtain point cloud data,
The optical information processing apparatus according to claim 5, wherein three-dimensional data of the obstacle is obtained by the laser scanner.   The optical information processing apparatus according to claim 1, wherein the display is a transmissive head-mounted display. An external orientation element calculation unit for calculating an external orientation element of the head-mounted display;
The optical information according to claim 11, further comprising: a position calculation unit that calculates a position of the unmanned aircraft on a display screen of the head-mounted display based on the position of the unmanned aircraft and an external orientation element of the head-mounted display. Processing equipment.   The direction calculation part which calculates the direction of the unmanned aircraft seen from the person wearing the head-mounted display based on the position of the unmanned aircraft and the external orientation element of the head-mounted display is provided. An optical information processing apparatus according to 1.   The external orientation element of the head-mounted display is calculated by a backward intersection method based on a plurality of points at which positions photographed by a camera fixed to the head-mounted display are specified. An optical information processing apparatus according to claim 1. A position measurement step of the unmanned aerial vehicle for measuring the position of the unmanned aircraft by measuring the distance to the UAV while optically tracking the unmanned aerial vehicle;
A shooting step of shooting the unmanned aerial vehicle;
Based on the position of the unmanned aircraft measured in the position measurement step, a flight state calculation step of calculating a parameter indicating the flight state of the unmanned aircraft;
An optical information processing method comprising: an image creating step of creating an image for displaying on a display a parameter indicating the flight state and an image of the unmanned aircraft taken by the camera. A program that is read and executed by a computer,
A position measuring unit for the unmanned aerial vehicle that measures the position of the unmanned aerial vehicle by measuring the distance to the UAV while optically tracking the unmanned aerial vehicle with a computer;
Based on the position of the unmanned aircraft measured by the position measurement unit, a flight state calculation unit that calculates a parameter indicating the flight state of the unmanned aircraft;
A program for optical information processing that functions as an image creation unit that creates an image for displaying a parameter indicating the flight state and a photographed image of the unmanned aircraft photographed by a camera on a display.

Images