JP2017163265A - Controlling support system, information processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system for supporting a safety flight of an unmanned flying object.SOLUTION: A flying object controlling support system 1000 includes: an unmanned flying object 20 on which an omnidirectional imaging device; an information processing device 10 that forms controlling supporting image; display means 12 of displaying the controlling supporting image; and visual field change means 302 of designating the change of a visual field of the controlling supporting image. The unmanned flying object includes: a photographing image transmission part 202 that transmits a photographing image photographed by the omnidirectional imaging device to the information processing device; and a sensor information transmission part 204 that transmits sensor information indicating a posture of an airframe at the photographing of the photographing image to the information processing device. The information processing device includes: a spherical image formation part 102 that forms a spherical image obtained by attaching an entire-celestial-sphere image to a virtual spherical surface on the basis of the photographing image received from the unmanned flying object; and a controlling supporting image formation part 108 in which an image region corresponding to an arbitrate visual field designated by the spherical image is cut out, and that forms the controlling supporting image on the basis of a plane-surface image formed by projecting and converting the image region.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a steering support system, an information processing apparatus, and a program.

  In recent years, public infrastructure such as tunnels and bridges constructed during the high growth period has been aging, and due to this, there have been catastrophic collapse accidents in various places. However, due to the recent shortage of labor and financial difficulties, the inspection work has not progressed as expected.

  In recent years, it has been studied to inspect public infrastructure using an unmanned air vehicle equipped with a camera. In this case, the inspector relies on the image transmitted from the unmanned air vehicle flying in a place where it cannot be directly seen, and simultaneously controls the unmanned air vehicle and checks the target location.

  Here, from the viewpoint of maneuverability and inspection efficiency of the aircraft, it is preferable that the driving mechanism that freely changes the shooting direction of the camera is mounted on the unmanned air vehicle, but on the other hand, the mounting of the driving mechanism is This increases power consumption and payload (loading weight), which is not preferable from the viewpoint of flight safety and longer time.

  On the other hand, a technique is known in which a video acquired by an omnidirectional camera is processed and displayed as a planar image in a range corresponding to an arbitrary gazing point designated by a user (for example, Patent Document 1).

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a flying object maneuvering support system for supporting safe flight of an unmanned flying object.

  As a result of earnest studies on the flying object maneuvering support system for supporting the safe flight of the unmanned aerial vehicle, the present inventor has arrived at the present invention and arrived at the present invention.

  That is, according to the present invention, an unmanned aerial vehicle equipped with an omnidirectional imaging device, an information processing device that generates a steering assistance image that supports the manipulation of the unmanned aerial vehicle, and a display unit that displays the steering assistance image And a field of view changing means for designating a change of the field of view of the steering assistance image, wherein the unmanned air vehicle provides the information processing device with a captured image taken by the omnidirectional imaging device. A captured image transmission unit for transmitting, and a sensor information transmission unit for transmitting sensor information indicating the attitude of the aircraft at the time of capturing the captured image to the information processing device, the information processing device from the unmanned aerial vehicle A spherical image generation unit that generates an omnidirectional image based on the received photographed image, pastes the omnidirectional image to a virtual spherical surface to generate a spherical image, and the visual field changing unit specifies from the spherical image Cut out image area corresponding to the field of view at will, steering assistance system that includes a steering assist image generation unit which generates the steering assist image based on the image area on the planar image obtained by projection transformation is provided.

  As described above, according to the present invention, a flying object maneuvering support system for supporting safe flight of an unmanned flying object is provided.

1 is a system configuration diagram of a flying object handling support system of an embodiment. The figure which shows the unmanned air vehicle and control apparatus of this embodiment. The functional block diagram of the flying object maneuvering assistance system of this embodiment. The figure for demonstrating the process which a spherical image production | generation part performs. The figure for demonstrating the process which a spherical image production | generation part performs. 6 is a flowchart of processing executed by the information processing apparatus according to the embodiment. 6 is a flowchart of processing executed by the information processing apparatus according to the embodiment. The figure for demonstrating the effect of this embodiment. 1 is a hardware configuration diagram of a flying object maneuvering support system of an embodiment.

  Hereinafter, although this invention is demonstrated with embodiment, this invention is not limited to embodiment mentioned later. In the drawings referred to below, the same reference numerals are used for common elements, and the description thereof is omitted as appropriate.

  FIG. 1 shows a system configuration of a flying object handling support system 1000 according to an embodiment of the present invention. As shown in FIG. 1, the flying object maneuvering support system 1000 according to this embodiment includes an unmanned aerial vehicle 20, a maneuvering device 30 for wirelessly maneuvering the unmanned aerial vehicle 20, an information processing device 10, and a display 12. It is configured to include.

  The unmanned air vehicle 20 of the present embodiment is an air vehicle that autonomously flies by radio control, and is preferably a multicopter. As shown in FIG. 1, the unmanned aerial vehicle 20 is equipped with an omnidirectional imaging device 22 that can shoot 360 degrees of moving images, and images captured by the omnidirectional imaging device 22 are stored in the information processing device 10. Real-time transmission by radio. Hereinafter, the omnidirectional imaging device 22 is referred to as an “omnidirectional camera 22”.

  The information processing apparatus 10 according to the present embodiment is a computer that executes predetermined image processing based on a captured image of the omnidirectional camera 22 transmitted in real time from the unmanned air vehicle 20, and supports remote control of the unmanned air vehicle 20. It plays a role of generating an image (hereinafter referred to as a steering assistance image) for real time in real time. The information processing apparatus 10 outputs the generated steering assistance image to the display unit 12, and the display unit 12 displays this. The connection method between the information processing apparatus 10 and the display 12 may be either wired or wireless. Hereinafter, the display means 12 is referred to as “display 12”.

  In addition to a known control function for remotely controlling the unmanned air vehicle 20 by radio, the control device 30 receives an operation signal for specifying a change in the field of view of the control support image displayed on the display 12 as an information processing device. 10 is provided. Note that the operation signal transmission method may be either wired or wireless.

  Here, detailed configurations of the unmanned air vehicle 20 and the control device 30 will be described with reference to FIG.

  FIG. 2A shows a front view and a side view of the unmanned air vehicle 20 together. In the present embodiment, the unmanned air vehicle 20 is equipped with a microcomputer (not shown), and the microcomputer performs autonomous flight control of the aircraft and data transmission to the information processing apparatus 10.

  The unmanned air vehicle 20 is equipped with various sensors for realizing stable flight. The microcomputer of the unmanned air vehicle 20 wirelessly transmits various sensor information to the information processing apparatus 10 in real time. The sensor here is, for example, a three-axis gyro sensor, a three-axis acceleration sensor, or a three-axis geomagnetic sensor, and the unmanned air vehicle 20 has at least a three-axis gyro sensor and three axes (x, y, z). Is obtained and transmitted as sensor information indicating the attitude of the aircraft.

  Further, as described above, the unmanned air vehicle 20 is equipped with the omnidirectional camera 22 that can shoot 360 degrees in all directions. The omnidirectional camera 22 in this embodiment has an imaging optical system including two fisheye lenses, and each fisheye lens has an angle of view exceeding 180 degrees. The omnidirectional camera 22 captures a range of more than 180 degrees ahead of the unmanned aerial vehicle 20 with one imaging optical system to obtain a first fisheye image, and the rear of the unmanned aircraft 20 with the other imaging optical system. The second fisheye image is obtained by photographing a range exceeding 180 degrees, and the microcomputer of the unmanned air vehicle 20 has two photographed images (the first images acquired by the omnidirectional camera 22). A fish-eye image and a second fish-eye image) are wirelessly transmitted to the information processing apparatus 10 in real time.

  FIG. 2B shows the control device 30. In addition to a known operation interface such as an operation lever 33 for remotely controlling the movement (front / back, left / right, up / down, and turning) of the unmanned air vehicle 20, the control device 30 displays an operation support image displayed on the display 12. A button group 32 including six buttons is provided as an interface for accepting an operation for designating a visual field. Here, the button group 32 includes two pan buttons (right and left) for accepting a pan operation for moving the visual field to the left and right, two tilt buttons (up and down) for accepting a tilt operation for moving the visual field up and down, It consists of a zoom-in button and a zoom-out button for accepting enlargement / reduction.

  In this system, the user checks the operation support image generated in real time on the display 12 based on the image captured by the omnidirectional camera 22 transmitted from the unmanned air vehicle 20, while operating the operation lever 33 of the control device 30. To remotely control the flight of the unmanned air vehicle 20. Here, in this system, the user moves the field of view up and down and left and right (pan, tilt), and enlarges / reduces the field of view (zoom in as necessary) via the button group 32 of the control device 30. By zooming out), not only the image corresponding to the field of view of the unmanned air vehicle 20 in the traveling direction but also the image corresponding to the field of view arbitrarily designated by the user can be viewed in real time. ing.

  Thus, the user can maneuver the unmanned aerial vehicle 20 while visually checking the omnidirectional situation that changes from moment to moment, and the unmanned aerial vehicle 20 can be safely operated even in a niche with many obstacles. It becomes possible to fly.

  However, the fact that the field of view of the unmanned air vehicle 20 other than the front of the unmanned air vehicle 20 can be freely seen (can be looked away) also causes the user to lose sight of the travel direction of the unmanned air vehicle 20. With this point, the present system is devised so that the user does not lose sight of the traveling direction of the unmanned air vehicle 20, and the details will be described later.

  As mentioned above, although the structure of each apparatus which comprises the flying object steering assistance system 1000 of this embodiment was demonstrated, it continues and the functional structure of each apparatus which comprises this system is demonstrated based on the functional block diagram shown in FIG. To do.

  The unmanned air vehicle 20 includes a captured image transmission unit 202 and a sensor information transmission unit 204.

  The captured image transmission unit 202 is a means for wirelessly transmitting the images (two fisheye images) captured by the omnidirectional camera 22 to the information processing apparatus 10 in real time.

  The sensor information transmission unit 204 is a means for wirelessly transmitting in real time to the information processing apparatus 10 as sensor information indicating the attitude of the airframe, the angular velocity of the three axes (x, y, z) that is at least the sensor output of the three-axis gyro sensor. is there.

  In the present embodiment, the unmanned aerial vehicle 20 functions as the above-described units when a microcomputer mounted on the unmanned air vehicle 20 executes a predetermined program.

  The control device 30 includes a visual field changing unit 302 and a flight control unit 304.

  The flight control means 304 is a known means for remotely controlling the movement (front / rear, left / right, up / down, turning) of the unmanned air vehicle 20.

  The field-of-view changing unit 302 is a unit that transmits an operation signal for designating the change of the field of view of the steering assistance image to the information processing apparatus 10 according to an operation via the button group 32.

  The information processing apparatus 10 includes a spherical image generating unit 102, a traveling direction specifying unit 104, a traveling direction drawing unit 106, and a steering assistance image generating unit 108.

  The spherical image generation unit 102 is a unit that generates an omnidirectional image based on the captured image received from the unmanned air vehicle 20, and generates a spherical image by pasting the omnidirectional image on a virtual spherical surface.

  The traveling direction specifying unit 104 obtains the current attitude of the unmanned aerial vehicle 20 based on the sensor information (angular velocity of three axes (x, y, z)) received from the unmanned aerial vehicle 20, and unmanned from the obtained current attitude. It is a means for specifying the current traveling direction of the flying object 20.

  The traveling direction drawing unit 106 is a unit that draws a predetermined mark at a position in the traveling direction in the spherical image generated by the spherical image generation unit 102.

  The steering support image generation unit 108 corresponds to an arbitrary field of view specified by an operation signal transmitted from the control device 30 (field of view changing unit 302) from a spherical image in which the traveling direction drawing unit 106 has drawn a predetermined mark. This is means for generating an operation support image based on a plane image formed by cutting out an image area to be cut and projecting the cut out image area.

  In the present embodiment, the information processing apparatus 10 functions as each unit described above when a processor mounted on the information processing apparatus 10 executes a predetermined program.

  The functional configuration of each device constituting this system has been described above. Next, the content of specific processing executed by each unit described above of the information processing device 10 will be described.

  First, processing executed by the spherical image generation unit 102 will be described.

In the present embodiment, the spherical image generation unit 102 generates a spherical image by the following procedures (1) to (3) based on the two fisheye images received from the unmanned air vehicle 20.
(1) Two fisheye images received from the unmanned air vehicle 20 are read out. FIG. 4A illustrates the two fisheye images.
(2) After distortion correction is performed on each fisheye image by mapping using a camera correction matrix corresponding to the characteristics of each camera lens, the two images after distortion correction are stitched (bonded). Then, top and bottom correction is performed on this to generate a single spherical image. FIG. 4B illustrates the generated omnidirectional image.
(3) Using a known texture mapping technique, a three-dimensional model in which the generated omnidirectional image is pasted on the inner surface of the virtual sphere is generated as a spherical image. FIG. 4C illustrates the generated spherical image.

  Next, processing executed by the steering assistance image generation unit 108 will be described.

  The steering assist image generation unit 108 cuts out an image region r corresponding to the field of view specified by the field of view changing unit 302 of the steering device 30 from the spherical image generated by the spherical image generation unit 102, and the cut image region r is extracted. The plane image R is generated by performing perspective projection transformation with the center of the virtual sphere as the projection center (origin). FIG. 5 conceptually shows a flow in which the image region r is cut out from the spherical image and is perspective-projected into the flat image R. In the present embodiment, among the perspective projection conversion parameters (line-of-sight direction, viewing angle, projection range (zNear, zFar), aspect ratio), the “line-of-sight direction” is the four buttons of the button group 32 of the control device 30 ( The field of view is specified by being changed by input via (pan and tilt).

  The processing executed by the spherical image generation unit 102 and the steering support image generation unit 108 has been described above. Subsequently, the processing shown in FIG. Based on

  First, in step 101, the direction and inclination (angular velocity) indicating the initial state of the cutout position of the image from the spherical image in FIG. 5 are set. Here, in the present embodiment, the steering assist image generation unit 108 holds quaternions (quaternions) having a real part and an imaginary part indicating the current image cut-out position from the spherical image and the inclination, respectively. In step 101, the four elements of the quaternion are set to zero.

  In the following step 102, the perspective value is set to the initial value.

  Thereafter, steps 103 to 113 described later are repeatedly executed.

  First, in step 103, it is determined whether or not the “pan” button or the “tilt” button is pressed. As a result, when no button is pressed (No at Step 103), the process proceeds to Step 107.

  On the other hand, when the “pan” button or the “tilt” button is pressed (step 103, Yes), in the subsequent step 104, the pressed button (pan (left), pan (right), tilt (up)). , Tilt (down)), the variable “angle” is updated. For example, if the pressed button is “pan (right)”, the time when the button is pressed is converted into an angular velocity, and the variable “angle” is updated so as to give the corresponding rotation to the right side. If steps 103 to 113 are repeated every 100 ms, under the rule that changes the π / 6 angular velocity per second when the button is pressed, the direction corresponding to the pressed button (left and right) is determined in step 104. Update the variable “angle” to give a rotation of π / 60 (= π / 6 × 0.1) to the top and bottom).

  In the subsequent step 105, a new quaternion is generated based on the updated angle. Here, the quaternion is uniquely obtained when the angular velocity and the x, y, and z axes are determined, and z is always 0 because the image is a plane.

  In subsequent step 106, the quaternion indicating the current image cutout position is updated by multiplying the quaternion indicating the current image cutout position by the quaternion generated in the previous step 105.

  In the following step 107, it is determined whether or not the “zoom in” button is pressed. As a result, when the “zoom-in” button is not pressed (No at Step 107), the process proceeds to Step 108.

  On the other hand, if the “zoom-in” button has been pressed (step 107, Yes), in the subsequent step 108, the perspective is added to the “zNear” side by an increment value.

  In the following step 109, it is determined whether or not the “zoom out” button is pressed. As a result, if the “zoom out” button has not been pressed (step 109, No), the process proceeds to step 111.

  On the other hand, when the “zoom-out” button is pressed (step 109, Yes), the subsequent step 110 adds the increment value to the “zFar” side.

  In the subsequent step 111, the advance direction specifying unit 104 and the advance direction drawing unit 106 cooperate to execute the “travel direction drawing process”. Here, the detailed content of the “traveling direction drawing process” will be described based on the flowchart shown in FIG.

  First, in step 201, the direction and inclination (angular velocity) indicating the initial posture state of the unmanned air vehicle 20 are set. Here, the maneuvering support image generation unit 108 holds quaternions (quaternions) in which the direction and inclination indicating the attitude state of the flying object are respectively real and imaginary parts. Set the element to 0.

  In the following step 202, the traveling direction specifying unit 104 acquires the latest angular velocity (three axes) from the sensor information received from the unmanned air vehicle 20, and in the subsequent step 203, the current direction of the aircraft is obtained by time angular integration. The attitude angle is obtained, and in the subsequent step 204, the quaternion indicating the current attitude state of the flying object is updated. Thereby, the latest traveling direction of the unmanned air vehicle 20 is specified.

  In the subsequent step 205, the traveling direction drawing unit 106 determines whether or not the current image includes the current traveling direction position of the unmanned air vehicle 20 indicated by the quaternion indicating the current attitude state of the unmanned air vehicle 20. To do. As a result, when the position of the unmanned air vehicle 20 in the current traveling direction is not included in the current image (No in Step 205), the “traveling direction drawing process” is ended.

  On the other hand, when the position indicated by the quaternion indicating the current posture state of the unmanned air vehicle 20 is included in the current image (step 205, Yes), the process proceeds to step 206. In the next step 206, the traveling direction drawing unit 106 draws an icon at the current traveling direction position of the unmanned air vehicle 20 in the image on the memory (frame buffer), and ends the “traveling direction drawing process”. Note that the icon drawn in step 206 may be any icon that can be recognized by the user as a mark indicating the traveling direction, and its specific mode is arbitrary.

  As described above, in the initial state, the lens front direction of the camera is the traveling direction of the unmanned air vehicle 20. Then, after the flight starts, the traveling direction of the unmanned air vehicle 20 at that time can be specified based on the latest attitude angle of the unmanned air vehicle 20. As a result, it is possible to grasp how much the image has moved in the x and y directions from the initial initial value coordinates, and it is possible to grasp the position of the image corresponding to the front of the image at that time by coordinates on the spherical image. As a result, the coordinates of the traveling direction of the unmanned air vehicle 20 in the spherical image whose orientation has been changed (pan / tilt) can be grasped, and when the image corresponding to the field of view designated by the user is cut out from the spherical image. It can be determined whether the image corresponding to the position in the traveling direction has been cut out. In accordance with this, the images are superimposed so that the traveling direction can be known.

  Returning again to FIG. 6, the description will be continued.

  After passing through the “traveling direction drawing process” in step 111, in the subsequent step 112, mapping is performed using the perspective and the current image cutout position, and the image on the memory (frame buffer) is updated. Thereafter, in step 113, the next image is taken and the process returns to step 103 again. Thereafter, the processes in steps 103 to 113 described above are repeated.

  Here, an effect obtained by repeating the above-described steps 103 to 113 will be described with reference to FIG.

  FIG. 8 shows a state where the user is operating the control device 30 while looking at the operation support image displayed on the display 12. In the state shown in FIG. 8A, a “◯” icon indicating the traveling direction of the unmanned air vehicle 20 is displayed in front of the operation support image. At this time, the user is displayed in front of the operation support image. The unmanned aerial vehicle 20 can be operated after recognizing that the image is an image ahead of the unmanned air vehicle 20 in the traveling direction.

  Subsequently, when a user who notices the target X reflected on the right side of the operation support image presses the “pan (right) button” of the control device 30 to observe the target X well, FIG. 8B shows. As described above, the image in which the target X is reflected is displayed in front of the operation support image, and the “◯” icon indicating the traveling direction of the unmanned air vehicle 20 moves to the left side of the operation support image. Thus, according to the present embodiment, the user can observe an arbitrary direction different from the traveling direction while confirming the traveling direction of the unmanned air vehicle 20.

  As described above, according to the present embodiment, the viewpoint of the video can be freely determined independently of the movement of the unmanned air vehicle without mounting the driving mechanism that causes the power consumption and the payload to be increased on the unmanned air vehicle. The system can be moved and zoomed, so when this system is applied to public infrastructure inspection work, flight safety and longer time and improved work efficiency are realized at the same time. .

  In addition, in the present embodiment, even when the user changes the field of view other than the front in the traveling direction, the traveling direction of the unmanned aerial vehicle is shown on the operation support image, so the user loses sight of the traveling direction of the unmanned aerial vehicle 20. Thus, the unmanned air vehicle 20 can safely fly.

  As mentioned above, although this invention was demonstrated with embodiment, this invention is not limited to embodiment mentioned above, A various design change is possible.

  For example, in the above-described embodiment, the control device 30 in which the visual field changing unit 302 and the flight control unit 304 are integrated has been described. However, the visual field changing unit 302 and the flight control unit 304 may be separate devices. The changing means 302 may be configured with a joystick or the like. Moreover, you may use as the control apparatus 30 what installed the application which has the function of the visual field change means 302 and the flight control means 304 in the tablet terminal or the smart phone.

  In the above-described embodiment, a flat panel display is exemplified as the display unit 12, but the display unit 12 may be a head mounted display.

  In the above-described embodiment, an omnidirectional imaging apparatus including two imaging optical systems is illustrated. However, an omnidirectional imaging apparatus including three or more imaging optical systems may be employed.

  Finally, the hardware configuration of the information processing apparatus 100 and the unmanned air vehicle 20 of the present embodiment will be described based on the hardware configuration diagram shown in FIG.

  As shown in FIG. 9A, the information processing apparatus 100 according to the present embodiment provides a processor 13 that controls the operation of the entire apparatus, a ROM 14 that stores a boot program, a firmware program, and the like, and a program execution space. The RAM 15, an auxiliary storage device 16 for storing a program or operating system (OS) for causing the information processing apparatus 100 to function as the above-described units, and an image output interface 17 for outputting an image to the display 12. A wireless communication interface 18 for establishing a wireless connection between the unmanned air vehicle 20 and the control device 30.

  On the other hand, as shown in FIG. 9B, the microcomputer mounted on the unmanned air vehicle 20 includes a processor 23 that controls the operation of the entire unmanned air vehicle 20, a ROM 24 that stores a boot program, a firmware program, and the like. A RAM 25 that provides an execution space for the program, an auxiliary storage device 26 for storing a program for causing the unmanned air vehicle 20 to function as the above-described means, and various sensors 21 (3-axis gyro sensor, 3-axis acceleration sensor) Wireless connection between the sensor interface 27 for connecting a three-axis geomagnetic sensor, the camera interface 28 for connecting the omnidirectional camera 22, and the information processing apparatus 100 and the control apparatus 30 is established. A wireless communication interface 29.

  Note that each function of the above-described embodiment can be realized by a device-executable program described in C, C ++, C #, Java (registered trademark), and the like. The program of this embodiment includes a hard disk device, a CD-ROM. , MO, DVD, flexible disk, EEPROM, EPROM and the like can be stored and distributed in a device-readable recording medium, and can be transmitted via a network in a format that other devices can.

  As described above, the present invention has been described with the embodiment. However, the present invention is not limited to the above-described embodiment, and the functions and effects of the present invention are within the scope of other embodiments that can be considered by those skilled in the art. As long as it plays, it is included in the scope of the present invention.

10. Information processing device 12. Display means (display)
13 ... Processor 14 ... ROM
15 ... RAM
16 ... Auxiliary storage device 17 ... Image output interface 18 ... Wireless communication interface 20 ... Unmanned air vehicle 21 ... Various sensors 22 ... Omnidirectional imaging device (omnidirectional camera)
23 ... Processor 24 ... ROM
25 ... RAM
26 ... Auxiliary storage device 27 ... Sensor interface 28 ... Camera interface 29 ... Wireless communication interface 30 ... Control device 32 ... Button group 33 ... Operation lever 100 ... Information processing device 102 ... Spherical image generation unit 104 ... Advancing direction specifying unit 106 ... Traveling direction drawing unit 108 ... Maneuvering support image generation unit 202 ... Captured image transmission unit 204 ... Sensor information transmission unit 302 ... Field of view changing means 304 ... Flight control means 1000 ... Aircraft maneuvering support system

JP 2005-56295 A

Claims (6)

An unmanned aerial vehicle equipped with an omnidirectional imaging device;
An information processing device for generating a maneuvering support image for maneuvering the unmanned air vehicle;
Display means for displaying the steering assist image;
Visual field changing means for designating a change in the visual field of the steering assistance image;
A steering support system including:
The unmanned air vehicle is
A captured image transmission unit that transmits a captured image captured by the omnidirectional imaging apparatus to the information processing apparatus;
A sensor information transmitting unit that transmits sensor information indicating the attitude of the aircraft at the time of capturing the captured image to the information processing apparatus;
Including
The information processing apparatus includes:
A spherical image generating unit that generates an omnidirectional image based on the captured image received from the unmanned air vehicle, and generates a spherical image by pasting the omnidirectional image on a virtual spherical surface;
A maneuvering support image generation unit configured to cut out an image region corresponding to an arbitrary field of view designated by the field of view changing unit from the spherical image, and to generate the maneuvering support image based on a planar image formed by projecting the image region; ,
including,
Maneuvering support system. The information processing apparatus includes:
A traveling direction specifying unit that obtains the attitude of the unmanned air vehicle based on the sensor information, and identifies the traveling direction of the unmanned air vehicle from the attitude;
A traveling direction drawing unit that draws a predetermined mark at a position of the traveling direction in the spherical image;
Including
The steering assist image generation unit
An image region corresponding to an arbitrary field of view designated by the field of view changing unit is cut out from the spherical image on which the predetermined mark is drawn, and the steering support image is converted based on a planar image formed by projecting the image region. Generate,
The steering assistance system according to claim 1. An information processing apparatus that generates a maneuvering support image that supports maneuvering of an unmanned air vehicle,
A spherical image generation unit that generates an omnidirectional image based on a captured image received from an unmanned aerial vehicle equipped with an omnidirectional imaging device, and generates a spherical image by pasting the omnidirectional image on a virtual spherical surface;
An operation support image generation unit that generates an image of the operation support image based on a plane image obtained by cutting out an image area corresponding to a specified arbitrary field of view from the spherical image and projecting the image area, and
including,
Information processing device. Based on sensor information indicating the attitude of the aircraft received from the unmanned aerial vehicle, the attitude of the unmanned aerial vehicle is obtained, and a traveling direction identifying unit that identifies the advancing direction of the unmanned aerial vehicle from the attitude;
A traveling direction drawing unit that draws a predetermined mark at a position of the traveling direction in the spherical image;
Including
The steering assist image generation unit
From the spherical image on which the predetermined mark is drawn, an image region corresponding to a specified arbitrary field of view is cut out, and the steering support image is generated based on a planar image formed by projecting the image region.
The information processing apparatus according to claim 3. Computer
A spherical image generating means for generating an omnidirectional image based on a photographed image received from an unmanned air vehicle equipped with an omnidirectional imaging device, and generating a spherical image by pasting the omnidirectional image on a virtual spherical surface;
An operation support image that generates an operation support image that supports operation of an unmanned air vehicle based on a plane image formed by cutting out an image area corresponding to a specified arbitrary field of view from the spherical image and projecting the image area. Generating means,
Program to function as. Said computer further
A traveling direction specifying means for obtaining the attitude of the unmanned aerial vehicle based on sensor information indicating the attitude of the airframe received from the unmanned aerial vehicle, and identifying the traveling direction of the unmanned aerial vehicle from the attitude;
A traveling direction drawing means for drawing a predetermined mark at a position of the traveling direction in the spherical image;
A program that functions as
The steering assist image generating means includes
From the spherical image on which the predetermined mark is drawn, an image region corresponding to a specified arbitrary field of view is cut out, and the steering support image is generated based on a planar image formed by projecting the image region.
The program according to claim 5.