JP7288947B1 - Flight control system, flight control method, and unmanned air vehicle - Google Patents

Abstract

An object of the present invention is to direct a camera to an unmanned flying object with a simple structure. A target angle acquisition unit (160) obtains a pitch axis ( Axa) and the axis (Ax2) along the yaw axis (Axb) of the first unmanned air vehicle. A second target angle (θ2), which is the angle in the direction of rotation about the center, is obtained. A drive control unit (170) controls a support structure for supporting the camera so that the camera faces in the direction of the first target angle in the rotation direction about the axis along the pitch axis of the first unmanned air vehicle. drive. A flight control unit (130) controls the first unmanned air vehicle so that the camera faces the direction of the second target angle in the direction of rotation about the axis along the yaw axis of the first unmanned air vehicle. swirl. [Selection drawing] Fig. 10

Description

The present disclosure relates to flight control systems, flight control methods, and unmanned air vehicles.

Patent Literature 1 below describes that an unmanned flying object that conducts practical work such as surveying the surface of a bridge is photographed by a camera of another unmanned airplane for monitoring, and the photographed image is transmitted to a control terminal and displayed on a monitor. ing.

Japanese Unexamined Patent Publication No. 2017-87916

Patent Document 1 does not describe how the camera is aimed at the unmanned airplane. As a method of controlling the orientation of the camera, for example, the direction along the pitch axis of the unmanned air vehicle (vertical direction) and the direction along the yaw axis ) to move the camera. However, the support structure that realizes the movement of the camera in two directions is generally complicated in structure, and adopting such a support structure may lead to the weight and cost of the unmanned aircraft. There is

An object of the present disclosure is to aim a camera at an unmanned air vehicle with a simple structure.

A flight control system according to the present disclosure is a flight control system including a first unmanned flying object and a second unmanned flying object that flies a predetermined flight route, wherein the first unmanned flying object flight control means for causing said first unmanned flying object to fly said predetermined flight route preceding or following said second unmanned flying object; and said second unmanned flying object from said first unmanned flying object position. and an axis line along the pitch axis of the first unmanned flying object with respect to the first unmanned flying object in a state in which the camera is attached to the first unmanned flying object and supports the camera. A support structure capable of moving the orientation of the camera in a direction of rotation about its center and an angle in the direction of rotation about an axis along the pitch axis based on a given camera direction that is the orientation of the camera. and a second target angle that is an angle in a rotational direction about an axis along the yaw axis of the first unmanned air vehicle; the pitch drive control means for driving the support structure to point the camera toward the first target angle in a rotational direction about an axis along the axis, wherein the flight control means is adapted to control the yaw Rotate the first unmanned air vehicle so that the camera points toward the second target angle in a rotational direction about the axis along the axis. According to this, it becomes possible to point the camera at the unmanned flying object with a simple structure.

Further, a flight control method according to the present disclosure is a flight control method for controlling a flight of a first unmanned air vehicle and a flight of a second unmanned air vehicle that flies along a predetermined flight route, of the unmanned flying object precedes or follows the second unmanned flying object and flies along the predetermined flight route to the first unmanned flying object, and uses a camera to move from the position of the first unmanned flying object photographing the second unmanned air vehicle, based on a given camera orientation that is the orientation of the camera, at an angle in a rotational direction about an axis along the pitch axis of the first unmanned air vehicle; obtaining a certain first target angle and a second target angle, which is an angle in the direction of rotation about the axis along the yaw axis of the first unmanned air vehicle; actuating a support structure attached to the first unmanned air vehicle and supporting the camera such that the camera is oriented toward the first target angle in a rotational direction about the center, and driving the yaw axis; in a rotational direction about an axis along which the camera turns to point in the direction of the second target angle. According to this, it becomes possible to point the camera at the unmanned flying object with a simple structure.

Further, an unmanned flying object according to the present disclosure is an unmanned flying object that flies on a predetermined flight route, and the unmanned flying object follows or follows another unmanned flying object that flies on the flight route. a flight control means for flying a flight route; a camera for photographing the other unmanned flying object from the position of the unmanned flying object; a support structure capable of moving the orientation of the camera in a rotational direction about an axis along the pitch axis of the unmanned air vehicle; A first target angle that is an angle in the direction of rotation about the axis along the axis, and a second target angle that is the angle in the direction of rotation about the axis along the yaw axis of the unmanned air vehicle. and drive control means for driving the support structure so that the camera faces the direction of the first target angle in a rotational direction about an axis along the pitch axis; wherein the flight control means turns the unmanned air vehicle so that the camera faces the direction of the second target angle in a rotational direction about an axis along the yaw axis. According to this, it becomes possible to point the camera at another unmanned flying object with a simple structure.

1 is a diagram showing the configuration of a flight control system that is an example of an embodiment of the present disclosure; FIG. 1 is a diagram showing an example of an unmanned air vehicle in flight; FIG. It is a figure which shows an example of the camera attached to an unmanned air vehicle. FIG. 4 is a diagram showing an example of flight control performed by the flight control system; FIG. 2 is a diagram showing an example of an image captured by a camera of an unmanned air vehicle; FIG. 10 is a diagram showing an example of a method for setting the center position of a determination region in an image; FIG. 2 is a diagram showing an example of an image captured by a camera of an unmanned air vehicle; An example of a method for calculating the position of an unmanned air vehicle from an image will be shown. An example of a method of calculating a target angle of a camera from an image is shown. 1 is a functional block diagram showing an example of functions of an unmanned air vehicle; FIG. FIG. 5 is a diagram showing an example of the flow of flight control processing; FIG. 7 is a diagram showing an example of the flow of camera direction control processing;

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[1. System configuration]
FIG. 1 is a diagram showing the configuration of a flight control system that is an example of an embodiment of the present disclosure. As shown in FIG. 1, the flight control system 1 includes a control device 10, an unmanned flying object 20A (first unmanned flying object), and an unmanned flying object 20B (second unmanned flying object). It is connected to network N. Note that the flight control system 1 may include an unmanned air vehicle (not shown) different from the two unmanned air vehicles 20A and 20B, and other devices such as user terminals connected to the network N. In the following description, each unmanned air vehicle 20A, 20B may be simply referred to as an unmanned air vehicle 20.

The control device 10 is a computer such as a server device that manages the flight of the unmanned air vehicle 20, and includes a processor 11, a storage unit 12, and a communication unit 13, as shown in FIG. The processor 11 is a program control device such as a CPU (Central Processing Unit) that operates according to a program installed in the control device 10, which is a computer, for example. The storage unit 12 is, for example, a storage element such as a ROM (Read Only Memory) or a RAM (Random Access Memory), a hard disk drive, or the like. Data such as a program executed by the processor 11 is stored in the storage unit 12 . The communication unit 13 is a communication interface such as a network board. The communication unit 13 may be connected to the network N via a repeater (not shown) such as a router. The network N is, for example, a wireless communication network for smart phones and mobile phones, and is implemented according to a predetermined communication standard (communication speed and frequency band standard) such as LTE (Long Term Evolution).

In the present embodiment, the storage unit 12 of the control device 10 stores data such as flight route information for the unmanned air vehicle 20 to fly. Also, the communication unit 13 of the control device 10 transmits and receives data to and from the unmanned flying object 20 via the network N. FIG. The communication unit 13 transmits, for example, information on the flight route along which the unmanned flying object 20 flies and instruction information on flight control (for example, information such as a stop instruction, a return instruction, and a landing instruction) to the unmanned flying object 20. .

[2. Configuration of unmanned air vehicle]
The unmanned flying object 20 is an aircraft that does not have a pilot or other person on board, and is, for example, a battery-driven aircraft (so-called drone). The unmanned air vehicle 20 may be an engine-driven aircraft or the like. Each of the two unmanned air vehicles 20A and 20B includes a processor 21, a storage unit 22, a communication unit 23, a positioning sensor 24, a camera 25a (first camera), and a camera 25b (second camera). Processor 21 is a program-controlled device such as a CPU. The storage unit 22 is, for example, a storage element such as ROM or RAM, or a hard disk drive. The storage unit 22 stores data such as programs to be executed by the processor 21 , and the processor 21 operates according to the programs stored in the storage unit 22 . The communication unit 23 is a communication interface such as an antenna, and is connected to the network N wirelessly.

In this embodiment, the communication unit 23 of the unmanned flying object 20 transmits telemetry information, which is information about the unmanned flying object 20 such as current position, speed, remaining battery power, etc., to the control device 10 . Further, the communication units 23 of the unmanned air vehicles 20A and 20B can exchange data with each other via the network N or other communication paths such as Wi-Fi (registered trademark), Bluetooth (registered trademark), and infrared rays. It is possible. For example, the communication unit 23 of the unmanned flying object 20A transmits telemetry information of the unmanned flying object 20A to the unmanned flying object 20B. Similarly, the communication unit 23 of the unmanned flying object 20B transmits telemetry information of the unmanned flying object 20B to the unmanned flying object 20A.

A positioning sensor 24 mounted on the unmanned flying object 20 is a sensor that detects the position (eg, latitude, longitude, and altitude) of the unmanned flying object 20 in the real world. The positioning sensor 24 of the unmanned flying object 20A detects the position of the unmanned flying object 20A, and the positioning sensor 24 of the unmanned flying object 20B detects the position of the unmanned flying object 20B. In addition, the positioning sensor 24 may include one or more sensors, for example, a GPS (Global Positioning System, Global Positioning Satellite) sensor that detects the latitude and longitude of the aircraft, and an altitude sensor that detects the altitude of the aircraft. may contain Also, the positioning sensor 24 may detect a relative position from the starting point, and may include a speed sensor or an acceleration sensor and a gyro sensor.

FIG. 2 is a diagram showing an example of an unmanned air vehicle 20 in flight. The unmanned air vehicle 20 includes the above-described processor 21 , storage unit 22 , housing 27 housing a battery and the like, motor 28 , and propeller 29 . The unmanned air vehicle 20 includes a plurality (four in the example shown in FIG. 2) of motors 28 and the same number of propellers 29 connected thereto. Each motor 28 and propeller 29 rotate in a direction centered on an axis along the yaw axis Axb of the unmanned air vehicle 20 . The processor 21 controls the rotation speed of each motor 28, that is, the rotation speed of each propeller 29, so that the unmanned air vehicle 20 can perform attitude control in the air, that is, flight control.

In this embodiment, the unmanned flying object 20 has a motor 28 and a propeller 29 that rotate clockwise and a motor 28 and a propeller 29 that rotate counterclockwise in plan view. In the unmanned air vehicle 20, the number of motors 28 rotating clockwise (the number of propellers 29) is the same as the number of motors 28 rotating counterclockwise (the number of propellers 29) (two in the example shown in FIG. 2). ). Further, in the example shown in FIG. 2, the motor 28 and the propeller 29 arranged with the housing 27 interposed therebetween rotate in the same direction. For example, the motor 28 and propeller 29 arranged on the left front and right rear of the housing 27 rotate clockwise, and the motor 28 and propeller 29 arranged on the right front and left rear of the housing 27 rotate counterclockwise. rotate to

The two cameras 25a and 25b attached to the unmanned flying object 20 take pictures in a predetermined direction (vertical direction, horizontal direction, or oblique directions thereof) from the position of the unmanned flying object 20. 20 on the outer surface. In the example shown in FIG. 2, cameras 25a and 25b are attached to the front and rear surfaces of a housing 27 of the unmanned air vehicle 20, respectively. The cameras 25 a and 25 b may be attached to the lower surface of the housing 27 without being limited to this.

The two cameras 25a and 25b face different directions. A camera 25 a attached to the front surface of a housing 27 of the unmanned flying object 20 faces forward of the unmanned flying object 20 , and a camera 25 b attached to the rear surface of the housing 27 faces rearward of the unmanned flying object 20 . ing. In the following description, the cameras 25a and 25b may be simply referred to as cameras 25 in some cases. The number of cameras 25 attached to the housing 27 of the unmanned flying object 20 may be one, or two or more. For example, four cameras 25 may be attached to the front, rear, right and left sides of the housing 27 .

Also, the camera 25 may be a stereo camera. In this case, the camera 25 takes a stereo image including distance information of the object to be shot in each pixel.

FIG. 3 is a diagram showing an example of the camera 25 and the support structure 26 attached to the unmanned air vehicle 20. As shown in FIG. As shown in FIG. 3, camera 25 includes lens 251 . The support structure 26 is attached to the unmanned air vehicle 20 to support the camera 25 and includes a base 261 attached to the housing 27 of the unmanned air vehicle 20 and an actuator 262 . Rotation of the actuator 262 about the axis Ax1 along the pitch axis Axa (see FIG. 2) of the unmanned air vehicle 20 causes the support structure 26 to rotate relative to the unmanned air vehicle 20 while supporting the camera 25. The direction of the camera 25 can be moved in the direction of rotation about the axis Ax1. In the following description, the support structure 26 attached to the front surface of the housing 27 and supporting the camera 25a is also referred to as the support structure 26a. Further, the support structure 26 that is attached to the rear surface of the housing 27 and supports the camera 25b is also referred to as a support structure 26b.

A damper may be provided between the base 261 and the actuator 262 in the support structure 26 . By doing so, it is possible to suppress the rocking of the camera 250 in response to the rocking of the unmanned flying object 20 . Further, the actuator 262 may move the camera 25 so that the camera 25 faces a predetermined direction with respect to the horizontal direction (direction parallel to the ground surface) in the direction centered on the axis Ax1. In other words, the support structure 26 may function as a single-axis gimbal. Also, the number of actuators included in the support structure 26 may be two or three. In this case, the axes (rotational centerlines) of the respective actuators may be perpendicular to each other, and the support structure 26 may function as a 2- or 3-axis gimbal.

[3. Overview of Flight Control]
[3-1. flight on the flight route]
FIG. 4 is a diagram showing an example of flight control performed by the flight control system. As shown in FIG. 4, in the flight control system 1, two unmanned air vehicles 20A and 20B fly a predetermined flight route 5. The flight route 5 is a continuous space having width and height connecting the departure points and arrival points of the unmanned air vehicles 20A and 20B. In the example shown in FIG. 4, the flight route 5 has a plurality of points from the departure point to the arrival point, and has a region of width ΔW and height ΔH centering on the points. The width and height of the area included in the flight route 5 may be the same or different at a plurality of points as shown in FIG.

In the flight control system 1, two unmanned air vehicles 20A and 20B fly within the flight route 5 with a predetermined distance in the traveling direction of the flight route 5. The unmanned flying object 20B flies on the flight route 5 ahead of the unmanned flying object 20A, and the unmanned flying object 20A flies on the flight route 5 while tracking the preceding unmanned flying object 20B. The unmanned flying object 20A flies within the flight route 5 while maintaining a predetermined distance from the unmanned flying object 20B so that the position of the unmanned flying object 20A does not deviate from the flight route 5.例文帳に追加Similarly, the unmanned flying object 20B flies within the flight route 5 while maintaining a predetermined distance from the unmanned flying object 20A so that the position of the unmanned flying object 20B does not deviate from the flight route 5. do. In the example shown in FIG. 4, a preceding unmanned flying object 20B is flying within a region of width ΔW and height ΔH centered at point O2, and unmanned flying object 20A tracking this is flying at a distance higher than point O2. It shows a state of flying in a region of width ΔW and height ΔH centered on point O1 in the rear (in the direction opposite to the traveling direction of flight route 5).

[3-2. Departure from Flight Route Detection]
In order to detect whether or not the unmanned air vehicles 20A and 20B have deviated from the flight route 5, the cameras 25 of the respective unmanned air vehicles 20A and 20B photograph predetermined directions during flight along the flight route 5. The camera 25 of the unmanned flying object 20A photographs the unmanned flying object 20B from the position of the unmanned flying object 20A flying on the flight route 5. - 特許庁Similarly, the camera 25 of the unmanned flying object 20B photographs the unmanned flying object 20A. In this embodiment, the camera 25 of the unmanned flying object 20A (the camera 25a attached to the front surface of the housing 27) basically captures the front of the unmanned flying object 20A (that is, the traveling direction of the flight route 5). . Also, the camera 25 of the preceding unmanned flying object 20B (the camera 25b attached to the rear surface of the housing 27) basically captures the rear of the unmanned flying object 20B.

5 and 7 are diagrams showing examples of images captured by the camera 25 of the unmanned flying object 20. FIG. 5 and 7 show an image 6 taken in front of the unmanned flying object 20A by the camera 25a of the unmanned flying object 20A. Image 6 shows an unmanned air vehicle 20B flying in flight route 5 . Based on this image 6, the unmanned flying object 20A detects whether the unmanned flying object 20B has deviated from the flight route 5 or not. Similarly, the unmanned flying object 20B detects whether the unmanned flying object 20A has deviated from the flight route 5 based on the image captured by the unmanned flying object 20B.

For example, the unmanned flying object 20A sets a determination region 61 in the image 6 for determining whether the unmanned flying object 20B has deviated from the flight route 5 or not. The determination area 61 is set as a virtual area that divides the inside and outside of the flight route 5 at the point O2 where the unmanned air vehicle 20B is flying. In this case, the unmanned flying object 20A determines whether the unmanned flying object 20B has deviated from the flight route 5 based on whether the unmanned flying object 20B appears within the determination area 61 in the image 6. For example, when the unmanned flying object 20B appears within the determination area 61, the unmanned flying object 20A determines that the unmanned flying object 20B does not deviate from the flight route 5. If the image 6 shows the unmanned flying object 20B or if the unmanned flying object 20B is not shown in the image 6, it is determined that the unmanned flying object 20B has deviated from the flight route 5.例文帳に追加

As the determination area 61, for example, an area centered on the position of the point O2 in the image 6 is set. The position of the point O2 in the image 6 can be set based on the direction in which the camera 25 of the unmanned air vehicle 20A is facing.

FIG. 6 is a diagram showing an example of a method of setting the position of the point O2, which is the center position of the determination area 61 in the image 6. As shown in FIG. As shown in FIG. 6, for example, the unmanned flying object 20A calculates the distance ΔL from the unmanned flying object 20A to the unmanned flying object 20B in the real world based on the resolution of the image 6 and the size of the unmanned flying object 20B shown in the image 6. can be calculated. In addition, the unmanned flying object 20A captures an image 6 with a camera 25, which is a stereo camera, and based on the distance information at the position of the unmanned flying object 20B shown in the image 6, which is a stereo image, the unmanned flying object in the real world. A distance ΔL from the body 20A to the unmanned air vehicle 20B may be calculated. By using the stereo camera in this way, it is possible to improve the calculation accuracy of the distance ΔL from the unmanned flying object 20A to the unmanned flying object 20B.

Next, based on the position (Xa, Ya, Za) of the unmanned flying object 20A in the real world obtained from the positioning sensor 24 of the unmanned flying object 20A and the calculated distance ΔL, the distance ΔL from the position of the unmanned flying object 20A is calculated. The position (X2, Y2, Z2) of the point O2 in the real world following the flight route 5 is calculated. Then, based on the position (Xa, Ya, Za) of the unmanned flying object 20A and the position (X2, Y2, Z2) of the point O2, the direction from the position of the unmanned flying object 20A to the point O2 is calculated, The direction in which the camera 25 of the unmanned air vehicle 20A is facing is specified. Next, the distance ΔD from the point Oa′ (ie, the center position of the image 6) to the point O2 is calculated, and the distance ΔD is calculated. Based on ΔD, the distance ΔL from the unmanned flying object 20A to the unmanned flying object 20B, and the resolution of the image 6, the offset amount Δd (see FIG. 5) from the center position Oa′ in the image 6 is calculated. Then, the unmanned flying object 20A sets the position of the point O2 at a position offset by the offset amount Δd from the center position Oa′ of the image 6, and sets the determination region 61 centering on the position of the point O2. The size of the determination area 61 in the image 6 is set based on the distance ΔL from the unmanned flying object 20A to the unmanned flying object 20B and the resolution of the image 6. FIG.

In addition, when the camera 25 is controlled to face the center of the flight route 5 (that is, when the direction of the camera 25 is controlled so that the point O2 appears in the center of the image 6), the unmanned flying object 20A may set a determination region 61 centered on the central position Oa′ of the image 6 .

Also, for example, the unmanned flying object 20A calculates the position of the unmanned flying object 20B in the real world based on the unmanned flying object 20B shown in the image 6. FIG. When the position of the unmanned flying object 20B deviates from the flight route 5, it is determined that the unmanned flying object 20B has deviated from the flight route 5. - 特許庁

FIG. 8 is a diagram showing an example of a method of calculating the position of the unmanned flying object 20B in the real world from the image 6. FIG. As shown in FIGS. 7 and 8, the unmanned flying object 20A is, for example, a reference direction in the real world (for example, the direction from the unmanned flying object 20A to the North Pole or the South Pole, or the traveling direction of the flight route 5). The direction in which the camera 25 of the flying object 20A is facing is identified, and the relative position (ΔXb , ΔYb, ΔZb) are calculated. Then, based on the calculated position (ΔXb, ΔYb, ΔZb) and the position (Xa, Ya, Za) of the unmanned flying object 20A in the real world acquired from the positioning sensor 24 of the unmanned flying object 20A, The position (Xb, Yb, Zb) of the unmanned air vehicle 20B is calculated. At least one of the latitude, longitude, and altitude of the position (Xb, Yb, Zb) of the unmanned flying object 20A thus calculated is included in the space set as the flight route 5. If not, it is determined that the unmanned air vehicle 20B has deviated from the flight route 5.

The communication unit 23 of the unmanned flying object 20A receives the position information measured by the positioning sensor 24 of the unmanned flying object 20B as the telemetry information of the unmanned flying object 20B. By determining whether or not the unmanned flying object 20B has deviated from the flight route 5 using the image 6 captured by the camera 25, the unmanned flight can be performed without receiving the position information of the unmanned flying object 20B from the unmanned flying object 20B. Departure of the body 20B from the flight route 5 can be detected. In other words, even if the unmanned flying object 20B becomes uncontrollable due to gusts or the like, or if the operation of devices such as the communication unit 23 and the positioning sensor 24 mounted on the unmanned flying object 20B becomes unstable, the unmanned flight can continue. It becomes possible to detect that the unmanned air vehicle 20B has deviated from the flight route 5 using the body 20A.

The unmanned flying object 20B also detects whether the unmanned flying object 20A has deviated from the flight route 5 using the image 6 captured by the camera 25 of the unmanned flying object 20B. By doing so, it is possible to detect that the unmanned flying object 20A has deviated from the flight route 5 using the unmanned flying object 20B without receiving the position information of the unmanned flying object 20A from the unmanned flying object 20A. Become.

When the unmanned flying object 20A detects that the unmanned flying object 20B has deviated from the flight route 5, the unmanned flying object 20A transmits information identifying a predetermined abnormality to the control device 10, and suspends the flight on the flight route 5. do. Here, the unmanned flying object 20A may transmit the position of the unmanned flying object 20B calculated using the image 6 to the control device 10. FIG. Further, when the unmanned flying object 20A detects that the unmanned flying object 20B has deviated from the flight route 5, the unmanned flying object 20A suspends the flight of the flight route 5 and hovers on the spot or lands at a predetermined place. or Also, the unmanned flying object 20A may fly off the flight route 5 by tracking the unmanned flying object 20B. In this case, the unmanned flying object 20A specifies the position of the unmanned flying object 20B that has interrupted the flight on the flight route 5 by deviating from the flight route 5, and after transmitting the position to the control device 10, Hover or land.

The unmanned flying object 20B also detects whether the unmanned flying object 20A has deviated from the flight route 5 using the image 6 captured by the camera 25 of the unmanned flying object 20B. By doing so, even when the unmanned flying object 20A becomes uncontrollable or the operation of equipment mounted on the unmanned flying object 20B becomes unstable, the unmanned flying object 20B can still be used for unmanned flight. Departure of the body 20A from the flight route 5 can be detected.

Even when the unmanned flying object 20B detects that the unmanned flying object 20A has deviated from the flight route 5, the unmanned flying object 20B transmits information identifying a predetermined abnormality to the control device 10, and resumes the flight on the flight route 5. interrupt. In this case, the unmanned flying object 20B hovers or lands on the spot. Further, the unmanned flying object 20B may hover or land on the spot after tracking the unmanned flying object 20A that has deviated from the flight route 5 and specifying the position of the unmanned flying object 20A.

[3-3. camera control]
The unmanned flying object 20 controls the direction in which the camera 25 faces from the unmanned flying object 20 (more specifically, the direction in which the lens 251 faces). In this embodiment, the camera 25 of the unmanned flying object 20A is controlled to face the unmanned flying object 20B. Similarly, the camera 25 of the unmanned flying object 20B is controlled to face the unmanned flying object 20A.

For example, as shown in FIG. 5, the unmanned flying object 20A sets a central area 62 of a predetermined size in the center of the image 6 captured by the camera 25 of the unmanned flying object 20A. Then, when the unmanned flying object 20B is displayed outside the central region 62, the direction in which the camera 25 faces is changed so that the unmanned flying object 20B is displayed within the central region 62.例文帳に追加By doing so, it is possible to prevent the unmanned flying object 20B from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20A (that is, the camera 25 of the unmanned flying object 20A losing sight of the unmanned flying object 20B). , the detection of whether the unmanned air vehicle 20B has deviated from the flight route 5 can be continued.

The unmanned flying object 20A is supported so that the camera 25 faces the direction in which the unmanned flying object 20B is positioned in the direction of rotation about the axis Ax1 along the pitch axis Axa (see FIG. 3) of the unmanned flying object 20A. The structure 26 (more specifically the actuator 262) is actuated. Further, the unmanned flying object 20A turns in the direction of rotation about the axis Ax2 along the yaw axis Axb of the unmanned flying object 20A so that the camera 25 faces the direction in which the unmanned flying object 20B is positioned. By doing so, it is possible to direct the camera 25 of the unmanned flying object 20A to the unmanned flying object 20B with a simple support structure 26 having one actuator 262 . In this embodiment, the yaw axis Axb of the unmanned air vehicle 20A and the axis Ax2 are aligned, but the axis Ax2 may be provided at a position different from the yaw axis Axb of the unmanned air vehicle 20A.

Unmanned air vehicle 20A rotates about axis Ax1 along pitch axis Axa of unmanned air vehicle 20A based on a given camera direction, which is the direction of camera 25 attached to unmanned air vehicle 20A. A first target angle θ1, which is an angle, and a second target angle θ2, which is an angle in the direction of rotation about the axis Axb along the yaw axis Axb of the unmanned air vehicle 20A, are acquired. Then, the unmanned flying object 20A drives the actuator 262 of the support structure 26 so that the camera 25 of the unmanned flying object 20A faces the direction of the first target angle θ1 in the direction of rotation about the axis Ax1, and In the direction of rotation about Ax2, the unmanned flying object 25A is turned so that the camera 25 of the unmanned flying object 20A faces the direction of the second target angle θ2.

Acquisition (calculation) of the first and second target angles θ1 and θ2 is performed based on the camera direction when the camera 25 of the unmanned flying object 20A faces the unmanned flying object 20B. The calculation of the first and second target angles θ1 and θ2 is performed, for example, based on the image 6 captured by the camera 25 of the unmanned flying object 20A.

FIG. 9 shows an example of a method of calculating the first and second target angles θ1 and θ2 from the image 6. FIG. As shown in FIG. 9, the unmanned flying object 20A identifies the camera direction when the camera 25 of the unmanned flying object 20A faces the unmanned flying object 20B based on the position of the unmanned flying object 20B in the image 6, and Based on the direction, the first and second target angles θ1 and θ2 are calculated. For example, the position of the unmanned flying object 20B in the image 6 and the resolution of the image 6 or the distance ΔL from the unmanned flying object 20A to the unmanned flying object 20B calculated based on the image 6, which is a stereo image including distance information, Based on this, the relative position (ΔWb, ΔZb) of the unmanned flying object 20B with respect to the position of the unmanned flying object 20A in the real world is calculated. Here, ΔWb indicates the distance in the horizontal direction of the camera 25 and ΔZb indicates the distance in the vertical direction of the camera 25 . Based on the distance ΔZb in the vertical direction of the camera 25, a first target angle θ1, which is the angle in the rotational direction around the axis Ax1 along the pitch axis Axa of the unmanned flying object 20A corresponding to this direction, is calculated. be able to. Also, based on the distance ΔWb in the horizontal direction of the camera 25, the second target angle θ2, which is the angle in the direction of rotation about the axis Ax2 along the yaw axis Axb of the unmanned flying object 20A corresponding to this direction, is determined. can be calculated.

Further, in this embodiment, one of the cameras 25a and 25b of the unmanned flying object 20A is controlled to face the unmanned flying object 20B. Similarly, one of the cameras 25a and 25b of the unmanned flying object 20B is controlled to face the unmanned flying object 20A. For example, when the unmanned flying object 20B is captured by the camera 25a of the unmanned flying object 20A (when the unmanned flying object 20A tracking the unmanned flying object 20B is facing forward on the flight route 5), the unmanned flight When the camera 25a of the body 20A is controlled to face the unmanned flying object 20B, and the camera 25b of the unmanned flying object 20A shows the unmanned flying object 20B (when the unmanned flying object 20A tracking the unmanned flying object 20B is flying) 5), the camera 25b of the unmanned flying object 20A is controlled to face the unmanned flying object 20B.

In this way, by controlling one of the two cameras 25a and 25b of the unmanned flying object 20A to face the unmanned flying object 20B, for example, when the unmanned flying object 20A faces the rear of the flight route 5 However, the unmanned flying object 20B can be photographed by the camera 25b attached to the rear surface of the housing 27 of the unmanned flying object 20A, and the camera 25 of the unmanned flying object 20A can be prevented from losing sight of the unmanned flying object 20B. Further, for example, by photographing the unmanned flying object 20B with the camera 25b attached to the rear surface of the housing 27 of the unmanned flying object 20A, the camera 25a attached to the front surface of the housing 27 of the unmanned flying object 20A can perform unmanned flight. The angle at which the unmanned flying object 20A turns to photograph the body 20B can be reduced, and the time for completing the turning of the unmanned flying object 20A can be shortened.

Also, when the unmanned flying object 20B is projected outside the central region 62, the unmanned flying object 20A may widen the angle of view of the camera 25 from the current angle of view. Even in this way, it is possible to prevent the unmanned flying object 20B from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20A.

Also, for the unmanned flying object 20B, the central region 62 is set to the image captured by the camera 25 of the unmanned flying object 20B. The direction in which the camera 25 faces is changed so that 20A can be seen. For example, if the unmanned flying object 20B, which flies ahead of the unmanned flying object 20A, is facing the front of the flight route 5, the camera 25b attached to the rear surface of the housing 27 of the unmanned flying object 20A will When the unmanned flying object 20B is directed to the rear of the flight route 5, the camera 25b attached to the front surface of the housing 27 is controlled to face the unmanned flying object 20A. be. This can prevent the unmanned flying object 20A from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20B (that is, the unmanned flying object 20B losing sight of the unmanned flying object 20A).

[4. Function block]
FIG. 10 is a functional block diagram showing an example of functions implemented in the flight control system 1. As shown in FIG. As shown in FIG. 3, the unmanned air vehicles 20A and 20B constituting the flight control system 1 functionally include a flight route information acquisition unit 110, a flight route information storage unit 120, a flight control unit 130, an image Including an acquisition unit 140, a central region setting unit 150, a target angle acquisition unit 160, a drive control unit 170, a camera direction identification unit 180, a determination unit 190, an abnormality transmission unit 200, and a flight interruption unit 210. . A flight route information acquisition unit 110, a flight control unit 130, an image acquisition unit 140, a central region setting unit 150, a target angle acquisition unit 160, a drive control unit 170, a camera direction identification unit 180, a determination unit 190, an abnormality transmission unit 200, and Flight interruption unit 210 is implemented mainly by processor 21 . The flight route information storage unit 120 is implemented mainly by the storage unit 22 . Note that the unmanned air vehicles 20A and 20B do not have to implement all the functions shown in FIG. 10, and may implement functions other than the functions shown in FIG.

[4-1. Flight route information acquisition unit, flight route information storage unit]
The flight route information acquisition unit 110 acquires information on a predetermined flight route. The flight route information storage unit 120 stores flight route information acquired by the flight route information acquisition unit 110 . In this embodiment, the unmanned air vehicles 20A and 20B fly the same flight route 5. FIG. Therefore, the flight route information acquiring unit 110 of the unmanned flying object 20A acquires the information of the flight route 5 along which the unmanned flying object 20B flies, and the flight route information acquiring unit 110 of the unmanned flying object 20B acquires Acquire information on flight route 5 to be flown.

[4-2. Flight control unit]
The flight control unit 130 controls the rotation speed of the motor 28 and the propeller 29 of the unmanned flying object 20 to cause the unmanned flying object 20 to fly along the flight route 5 indicated by the information acquired by the flight route information acquiring unit 110. . Here, the flight control unit 130 causes the unmanned flying object 20 to fly the flight route 5 based on the position of the unmanned flying object 20 measured by the positioning sensor 24 of the unmanned flying object 20 .

In this embodiment, the flight control unit 130 of the unmanned flying object 20B causes the unmanned flying object 20B to fly the flight route 5 ahead of the unmanned flying object 20A. Further, the flight control unit 130 of the unmanned flying object 20A tracks the unmanned flying object 20B preceding the flight route 5, and causes the unmanned flying object 20A to fly the flight route 5.

The flight control unit 130 of the unmanned flying object 20A, for example, maintains a predetermined distance between the unmanned flying object 20A and the unmanned flying object 20B so that the position of the unmanned flying object 20A does not deviate from the flight route 5. , causes the unmanned air vehicle 20A to fly a flight route 5. Here, the flight control unit 130 controls the position information of the unmanned flying object 20B received from the unmanned flying object 20B and the image 6 captured by the camera 25 of the unmanned flying object 20A (an image obtained by the image obtaining unit 140 described later). Based on the size of the unmanned flying object 20B, it is determined whether the unmanned flying object 20A is too close to the unmanned flying object 20B with respect to a predetermined distance, or whether the unmanned flying object 20A is too far away from the unmanned flying object 20B. , the movement speed of the unmanned flying object 20A on the flight route 5 is controlled based on the determination result. Similarly, the flight control unit 130 of the unmanned flying object 20B that precedes the flight route 5 also determines whether the unmanned flying object 20B is too close to or too far away from the unmanned flying object 20A. may control the movement speed of the unmanned air vehicle 20B.

In the present embodiment, the flight control unit 130 of the unmanned flying object 20A sets a second target angle θ2 (a target angle described later) that is an angle in the direction of rotation about the axis Ax2 along the yaw axis Axb of the unmanned flying object 20A. The unmanned flying object 20A is turned so that the camera 25 faces the direction of the angle obtained by the obtaining unit 160). By doing so, it is possible to prevent the unmanned flying object 20B from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20A (losing sight of the unmanned flying object 20B).

Similarly, the flight control unit 130 of the unmanned flying object 20B directs the camera 25 in the direction of the second target angle θ2, which is the rotational angle about the axis Ax2 along the yaw axis Axb of the unmanned flying object 20B. turn the unmanned air vehicle 20B so that the By doing so, it is possible to prevent the unmanned flying object 20A from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20B (losing sight of the unmanned flying object 20A).

In this embodiment, the unmanned air vehicle 20 is attached with two cameras 25a and 25b. For example, when the image acquisition unit 140 of the unmanned flying object 20A, which will be described later, selects the image 6 captured by the camera 25a (the unmanned flying object 20A faces forward on the flight route 5), the flight control unit 130 of the unmanned flying object 20A and the unmanned flying object 20B is captured in the image 6 captured by the camera 25a attached to the front surface of the housing 27 of the unmanned flying object 20A), the camera 25a is directed in the direction of the second target angle θ2. and the image acquisition unit 140 of the unmanned flying object 20A selects the image 6 taken by the camera 25b (the unmanned flying object 20A is facing the rear of the flight route 5 and the unmanned flying object When the unmanned flying object 20B is captured in the image 6 captured by the camera 25b attached to the rear surface of the housing 27 of the housing 27 of the unmanned flying object 20A, the unmanned flying object 20A is positioned so that the camera 25b faces the direction of the second target angle θ2. swirl.

As described above, the unmanned flying object 20 has the motor 28 and propeller 29 that rotate clockwise and the motor 28 and propeller 29 that rotate counterclockwise in plan view. For example, when the unmanned air vehicle 20 turns clockwise with respect to the axis Ax2, the flight control unit 130 increases the number of rotations of the motor 28 that rotates in the clockwise direction, and rotates in the clockwise direction. Decrease the rotation speed of the motor 28 . As a result, the unmanned flying object 20 gradually turns clockwise.

Further, the flight control unit 130 of the unmanned flying object 20A controls the unmanned flying object 20A when the unmanned flying object 20B does not appear in the central region 62 set in the image 6 captured by the camera 25 of the unmanned flying object 20A. The unmanned flying object 20A may be turned so that the camera 25 faces the unmanned flying object 20B (that is, the unmanned flying object 20B is shown in the central region 62). Similarly, the flight control unit 130 of the unmanned flying object 20B also detects when the camera 25 of the unmanned flying object 20B does not show the unmanned flying object 20A in the central region 62 of the image 6 captured by the camera 25 of the unmanned flying object 20B. The unmanned flying object 20B may be turned so as to face the flying object 20A.

[4-3. Image Acquisition Section, Central Area Setting Section]
The image acquisition unit 140 acquires the image 6 captured by the camera 25 of the unmanned air vehicle 20 . In this embodiment, the image acquisition unit 140 of the unmanned flying object 20A causes the camera 25 of the unmanned flying object 20A to capture images while both of the two unmanned flying objects 20A and 20B are flying along the flight route 5. continue to take image 6. Similarly, the image acquisition unit 140 of the unmanned flying object 20B also obtains images captured by the camera 25 of the unmanned flying object 20B while both of the two unmanned flying objects 20A and 20B are flying along the flight route 5. continue to take image 6.

In this embodiment, two cameras 25a and 25b are attached to each unmanned air vehicle 20. As shown in FIG. In this case, the image acquisition unit 140 of the unmanned flying object 20A selects the image 6 showing the unmanned flying object 20B from the images 6 captured by the cameras 25a and 25b. Further, the image acquisition unit 140 of the unmanned flying object 20B selects the image 6 showing the unmanned flying object 20A from the images 6 captured by the cameras 25a and 25b. Note that when a plurality of cameras 25 are attached to each unmanned flying object 20, the image acquisition unit 140 of the unmanned flying object 20A captures images at the positions of the unmanned flying object 20B in the images 6 captured by the cameras 25a and 25b, respectively. Based on this, image 6 may be selected. For example, the image acquisition unit 140 of the unmanned flying object 20A may select the image 6 in which the unmanned flying object 20B is positioned closest to the center Oa' of the image 6.

The central area setting unit 150 sets the central area 62 in the image 6 captured by the camera 25 of the unmanned air vehicle 20 . Central region setting unit 150 sets central region 62 so that the center of central region 62 is at the same position as image 6 . The central area setting section 150 may determine the size of the central area 62 based on the angle of view of the camera 25 and the resolution of the image 6 .

[4-4. Target angle acquisition unit]
The target angle acquisition unit 160 obtains the angle in the rotation direction about the axis Ax1 along the pitch axis Axa (see FIG. 3) of the unmanned air vehicle 20, based on the given camera direction, which is the direction of the camera 25. , and a second target angle θ2 that is the angle in the direction of rotation about the axis Ax2 along the yaw axis Axb of the unmanned air vehicle 20 . The target angle acquisition unit 160 of the unmanned flying object 20A acquires the first and second target angles θ1 and θ2 based on the camera direction when the camera 25 of the unmanned flying object 20A faces the unmanned flying object 20B. Similarly, the target angle acquisition unit 160 of the unmanned flying object 20B also obtains the first and second target angles θ1, Get θ2.

In this embodiment, the target angle acquisition unit 160 of the unmanned flying object 20A identifies the camera direction when the camera 25 of the unmanned flying object 20A faces the unmanned flying object 20B based on the image 6 captured by the camera 25 of the unmanned flying object 20A. By doing so, the first and second target angles θ1 and θ2 are obtained. Similarly, based on the image 6 captured by the camera 25 of the unmanned flying object 20B, the target angle acquisition unit 160 of the unmanned flying object 20B identifies the camera direction when the camera 25 faces the unmanned flying object 20A. By doing so, the first and second target angles θ1 and θ2 are obtained.

Based on the position of the unmanned flying object 20B in the image 6 captured by the camera 25 of the unmanned flying object 20A, the target angle acquisition unit 160 of the unmanned flying object 20A calculates the camera direction when the camera 25 faces the unmanned flying object 20B. , the first and second target angles θ1 and θ2 are obtained. The target angle acquisition unit 160 of the unmanned flying object 20A is calculated based on, for example, the position of the unmanned flying object 20B in the image 6 shown in FIG. Based on the distance ΔL from the unmanned flying object 20A to the unmanned flying object 20B, the relative position (ΔWb, ΔZb) of the unmanned flying object 20B with respect to the position of the unmanned flying object 20A in the real world is calculated. Then, based on the distance ΔZb in the vertical direction of the camera 25, the first target angle θ1, which is the angle in the direction of rotation about the axis Ax1 along the pitch axis Axa of the unmanned flying object 20A corresponding to this direction, is determined. Also, based on the distance ΔWb in the horizontal direction of the camera 25, the second target angle is the angle in the direction of rotation about the axis Ax2 along the yaw axis Axb of the unmanned flying object 20A corresponding to this direction. Calculate θ2.

In this embodiment, the unmanned air vehicle 20 is attached with two cameras 25a and 25b. In this case, the target angle acquisition section 160 calculates the first and second target angles θ1 and θ2 based on the image 6 captured by one of the two cameras 25a and 25b. For example, when the image acquisition unit 140 of the unmanned flying object 20A selects the image 6 (first image) captured by the camera 25a (the unmanned flying object 20A is the flight route 5 and the unmanned flying object 20B appears in the image 6 captured by the camera 25a attached to the front surface of the housing 27 of the unmanned flying object 20A), the unmanned flying object in the image 6 captured by the camera 25a Based on the position of the flying object 20B, the camera direction when the camera 25a faces the unmanned flying object 20B is specified to obtain the first and second target angles θ1 and θ2, and the image of the unmanned flying object 20A is obtained. When the acquisition unit 140 selects the image 6 (second image) captured by the camera 25b (the unmanned flying object 20A faces the rear of the flight route 5 and is attached to the rear surface of the housing 27 of the unmanned flying object 20A). When the unmanned flying object 20B appears in the image 6 shot by the camera 25b captured by the camera 25b), the camera 25b faces the unmanned flying object 20B based on the position of the unmanned flying object 20B in the image 6 shot by the camera 25b. First and second target angles θ1 and θ2 are obtained by specifying the camera directions of .

Note that the target angle acquisition unit 160 of the unmanned flying object 20A is based on each position information of the unmanned flying objects 20A and 20B measured by the positioning sensor 24 and the direction of the camera 25 specified by the camera direction specifying unit 180 described later. , the first and second target angles θ1 and θ2 may be obtained. Further, the target angle acquisition unit 160 of the unmanned flying object 20A acquires the first angle based on the position information measured by the positioning sensor 24 of the unmanned flying object 20A and the direction of the camera 25 specified by the camera direction specifying unit 180, which will be described later. and second target angles θ1 and θ2 may be obtained.

[4-5. drive control unit]

The drive control unit 170 of the unmanned flying object 20A rotates the unmanned flying object 20A in the direction of the first target angle θ1 acquired by the target angle acquiring unit 160 in the direction of rotation about the axis Ax1 along the pitch axis Axa of the unmanned flying object 20A. The support structure 26 (more specifically, the actuator 262 of the support structure 26) is driven so that the camera 25 of the aircraft 20A is directed. By doing so, it is possible to prevent the unmanned flying object 20B from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20A (losing sight of the unmanned flying object 20B).

Similarly, the drive control unit 170 of the unmanned flying object 20B detects the first target angle acquired by the target angle acquiring unit 160 in the rotation direction about the axis Ax1 along the pitch axis Axa of the unmanned flying object 20B. The actuator 262 of the support structure 26 is driven so that the camera 25 of the unmanned flying object 20B faces in the direction of θ1. By doing so, it is possible to prevent the unmanned flying object 20A from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20B (losing sight of the unmanned flying object 20A).

In this embodiment, the unmanned air vehicle 20 is attached with two cameras 25a and 25b. For example, when the image acquisition unit 140 of the unmanned flying object 20A selects the image 6 captured by the camera 25a (the image attached to the front surface of the housing 27 of the unmanned flying object 20A), the drive control unit 170 of the unmanned flying object 20A When the unmanned flying object 20B appears in the image 6 captured by the camera 25a), the actuator 262 of the support structure 26a is driven so that the camera 25a faces the direction of the first target angle θ1, and the unmanned flying object 20A When the image acquisition unit 140 selects the image 6 captured by the camera 25b (when the unmanned flying object 20B appears in the image 6 captured by the camera 25b attached to the rear surface of the housing 27 of the unmanned flying object 20A) , the actuator 262 of the support structure 26b is driven so that the camera 25b faces in the direction of the first target angle θ1.

For example, when the position of the unmanned flying object 20B in the image 6 captured by the camera 25 of the unmanned flying object 20A is outside the central region 62, the drive control unit 170 of the unmanned flying object 20A determines that the camera 25 is the unmanned flying object. Support structure 26 may be driven to face 20B. For example, in the image 6, when the unmanned flying object 20B is shown under the central region 62, the drive control unit 170 of the unmanned flying object 20A tilts the camera 25 downward to place the unmanned flying object in the center of the image 6. The support structure 26 may be actuated to reflect the body 20B.

Further, when the position of the unmanned flying object 20B in the image 6 is outside the central region 62, the drive control unit 170 of the unmanned flying object 20A changes the angle of view of the camera 25 to a wider angle of view than the current angle of view. may By doing so, it is also possible to prevent the unmanned flying object 20B from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20A.

Note that when the position of the unmanned flying object 20B is inside the central region 62 in the image 6, the drive control unit 170 of the unmanned flying object 20A adjusts the angle of view of the camera 25 of the unmanned flying object 20A from the current angle of view. may be changed to a narrower angle of view. By doing so, when specifying the position of the unmanned flying object 20B from the image 6, it is possible to suppress the detection of another object appearing in the image 6 as the unmanned flying object 20B.

The drive control unit 170 of the unmanned flying object 20B also directs the camera 25 toward the unmanned flying object 20A when the unmanned flying object 20A is outside the central region 62 in the image 6 captured by the camera 25 of the unmanned flying object 20B. Alternatively, the angle of view of the camera 25 may be changed to be wider than the current angle of view. This can prevent the unmanned flying object 20A from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20B (losing sight of the unmanned flying object 20A).

[4-6. Camera direction identification unit]
The camera direction identification unit 180 identifies the direction in which the camera 25 of the unmanned air vehicle 20 is facing. The camera direction identifying unit 180 of the unmanned flying object 20A identifies the direction in which the camera 25 of the unmanned flying object 20A is facing, and the camera direction identifying unit 180 of the unmanned flying object 20B identifies the direction in which the camera 25 of the unmanned flying object 20B faces. Identify the direction you are facing. "Specifying the direction in which the camera 25 is facing" means, for example, the angle formed between the reference direction and the direction in which the camera 25 is facing (the angle in the latitude direction, the angle in the longitude direction, and the angle in the height direction). angle). Also, "identifying the direction in which the camera 25 is facing" may be identifying the position in the real world to which the camera 25 is facing.

The camera direction specifying unit 180 may specify, for example, the direction in which the camera 25 is facing with respect to the reference direction, which is the direction along the traveling direction of the flight route 5 at the position of the unmanned air vehicle 20 . For example, the camera direction specifying unit 180 of the unmanned flying object 20A uses the direction from the position of the unmanned flying object 20A to the point O2 (see FIGS. 4 to 6), which is the center of the area defining the flight route 5, as the reference direction. Alternatively, the direction from the unmanned air vehicle 20A to the North Pole or the South Pole may be used as the reference direction. Similarly, the camera direction specifying unit 180 of the unmanned flying object 20B may set the direction from the position of the unmanned flying object 20B to the point O1, which is the center of the area defining the flight route 5, as a reference direction. A direction from 20B toward the North Pole or the South Pole may be used as the reference direction.

[4-7. Judgment part]
The determination unit 190 of the unmanned flying object 20A determines whether the unmanned flying object 20B has deviated from the flight route 5 based on the image 6 captured by the camera 25 of the unmanned flying object 20A. More specifically, the determination unit 190 of the unmanned flying object 20A determines the position of the unmanned flying object 20A measured by the positioning sensor 24 of the unmanned flying object 20A, the direction in which the camera 25 of the unmanned flying object 20A is facing, and , based on the image 6 captured by the camera 25, it is determined whether or not the unmanned air vehicle 20B has deviated from the flight route 5. Similarly, the determination unit 190 of the unmanned flying object 20B determines whether the unmanned flying object 20A has deviated from the flight route 5 based on the image 6 captured by the camera 25 of the unmanned flying object 20B.

For example, as shown in FIGS. 5 and 6, the determining unit 190 of the unmanned flying object 20A determines the traveling direction of the flight route 5 at the position of the unmanned flying object 20A (for example, the direction from the unmanned flying object 20A to the point O2). ), a determination region 61 within the flight route 5 is set in the image 6 captured by the camera 25 based on the direction in which the camera 25 of the unmanned air vehicle 20A is facing. Then, when it is determined that the unmanned flying object 20B does not appear within the set determination area 61, it is determined that the unmanned flying object 20B has deviated from the flight route 5.例文帳に追加Similarly, the determination unit 190 of the unmanned flying object 20B determines whether the unmanned flying object 20B is unmanned in the direction opposite to the traveling direction of the flight route 5 (for example, the direction from the unmanned flying object 20B to the point O1). A case in which a judgment area 61 is set in the image 6 captured by the camera 25 based on the direction in which the camera 25 of the flying object 20B is facing, and it is judged that the unmanned flying object 20A is not captured in the judgment area 61. Then, it may be determined that the unmanned air vehicle 20A has deviated from the flight route 5.

7 and 8, the determining unit 190 of the unmanned flying object 20A determines the position of the unmanned flying object 20A, the direction in which the camera 25 of the unmanned flying object 20A is directed, and the Based on the position of the unmanned flying object 20B in the image 6 captured by 25, the position of the unmanned flying object 20B in the real world is calculated. , it may be determined that the unmanned air vehicle 20B has deviated from the flight route 5. The determining unit 190 of the unmanned flying object 20A calculates the relative position of the unmanned flying object 20B with respect to the position of the unmanned flying object 20A. The position of the unmanned flying object 20B in the real world may be calculated based on the position of the unmanned flying object 20A. Similarly, the determination unit 190 of the unmanned flying object 20B determines the position of the unmanned flying object 20B, the direction in which the camera 25 of the unmanned flying object 20B is facing, and the unmanned flying object in the image 6 captured by the camera 25. The position of the unmanned flying object 20A in the real world is calculated based on the position of the unmanned flying object 20A. It may be judged that it has come off.

In this way, the determining unit 190 of the unmanned flying object 20A determines whether or not the unmanned flying object 20B has deviated from the flight route 5 using the image 6 captured by the camera 25 of the unmanned flying object 20A. Departure of the unmanned air vehicle 20B from the flight route 5 can be detected without the administrator checking the image taken by the camera 25.例文帳に追加Similarly, the determination unit 190 of the unmanned flying object 20B uses the image 6 captured by the camera 25 of the unmanned flying object 20B to determine whether the unmanned flying object 20A has deviated from the flight route 5. It becomes possible to detect that the unmanned air vehicle 20A has deviated from the flight route 5 without depending on the administrator.

[4-8. Abnormal transmission unit, flight interruption unit]
The abnormality transmission unit 200 transmits identification information for identifying a predetermined abnormality (predetermined error code, wording indicating the abnormality, image, etc.) to the control device 10 when an abnormality occurs in the unmanned air vehicle 20 . In this embodiment, when the determining unit 190 determines that the unmanned flying object 20B has left the flight route 5, the abnormality transmitting unit 200 of the unmanned flying object 20A detects that the unmanned flying object 20B has left the flight route 5. Identification information indicating that is transmitted to the control device 10 . Similarly, when it is determined that the unmanned flying object 20A has deviated from the flight route 5, the abnormality transmitting unit 200 of the unmanned flying object 20B has an identification signal indicating that the unmanned flying object 20A has deviated from the flight route 5. Information is transmitted to the control device 10 .

For example, the abnormality transmission unit 200 of the unmanned flying object 20A sends the position of the unmanned flying object 20A measured by the positioning sensor 24 of the unmanned flying object 20A and the position of the unmanned flying object 20B received from the unmanned flying object 20B to the control device 10. You may send. When the unmanned flying object 20A calculates the position of the unmanned flying object 20B based on the image 6, the calculated position of the unmanned flying object 20B is displayed together with the identification information indicating that the unmanned flying object 20B has deviated from the flight route 5. You may transmit to the control apparatus 10. FIG. Similarly, even if the abnormality transmission unit 200 of the unmanned flying object 20B also transmits the identification information indicating that the unmanned flying object 20A has deviated from the flight route 5 and the position of the unmanned flying object 20A to the control device 10. good.

The flight suspension unit 210 suspends the flight on the flight route 5 by the unmanned flying object 20 when an abnormality occurs in the unmanned flying object 20 . In this embodiment, the flight suspension unit 210 of the unmanned flying object 20A causes the unmanned flying object 20A to suspend the flight on the flight route 5 when it is determined that the unmanned flying object 20B has deviated from the flight route 5. Similarly, the flight suspension unit 210 of the unmanned flying object 20B causes the unmanned flying object 20B to suspend the flight on the flight route 5 when it is determined that the unmanned flying object 20A has deviated from the flight route 5.

Interrupting the flight on the flight route 5 by the unmanned flying object 20 may be, for example, causing the unmanned flying object 20 to remain at its current position (hovering) or landing at a predetermined point. may be For example, when the unmanned flying object 20B is not captured in the image 6 captured by the camera 25 of the unmanned flying object 20A (the unmanned flying object 20B is lost), the flight interruption unit 210 of the unmanned flying object 20A hoveres over the unmanned flying object 20A. In addition, changing the direction of the unmanned flying object 20A and changing the direction of the camera 25 of the unmanned flying object 20A are repeated until the unmanned flying object 20B is reflected in the image 6 captured by the camera 25 (that is, in situ search for the unmanned air vehicle 20B).

Further, when the unmanned flying object 20B deviates from the flight route 5, the flight interruption unit 210 of the unmanned flying object 20A causes the unmanned flying object 20A to track the unmanned flying object 20B within a predetermined flight range. The aircraft 20A may be caused to fly off the flight route 5. Further, the abnormality transmission unit 200 of the unmanned flying object 20A may transmit the position of the unmanned flying object 20B to the control device 10. By doing so, the position of the unmanned flying object 20B after leaving the flight route 5 can be detected using the unmanned flying object 20A. Thereafter, when the administrator retrieves the landed unmanned flying object 20B, the search for the unmanned flying object 20B can be smoothly performed.

[5. flowchart]
FIG. 11 is a diagram showing an example of the flow of flight control processing performed by the flight control system 1. As shown in FIG. The processing executed by the processor 21 of the unmanned flying object 20A when the unmanned flying object 20A tracks the unmanned flying object 20B and flies along the flight route 5 will be described below. The processor 21 of the flying object 20B can also perform similar processing.

When the unmanned flying object 20A is flying along the flight route 5, the processor 11 of the unmanned flying object 20A executes the camera direction control process (step S101), and the positioning sensor 24 of the unmanned flying object 20A ( self) is measured (step S102). Based on this position, the flight control unit 130 of the unmanned flying object 20A causes the unmanned flying object 20A to fly along the flight route 5 indicated by the information acquired by the flight route information acquiring unit 110.

FIG. 12 is a diagram showing an example of the flow of camera direction control processing executed in step S101. As shown in FIG. 12, the image acquisition unit 140 of the unmanned flying object 20A selects the image 6 showing the unmanned flying object 20B from among the images 6 captured by the two cameras 25a and 25b of the unmanned flying object 20A. (step S201). Then, the central region setting unit 150 of the unmanned flying object 20A sets a central region 62 (see FIG. 5) having a center at the same position as the center Oa' of the image 6 selected in step S201 (see FIG. 5). step S202).

Next, the processor 11 of the unmanned flying object 20A determines whether the position of the unmanned flying object 20B in the image 6 selected at step S201 is within the central region 62 set at step S202 (step S203). If the position of the unmanned flying object 20B is not within the central region 62 (No in step S203), the processor 11 of the unmanned flying object 20A sequentially executes the processes of steps S205 to S207.

In step S204, the target angle acquisition unit 160 of the unmanned flying object 20A obtains first and second target angles based on the direction in which the camera 25 that captured the image 6 selected in step S201 faces the unmanned flying object 20B. Get θ1 and θ2. For example, based on the image 6 selected in step S201, the target angle acquisition unit 160 of the unmanned flying object 20A identifies the direction in which the camera 25 that captured the image 6 faces the unmanned flying object 20B. and second target angles θ1 and θ2. The target angle acquisition unit 160 of the unmanned flying object 20A calculates the relative position of the unmanned flying object 20B with respect to the position of the unmanned flying object 20A in the real world, based on the position of the unmanned flying object 20B in the image 6 shown in FIG. 9, for example. (ΔWb, ΔZb) are calculated. Then, based on the distance ΔZb in the vertical direction of the camera 25, a first target angle θ1, which is the angle in the rotational direction corresponding to this direction, is calculated, and based on the distance ΔWb in the horizontal direction of the camera 25, this target angle θ1 is calculated. A second target angle θ2, which is an angle in the rotation direction corresponding to the direction, is calculated.

In step S205, the drive control unit 170 of the unmanned flying object 20A directs the camera 25 that captured the image 6 selected in step S201 in the direction of the first target angle θ1 acquired (calculated) in step S204. Then, the support structure 26 that supports the camera 25 (more specifically, the actuator 262 of the support structure 26) is driven. For example, of the two cameras 25a and 25b attached to the unmanned flying object 20A, when the image 6 including the unmanned flying object 20B is captured by the camera 25a, the drive control unit 170 of the unmanned flying object 20A is oriented in the direction of the first target angle θ1. The support structure 26b that supports the camera 25b is driven so as to face in the direction of the target angle .theta.1.

In step S206, the flight control unit 130 of the unmanned flying object 20A directs the camera 25 that captured the image 6 selected in step S201 in the direction of the second target angle θ2 acquired (calculated) in step S204. Then, the unmanned air vehicle 20A (self) is turned. For example, of the two cameras 25a and 25b attached to the unmanned flying object 20A, when the image 6 including the unmanned flying object 20B is captured by the camera 25a, the flight control unit 130 of the unmanned flying object 20A is directed in the direction of the second target angle θ2, and the image 6 including the unmanned flying object 20B is captured by the camera 25b. The unmanned flying object 20A is turned so as to face the direction.

Note that the process of step S205 and the process of step S206 may be started at the same time. Also, the order in which the process of step S205 and the process of step S206 are executed may be reversed.

Returning to FIG. 11, in step S103, the camera direction specifying unit 180 of the unmanned flying object 20A determines the direction from the position of the unmanned flying object 20A toward the traveling direction of the flight route 5 (for example, from the unmanned flying object 20A shown in FIG. 4). The direction toward the point O2), the direction toward the North Pole or the South Pole from the position of the unmanned air vehicle 20A, and the like are used as reference directions, and the direction in which the camera 25 is facing is specified with respect to this reference direction. Then, in step S104, the determination unit 190 of the unmanned flying object 20A determines the position of the unmanned flying object 20A measured in step S101, the direction in which the camera 25 is directed identified in step S105, and the image 6 acquired in step S102. , it is determined whether or not the unmanned air vehicle 20B has deviated from the flight route 5.

In step S104, for example, as shown in FIGS. 5 and 6, step A determination area 61, which is an area within the flight route 5, may be set in the image 6 acquired in S102. Then, whether or not the unmanned flying object 20B has deviated from the flight route 5 may be determined based on whether the unmanned flying object 20B appears within the set determination area 61 or not.

Further, in step S104, for example, as shown in FIGS. 7 and 8, the position of the unmanned flying object 20A, the direction in which the camera 25 of the unmanned flying object 20A is facing, and the unmanned object in the image 6 captured by the camera 25 are determined. Based on the position of the flying object 20B, the position of the unmanned flying object 20B in the real world is calculated. It may be determined that route 5 has been deviated.

When it is determined in step S104 that the unmanned flying object 20B has not deviated from the flight route 5 (No in step S105), the processor 21 of the unmanned flying object 20A repeats the processing from steps S101 to S104. That is, the processor 21 of the unmanned flying object 20A keeps the direction of the camera 25 of the unmanned flying object 20A until the unmanned flying object 20B arrives at the arrival point, which is the destination, or until the unmanned flying object 20A detects a predetermined abnormality. The control camera direction control process and the determination process (monitoring process) of whether or not the unmanned air vehicle 20B has deviated from the flight route 5 are continued.

If it is determined in step S104 that the unmanned flying object 20B has deviated from the flight route 5 (Yes in step S105), the abnormality transmitting unit 200 of the unmanned flying object 20A transmits an abnormality that the unmanned flying object 20B has deviated from the flight route 5. The identification information indicated is transmitted to the control device 10, and the flight suspension unit 210 of the unmanned air vehicle 20A causes the unmanned air vehicle 20A to suspend the flight on the flight route 5 (step S106).

In step S106, the abnormality transmitting unit 200 of the unmanned flying object 20A sends identification information indicating that the unmanned flying object 20B has deviated from the flight route 5, as well as the position of the unmanned flying object 20A measured in step S101 and the unmanned flying object 20B. The position of the unmanned air vehicle 20B received from may be transmitted to the control device 10. Further, when the unmanned flying object 20A calculates the position of the unmanned flying object 20B in step S104, the calculated position of the unmanned flying object 20B may be transmitted to the control device 10 in step S106.

Further, in step S106, the flight interruption unit 210 of the unmanned flying object 20A may cause the unmanned flying object 20 to hover at the current position or land at a predetermined point. The flight interruption unit 210 of the unmanned flying object 20A determines the landing point of the unmanned flying object 20A (the departure point, a point near the flight route 5, etc.) based on, for example, the remaining amount of the battery of the unmanned flying object 20A. may Further, in step S106, the flight interruption unit 210 of the unmanned flying object 20A may cause the unmanned flying object 20A to track the unmanned flying object 20B that has deviated from the flight route 5 within a predetermined flight range. Here, too, the abnormality transmission unit 200 of the unmanned flying object 20A may transmit the position of the unmanned flying object 20B specified from the image 6 to the control device 10. By doing so, the position of the unmanned flying object 20B after deviating from the flight route 5 can be detected using the unmanned flying object 20A, and the unmanned flying object 20B can be recovered smoothly thereafter. can.

[6. summary]
As described above, in this embodiment, the unmanned flying object 20A flies along the flight route 5 while tracking the unmanned flying object 20B. It is determined whether or not the body 20B has deviated from the flight route 5. According to this, it is possible to detect that the unmanned flying object 20B has deviated from the flight route without the administrator having to look at the image (video) of the camera 25 .

Further, the unmanned flying object 20A is arranged so that the camera 25 faces the unmanned flying object 20 when the position of the unmanned flying object 20B in the image 6 captured by the camera 25 of the unmanned flying object 20A is outside the central region 62. Additionally, the orientation of the unmanned flying object 20A and the orientation of the camera 25 may be controlled. According to this, it is possible to prevent the unmanned flying object 20B from disappearing from the image 6 captured by the camera 25 of the unmanned flying object 20A (that is, the camera 25 of the unmanned flying object 20A losing sight of the unmanned flying object 20B). It can continue to detect whether the aircraft 20B has deviated from the flight route 5 or not.

Here, the unmanned flying object 20A rotates about the axis Ax1 along the pitch axis Axa of the unmanned flying object 20A based on a given camera direction, which is the direction of the camera 25 of the unmanned flying object 20A. A first target angle θ1, which is an angle, and a second target angle θ2, which is an angle in the direction of rotation about the axis Ax2 along the yaw axis Axb of the unmanned air vehicle 20A, may be acquired. Then, the support structure 26 that supports the camera 25 is driven so that the camera 25 faces the direction of the first target angle θ1 in the rotation direction about the axis Ax1 along the pitch axis Axa of the unmanned air vehicle 20A. In addition, rotate the unmanned flying object 20A so that the camera 25 of the unmanned flying object 20A faces the direction of the second target angle θ2 in the rotation direction about the axis Ax2 along the yaw axis Axb of the unmanned flying object 20A. You can turn it around. By doing so, for example, with a simple support structure 26 having one actuator 262, the camera 25 of the unmanned air vehicle 20A can be directed toward the unmanned air vehicle 20B.

Also, the unmanned flying object 20B that flies ahead on the flight route 5 determines whether the unmanned flying object 20A has deviated from the flight route 5 based on the image 6 captured by the camera 25 of the unmanned flying object 20B. you can This makes it possible to detect that the unmanned air vehicle 20A has deviated from the flight route. Also, the unmanned flying object 20B also obtains the first and second target angles θ1 and θ2 based on a given camera direction, which is the direction of the camera 25 of the unmanned flying object 20B, and the camera 25 is the first target angle. The support structure 26 that supports the camera 25 may be driven to face the direction of the angle θ1, and the unmanned air vehicle 20B may be turned so that the camera 25 faces the direction of the second target angle θ2. By doing so, the camera 25 of the unmanned air vehicle 20B can be directed toward the unmanned air vehicle 20A with a simple support structure 26. FIG.

In addition, this invention is not limited to the above embodiment. For example, if the camera 25 of the unmanned flying object 20A is controlled to face outside the flight route 5, and the unmanned flying object 20B is reflected in the image 6 captured by the camera 25, the unmanned flying object 20B is directed toward the flight route 5. It may be detected that the By doing so, it is also possible to detect that the unmanned air vehicle 20B has deviated from the flight route without the administrator having to look at the image (video) of the camera 25. FIG.

1 flight control system, 5 flight route, 10 control device, 20, 20A, 20B unmanned flying object, 11, 21 processor, 12, 22 storage unit, 13, 23 communication unit, 24 positioning sensor, 25, 25a, 25b camera, 251 lens, 26, 26a, 26b support structure, 261 base, 262 actuator, 27 housing, 28 motor, 29 propeller, 6 image, 61 determination area, 62 central area, 110 flight route information acquisition unit, 120 flight route information storage Section 130 Flight Control Section 140 Image Acquisition Section 150 Central Area Setting Section 160 Target Angle Acquisition Section 170 Drive Control Section 180 Camera Direction Identification Section 190 Judgment Section 200 Abnormal Transmission Section 210 Flight Interruption Section.

Claims (6)

A flight control system including a first unmanned air vehicle and a second unmanned air vehicle that flies a flight route from a departure point to an arrival point ,
The first unmanned air vehicle,
flight control means for causing the first unmanned flying object to fly a flight route from the departure point to the arrival point on which the second unmanned flying object flies, preceding or following the second unmanned flying object; ,
a camera for photographing the second unmanned air vehicle from the position of the first unmanned air vehicle;
In a state of being attached to the first unmanned air vehicle and supporting the camera, in a rotational direction about an axis along the pitch axis of the first unmanned air vehicle with respect to the first unmanned air vehicle a support structure capable of moving the orientation of the camera;
A first target angle, which is an angle in a rotational direction about an axis along the pitch axis, and a yaw axis of the first unmanned air vehicle, based on a given camera direction, which is the direction of the camera. a target angle acquisition means for acquiring a second target angle that is an angle in the direction of rotation about the axis along the
drive control means for driving the support structure to point the camera toward the first target angle in a rotational direction about an axis along the pitch axis;
The flight control means turns the first unmanned air vehicle so that the camera faces the direction of the second target angle in a rotational direction about an axis along the yaw axis. Flight control system . 2. The target angle acquisition means according to claim 1, wherein said target angle acquisition means acquires said first target angle and said second target angle based on a camera direction when said camera faces said second unmanned air vehicle. flight control system. The target angle obtaining means identifies a camera direction when the camera faces the second unmanned flying object based on the position of the second unmanned flying object in the image captured by the camera, 2. The flight control system of claim 1, wherein the first target angle and the second target angle are obtained. The first unmanned air vehicle,
a first camera and a second camera facing in different directions;
A first camera capable of moving the direction of the first camera in a rotational direction about an axis along the pitch axis while being attached to the first unmanned air vehicle and supporting the first camera. a support structure for
A second camera capable of moving the direction of the second camera in a rotational direction about an axis along the pitch axis while being attached to the first unmanned air vehicle and supporting the second camera. a support structure for
an image selection means for selecting an image showing the second unmanned air vehicle from among the first image captured by the first camera and the second image captured by the second camera; has
When the image selection means selects the first image, the target angle acquisition means determines that the first camera is positioned in the first image based on the position of the second unmanned air vehicle in the first image. When the first target angle and the second target angle are obtained by specifying the camera direction when facing the unmanned air vehicle in 2, and the image selection means selects the second image wherein, based on the position of the second unmanned air vehicle in the second image, specifying a camera direction when the second camera faces the second unmanned air vehicle, obtain the target angle of and the second target angle of
The drive control means drives the first support structure so that the first camera faces the direction of the first target angle when the image selection means selects the first image. driving the second support structure such that the second camera is oriented toward the first target angle when the image selection means selects the second image;
The flight control means turns the first unmanned air vehicle so that the first camera faces the direction of the second target angle when the image selection means selects the first image. and when the image selection means selects the second image, the first unmanned flying object is turned so that the second camera faces the direction of the second target angle. A flight control system as described in . A flight control method for controlling the flight of a first unmanned air vehicle and the flight of a second unmanned air vehicle that flies along a flight route from a departure point to an arrival point , comprising:
The first unmanned air vehicle,
preceding or following the second unmanned flying object and flying along a flight route from the departure point on which the second unmanned flying object flies to the arrival point ;
photographing the second unmanned air vehicle from the position of the first unmanned air vehicle using a camera;
A first target angle, which is an angle in a rotational direction about an axis along a pitch axis of the first unmanned air vehicle, based on a given camera direction, which is the direction of the camera; obtaining a second target angle that is an angle in the direction of rotation about the axis along the yaw axis of the unmanned air vehicle;
A support attached to the first unmanned air vehicle for supporting the camera such that the camera is oriented toward the first target angle in a rotational direction about an axis along the pitch axis. drive the structure,
A flight control method comprising: turning such that the camera faces toward the second target angle in a rotational direction about an axis along the yaw axis. An unmanned air vehicle that flies on a predetermined flight route,
flight control means for causing the unmanned flying object to fly the flight route from the departure point to the arrival point by preceding or following another unmanned flying object flying on the flight route from the departure point to the arrival point ;
a camera for photographing the other unmanned flying object from the position of the unmanned flying object;
With the camera attached to the unmanned air vehicle and supporting the camera, the direction of the camera can be moved relative to the unmanned air vehicle in a rotational direction about an axis along the pitch axis of the unmanned air vehicle. a support structure;
Based on a given camera direction, which is the direction of the camera, a first target angle, which is an angle in a rotational direction about an axis along the pitch axis, and a a target angle acquiring means for acquiring a second target angle that is an angle in the direction of rotation about the axis;
drive control means for driving the support structure to point the camera toward the first target angle in a rotational direction about an axis along the pitch axis;
The flight control means rotates the unmanned flying body so that the camera faces the direction of the second target angle in a rotational direction about an axis along the yaw axis.

Images