JP2021093215A - Flying body control device, flying body control method and program - Google Patents

Abstract

To enable operation of a UAV at a low risk in a place where a human exist and a surrounding air area.SOLUTION: A flying body control device 100 includes a detection unit 101 that detects a target from video data output by an imaging device 201 for imaging a lower part of a flying body 200, a fall range estimation unit 102 that estimates a fall range of the flying body 200, and a control unit 103 that controls the flying body 200 so that a position of the detected target is not overlapped with the fall range.SELECTED DRAWING: Figure 1

Description

The present invention relates to an air vehicle control device for controlling a flight route of an air vehicle, particularly an unmanned aerial vehicle (UAV), a vehicle body control method, and an air vehicle control device and an air vehicle control for realizing these. Regarding methods and programs.

In recent years, UAVs have received great expectations as a means of effectively utilizing low airspace. This is because small UAVs have come to be offered at low cost due to the miniaturization and high output of electric motors and batteries. In particular, from the viewpoint of maneuverability and stability, most of the small UAVs are multicopter types equipped with a plurality of rotors.

In addition, due to the high performance and miniaturization of computers, UAVs are also equipped with computers, unlike conventional remote-controlled model helicopters. For this reason, as a method of controlling the flight route of the UAV, in addition to the method of manually maneuvering by the operator, a method of autonomously flying along a preset coordinate path using GPS (Global Positioning System) is also realized. Has been done.

Further, when the UAV is to fly autonomously, it is necessary to avoid a collision between the UAV and an obstacle such as a steel tower or a power transmission line. Therefore, Patent Document 1 provides a technique for automatically avoiding an obstacle in the UAV when the UAV approaches the obstacle too much by inputting the coordinates of the obstacle into the UAV in advance together with the flight route. is suggesting.

Furthermore, UAVs are also expected to be used in the field of crime prevention. For example, Patent Document 2 discloses a technique in which when a surveillance camera installed in a building detects an intruder, the UAV is rushed there, and the camera mounted on the UAV detects and tracks the intruder. There is.

Japanese Unexamined Patent Publication No. 2003-127994 Japanese Unexamined Patent Publication No. 2014-149620

However, in utilizing UAVs, especially multicopter type UAVs, there is a problem that there is a high risk of sudden falling. In addition, the cause of the sudden fall may be a change in environmental conditions due to a steering error, a dead battery, a dead fuel, a gust of wind, or the occurrence of a communication failure situation.

If the UAV should fall over the head of a pedestrian due to such a sudden fall, it would be a life-threatening situation. Moreover, it is difficult to predict such a sudden fall. For this reason, under the present circumstances, measures against a sudden fall of a UAV may limit the flight of the UAV to operation in an unoccupied space or be willing to risk a fall in order to avoid the risk of the fall. There is no choice but to limit the operation in.

An example of an object of the present invention provides an air vehicle control device, an air vehicle control method, and a program that can solve the above problems and enable the UAV to be operated with low risk even in the airspace where a person exists and its vicinity. To provide.

In order to achieve the above object, the flying object control device in one aspect of the present invention is
A detector that detects the target from the video data output by the image pickup device that captures the lower part of the aircraft.
A fall range estimation unit that estimates the fall range of the flying object,
A control unit that controls the flying object so that the detected position of the target and the fall range do not overlap with each other.
It is characterized by having.

Further, in order to achieve the above object, the flying object control method in one aspect of the present invention is:
(A) A step of detecting an object from the video data output by the image pickup device that captures the lower part of the flying object.
(B) A step for estimating the fall range of the flying object, and
(C) A step of controlling the flying object so that the detected position of the target and the falling range do not overlap with each other.
It is characterized by having.

Further, in order to achieve the above object, the computer-readable recording medium in one aspect of the present invention may be used.
On the computer
(A) A step of detecting an object from the video data output by the image pickup device that captures the lower part of the flying object.
(B) A step for estimating the fall range of the flying object, and
(C) A step of controlling the flying object so that the detected position of the target and the falling range do not overlap with each other.
It is characterized by recording a program including an instruction to execute.

As described above, according to the present invention, the UAV can be operated with a low risk even in the airspace in and near the place where a person exists.

FIG. 1 is a block diagram showing a configuration of an air vehicle control device according to a first embodiment of the present invention. FIG. 2 is a diagram showing an example of an air vehicle to be controlled and a fall range thereof in the first embodiment of the present invention. FIG. 3 is a diagram showing an example of the relationship between the flying object and the wind direction. FIG. 4 is a diagram showing an example of a fall range according to the wind speed and altitude. FIG. 5 is a diagram showing an example of the positional relationship between a person existing on the ground and a fall range. FIG. 6 is a diagram showing a case where the altitude of the flying object 200 is lowered from the state shown in FIG. FIG. 7 is a flow chart showing the operation of the flying object control device according to the first embodiment of the present invention. FIG. 8 is a block diagram showing a configuration of an air vehicle control device according to a second embodiment of the present invention. FIG. 9 is a flow chart showing the operation of the flying object control device according to the second embodiment of the present invention. FIG. 10 is a block diagram showing a configuration of an air vehicle control device according to a third embodiment of the present invention. FIG. 11 is a flow chart showing the operation of the flying object control device according to the third embodiment of the present invention. FIG. 12 is a block diagram showing another example of the configuration of the flying object control device according to the third embodiment of the present invention. FIG. 13 is a block diagram showing an example of a computer that realizes the flying object control device according to the first to third embodiments of the present invention.

(Embodiment 1)
Hereinafter, the flying object control device, the flying object control method, and the program according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 7.

[Device configuration]
First, the configuration of the flying object control device according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an air vehicle control device according to a first embodiment of the present invention.

The flying object control device 100 shown in FIG. 1 is a device for controlling the flying object 200. Here, the air vehicle 200 includes all things that can fly, such as an airplane, a helicopter, and an airship. Further, in FIG. 1, an unmanned aerial vehicle (UAV) is shown as an example of the air vehicle 200, but the air vehicle 200 may be manned or unmanned.

As shown in FIG. 1, the flying object control device 100 includes a detection unit 101, a fall range estimation unit 102, and a control unit 103. Of these, the detection unit 101 detects the target from the video data output by the image pickup device 201 that captures the lower part of the flying object 200. Targets include those that need to avoid collision with the flying object 200 due to the fall of the flying object 200, such as people, strollers, bicycles, automobiles, trains, ships, aircraft, endangered wild animals, and buildings. Can be mentioned.

Further, the fall range estimation unit 102 estimates the fall range of the flying object 200. Specifically, the fall range estimation unit 102 estimates the fall range when the flying object 200 falls by stopping or lowering the function of the flying object 200. The control unit 103 controls the flying object 200 so that the detected position of the target and the fall range do not overlap.

In this way, the flying object control device 100 detects an object whose collision with the flying object 200 is desired to be avoided from the image of the image pickup device 201 mounted on the flying object 200. Further, the flying object control device 100 controls so that the flying object 200 does not collide with the object when there is a possibility of collision with the object when the flying object 200 falls. Therefore, according to the flying object control device 100, it is possible to operate the flying object 200 with low risk even in the airspace in or near the place where a person exists.

Subsequently, the configuration of the flying object control device according to the first embodiment will be described more specifically with reference to FIG. 2 in addition to FIG. FIG. 2 is a diagram showing an example of an air vehicle to be controlled and a fall range thereof in the first embodiment of the present invention.

As shown in FIG. 2, in the first embodiment, a multicopter type UAV including four propellers is used as the flying object 200. Further, the flying object 200 is equipped with an imaging device 201 for photographing an object existing below the flying object 200. In FIG. 2, 206 indicates a collision avoidance target. In FIG. 2, the target 206 represents a person. Further, as shown in FIG. 1, the flying object 200 includes a motor 203 for driving each propeller and a motor driving unit 202 for supplying electric power to each motor in response to an instruction from the control unit 103.

Specific examples of the image pickup apparatus 201 include a visible light camera, a far infrared camera, and a ToF (Time of Flight) camera. Further, the image pickup apparatus 201 may be configured by combining two or more of these.

In the example of FIG. 2, the photographing direction of the image pickup apparatus 201 is set vertically below the flying object 200, for example. However, the shooting direction is not limited to the vertically downward direction, and may be set below the flying object 200 so that the falling range described later can be photographed. For example, the shooting direction may be a direction tilted downward from the traveling direction of the flying object 200.

Further, for example, as disclosed in Patent Document 2, when the flying object 200 includes a camera for tracking a person, the image pickup device 201 may also be used as the tracking camera. Alternatively, the camera may be separate from the tracking camera.

As described above, the detection unit 101 detects a person or the like to be subject to collision avoidance from the video data output by the image pickup apparatus 201. As a target detection method by the detection unit 101, various known detection methods can be used.

For example, when the imaging device 201 is a visible light camera, a person can be detected by using a recognition dictionary constructed in advance by using a machine learning method and in which feature quantities corresponding to recognition targets are registered. Specifically, first, various rectangular regions are cut out from the video specified by the video data, and then the feature amount is calculated from each of the cut out rectangular regions. After that, the calculated feature amount is collated with the recognition dictionary, whereby the area indicating the person is specified, and the specified area is detected as the person.

Moreover, as a learning algorithm when constructing a recognition dictionary, SVM (Support Vector Machine), a cascade type classifier and the like can be mentioned. Further, as the feature amount, HoG (Histograms of Oriented Gradients) feature amount, Haar-Like feature amount and the like can be used.

Further, for example, when the image pickup device 201 is a far-infrared camera, pixels whose pixel values correspond to temperatures close to body temperature are extracted from the infrared image specified by the image data, and these adjacent pixels are joined. A person can be detected by extracting a two-dimensional area. Further, even when the image pickup device 201 is a far-infrared camera, a person can be detected by using a recognition dictionary as in the case of a visible light camera. Specifically, in this case, a portion where the temperature suddenly changes is extracted from the infrared image specified by the video data by image processing. Then, the feature amount is calculated from the extracted part, and then the calculated feature amount is collated with the recognition dictionary, whereby the person is detected.

In addition, even when the image pickup device 201 is a ToF camera, a person can be detected by using a recognition dictionary as in the case of a visible light camera. Specifically, in this case, a portion where the distance between adjacent pixels is discontinuous is extracted from the distance image specified by the video data. Then, the feature amount is calculated from the extracted part, and then the calculated feature amount is collated with the recognition dictionary, whereby the person is detected.

Further, when the detection unit 101 detects the target, the detection unit 101 identifies the position of the target on the ground plane. Specifically, first, the three-dimensional coordinate system unique to the flying object 200, the internal parameters of the imaging device 201, the external parameters of the imaging device 201 with respect to the three-dimensional coordinate system unique to the flying object 200, and the ground height of the flying object 200 are described. Suppose it is known. In this case, the detection unit 101 identifies the coordinates of the target in the image, and executes the geometric calculation using the specified coordinates and the above-mentioned known information on the ground surface of the object in the image. The position can be specified.

The three-dimensional coordinate system unique to the air vehicle 200 is a coordinate system set inside the air vehicle 200 in order to specify the position of each part of the air vehicle 200, and may be set in advance. The internal parameters of the image pickup apparatus 201, such as the focal length, may be measured in advance.

The external parameters of the image pickup device 201 with respect to the three-dimensional coordinate system peculiar to the flying object 200 are, for example, the angle of the optical axis of the lens with respect to the flying object 200, the position of the lens, and the like. When the position and orientation of the image pickup device 200 are fixed with respect to the flying object 200, this external parameter may be measured in advance. On the other hand, when the position and orientation of the image pickup device 200 are movable with respect to the flying object 200, the external parameters include the previously measured value and the position information from the sensor that detects the position of the imaging device 200 with respect to the flying object 200. Calculated by integrating and performing computational geometry.

If the image pickup device 201 is a ToF camera, the ground height of the aircraft 200 is measured by calculating the distance from the aircraft 200 to the ground based on the distance information output by the ToF camera, for example. Good. When the flying object 200 is provided with a device for measuring altitude such as a radio altimeter, the ground height of the flying object 200 may be calculated based on the information from this device.

Further, in the detection unit 101, a coordinate system is set in order to express the position of the target on the ground surface. Here, this coordinate system is referred to as a "ground plane coordinate system". In the first embodiment, any coordinate system may be set as the surface plane coordinate system, and the coordinate system is not particularly limited.

Specifically, as the surface plane coordinate system, the origin is the point where the perpendicular line extending vertically from the flying object 200 to be set toward the ground surface intersects the ground surface, and the front direction (traveling direction) of the flying object 200 is set as the origin. ) Is the x-axis, and the axis orthogonal to the x-axis is the y-axis. This coordinate system is a coordinate system set around the flying object 200 to be set, and the origin moves every moment as the flying object 200 moves. However, in the first embodiment, if the relative position of the target with respect to the flying object 200 can be determined, the flying object 200 can be controlled. Therefore, it can be said that this ground plane coordinate system is a coordinate system suitable for the object of the present invention.

Further, as the ground plane coordinate system, a coordinate system set in the image from the image pickup apparatus 201, that is, an image coordinate system can also be mentioned. Further, as the surface plane coordinate system, a global coordinate system such as an ECEF (Earth Centered Earth Fixed) Cartesian coordinate system used in GPS or the like can also be mentioned. When the global coordinate system is used, the position in the global coordinate system of the target from the position in the global coordinate system of the flying object 200 to be set and the relative position of the target with respect to the flying object 200 to be set. Is calculated. As a result, the position of the target on the surface of the earth is represented by the global coordinate system.

In the first embodiment, the fall range estimation unit 102 estimates the fall range on the ground that is assumed when the flight object 200 to be controlled starts falling at the current time. When the flying object 200 is a multicopter type, it is difficult for the flying object 200 to glide when falling. Therefore, as shown in FIG. 2, for example, in the case of no wind, the flying object 200 is considered to fall into the circle 205 centered on the point 204 located immediately below the flying object 200. Therefore, in this case, the fall range estimation unit 102 estimates the circle 205 as the fall range. Further, since the radius of the circle 205 changes according to the altitude of the flying object 200, the fall range estimation unit 102 sets the radius of the circle 205 based on the relationship between the preset altitude and the radius of the circle 205. Set.

Also, more practically, the fall range is significantly affected by the wind. Therefore, in estimating the fall range, it is preferable that the direction of the wind and the wind speed are used as parameters in addition to the altitude of the flying object 200. Therefore, in the first embodiment, the fall range estimation unit 102 holds at least the fall range for the combination of altitude and wind speed from the ground as a database in advance, and estimates the fall range by referring to the database. Is preferable. This point will be described with reference to FIGS. 3 and 4.

FIG. 3 is a diagram showing an example of the relationship between the flying object and the wind direction. FIG. 4 is a diagram showing an example of a fall range according to the wind speed and altitude. As shown in FIG. 3, it is assumed that the flying object 200 is affected by the wind. In this case, the fall range estimation unit 102 sets a direction (direction indicated by arrow 301) obtained by projecting a virtual arrow indicating the direction of the wind on the ground along the vertical direction as the main axis direction. The orthogonal direction (direction indicated by the arrow 302) is set in the sub-axis direction.

Then, as shown in FIG. 4, the fall ranges 401 to 403 are set for various combinations of the wind speed and the current altitude of the flying object 200 as parameters. The set drop ranges 401 to 403 are associated with the corresponding combinations and are retained as a database.

Further, instead of using the database, the fall range estimation unit 102 executes a physical simulation when the flying object 200 falls based on the direction of the wind, the wind speed, and the altitude of the flying object 200 in real time. The fall range can also be calculated.

Further, in FIGS. 3 and 4, the wind direction and the wind speed can be estimated by using the attitude control information of the flying object 200 at the time when the autonomous control before the start of the fall was functioning well. However, in this case, since the wind direction and the wind speed are estimated on the three-dimensional coordinate system peculiar to the flying object 200, the fall range estimation unit 102 has the computational geometry as in the case of the target detection by the detection unit 101. Is used to set the fall range in the surface plane coordinate system.

In the examples of FIGS. 3 and 4, an example is described in which the direction 301 obtained by projecting a virtual arrow indicating the wind direction onto the ground along the vertical direction is the main axis direction, but there is no wind. In the case of, the spindle direction is indefinite. However, in the case of no wind, the fall range is concentric circles centered on the current position of the flying object 200, so that there is no problem even if the spindle direction is arbitrarily set.

In the first embodiment, the control unit 103 determines whether or not the position of the target and the fall range are represented by the surface plane coordinate system, so that they overlap on the ground plane coordinate system. As a result of the determination, when the two overlap, the control unit 103 positions the flying object 200 so that the target detected by the detection unit 101 does not fall within the fall range estimated by the fall range estimation unit 102. To control. Specifically, the control unit 103 adjusts the output of each motor 203 via the motor drive unit 202 so that the target position and the drop range do not overlap, and causes the flying object 200 to perform an avoidance motion. ..

The following two are the main avoidance movements of the flying object 200 performed in order to avoid the overlap between the position of the target and the fall range. One is an avoidance motion that moves the flying object 200 in the horizontal direction so that the detected object does not fall within the fall range. The other is an avoidance motion in which the flying object 200 is lowered to lower the altitude so that the detected target does not fall within the fall range.

The merit of the former is that the relative position of the fall range with respect to the position of the flying object 200 basically does not change, so it is sufficient to consider moving the fall range horizontally so that the detected target does not fall within the fall range. Is easy. The direction and distance for horizontal movement of the fall range correspond to the direction and distance for moving the flying object 200.

The advantage of the latter is that the area of the fall range can be narrowed down to make it smaller, and the impact in the event of a fall can be made smaller. The reason why the area of the fall range can be reduced is that when the latter avoidance motion is performed, the fall time until the flying object 200 reaches the ground is shortened, and the amount of movement of the flying object 200 in the horizontal direction at the time of falling is surely reduced. This is because it decreases.

Here, with reference to FIGS. 5 and 6, the merits of the avoidance motion for lowering the altitude of the flying object will be specifically described. FIG. 5 is a diagram showing an example of the positional relationship between a person existing on the ground and a fall range. FIG. 6 is a diagram showing a case where the altitude of the flying object 200 is lowered from the state shown in FIG.

As shown in FIG. 5, for example, when a plurality of people 502 exist on the ground and the flying object 200 is located at the current altitude, the flying object 200 falls no matter how horizontally it moves. It is assumed that the person 502 is in the range 501. In such a state, when the flying object 200 is made to perform an avoidance motion for lowering the altitude, as shown in FIG. 6, the fall range 601 is narrower than the fall range 501, so that the positions of the fall range 601 and the person 602 are located. And can be prevented from overlapping.

[Device operation]
Next, the operation of the flying object control device 100 according to the first embodiment of the present invention will be described with reference to FIG. 7. FIG. 7 is a flow chart showing the operation of the flying object control device according to the first embodiment of the present invention. In the following description, FIGS. 1 to 6 will be referred to as appropriate. Further, in the first embodiment, the flying object control method is implemented by operating the flying object control device 100. Therefore, the description of the flying object control method in the first embodiment is replaced with the following description of the operation of the flying object control device 100.

As shown in FIG. 7, first, the detection unit 101 acquires the video data output by the image pickup apparatus 201 (step A1). Next, the detection unit 101 detects the target from the video data acquired in step A1 (step A2). Further, in step A2, the detection unit 101 applies various parameters to the coordinates of the target in the image, executes geometric calculation, and specifies the position of the detected target on the ground plane.

Next, the fall range estimation unit 102 estimates the fall range when the flying object 200 falls (step A3). At this time, the fall range estimation unit 102 estimates the fall range in consideration of the wind direction and the wind speed in the airspace where the flying object 200 is flying.

After that, the control unit 103 determines whether or not the target detected in step A2 and the fall range estimated in step A3 overlap, and if they do overlap, the flying object 200 is moved so that the target does not fall within the fall range. Control (step A4). Specifically, the control unit 103 causes the flying object 200 to perform avoidance movements of either or both of the movement in the horizontal direction and the decrease in altitude, and eliminates the overlap between the target and the fall range.

In step A4, for example, when the control unit 103 cannot eliminate the overlap between the detected target position and the fall range even if the control unit 103 controls the flight body 200 to move in the horizontal direction, the flight body 200 The flying object 200 may be controlled so that the altitude of the aircraft 200 is lowered so that the detected position of the target and the fall range do not overlap.

[Effect of Embodiment 1]
As described above, in the first embodiment, it is possible to reduce the risk of falling in the operation of the flying object 200. According to the first embodiment, it is possible to operate the flying object 200 with low risk even in the airspace in and near the place where a person exists.

[program]
The program according to the first embodiment may be any program that causes a computer to execute steps A1 to A4 shown in FIG. 7. By installing this program on a computer and executing it, the flying object control device 100 and the flying object control method according to the first embodiment can be realized. In this case, the CPU (Central Processing Unit) of the computer functions as a detection unit 101, a drop range estimation unit 102, and a control unit 103 to perform processing.

Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the detection unit 101, the drop range estimation unit 102, and the control unit 103, respectively.

(Embodiment 2)
Next, the flying object control device, the flying object control method, and the program according to the second embodiment of the present invention will be described with reference to FIGS. 8 and 9.

[Device configuration]
First, the configuration of the flying object control device according to the second embodiment will be described with reference to FIG. FIG. 8 is a block diagram showing a configuration of an air vehicle control device according to a second embodiment of the present invention.

As shown in FIG. 8, the flying object control device 120 according to the second embodiment has a movement range estimation unit 104 in addition to the detection unit 101, the fall range estimation unit 102, and the control unit 103, which are also shown in FIG. Is further equipped. The flying object control device 120 according to the second embodiment is different from the flying object control device 100 according to the first embodiment shown in FIG. 1 in this respect. Hereinafter, the differences from the first embodiment will be mainly described.

Assuming that the flying object 200 has begun to fall, the movement range estimation unit 104 has a range in which the target detected by the detection unit 101 may move on the ground between the start of the fall and the arrival at the ground. (Hereinafter referred to as "expected movement range") is estimated.

For example, it is assumed that the flying object 200 starts to fall from the current time, it takes 5 seconds to reach the ground, and the target person runs at a maximum speed of 7 m / sec. In this case, the movement range estimation unit 104 estimates a circular range with a radius of 35 m centered on the existence position of the target person as the expected movement range. The expected movement range is also finally expressed in the ground plane coordinate system.

Further, in the second embodiment, unlike the first embodiment, the control unit 103 estimates the expected movement range for the target by the movement range estimation unit 104, and the expected movement range estimated by the movement range estimation unit 104. Determines if and the fall range overlap. As a result of the determination, when the two overlap, the control unit 103 controls the flying object 200 so that the two do not overlap. As described above, since the expected movement range is also expressed in the ground plane coordinate system like the fall range, it is easy for the control unit 103 to determine the overlap.

[Device operation]
Next, the operation of the flying object control device 120 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 9 is a flow chart showing the operation of the flying object control device according to the second embodiment of the present invention. In the following description, FIG. 8 will be referred to as appropriate. Further, in the second embodiment, the flying object control method is implemented by operating the flying object control device 120. Therefore, the description of the flying object control method in the second embodiment is replaced with the following description of the operation of the flying object control device 120.

As shown in FIG. 9, first, the detection unit 101 acquires the video data output by the image pickup apparatus 201 (step B1). Next, the detection unit 101 detects the target from the video data acquired in step B1 (step B2). Further, in step B2, the detection unit 101 specifies the position of the detected target on the ground plane. Note that steps B1 and B2 are the same steps as steps A1 and A2 shown in FIG. 7, respectively.

Next, the movement range estimation unit 104 estimates the expected movement range of the target detected in step B2 (step B3). Specifically, the moving range estimation unit 104 calculates the moving speed of the target on the video data, and further, from the altitude and speed of the flying object 200, the flying object 200 when it starts to fall at the present time is on the ground. The time to reach is also calculated. Then, the movement range estimation unit 104 estimates the expected movement range using the calculated movement speed and time of the target.

Next, the fall range estimation unit 102 estimates the fall range when the flying object 200 falls (step B4). After that, the control unit 103 determines whether or not the expected movement range estimated in step B3 and the fall range estimated in step B4 overlap, and if they do overlap, the predicted movement range and the fall range do not overlap. Control the flying object 200 (step B5). Specifically, the control unit 103 causes the flying object 200 to perform avoidance movements of either or both of the movement in the horizontal direction and the decrease in altitude, and eliminates the overlap between the expected movement range and the fall range. Note that step B4 is the same step as step A3 shown in FIG.

In step B5, for example, when the control unit 103 cannot eliminate the overlap between the detected target position and the fall range even if the control unit 103 controls the flight body 200 to move in the horizontal direction, the flight body 200 The flying object 200 may be controlled so that the altitude of the aircraft 200 is lowered so that the expected movement range and the fall range do not overlap.

[Effect of Embodiment 2]
As described above, according to the second embodiment, the collision between the flying object 200 and the target at the time of falling can be more reliably avoided, so that the risk due to the falling is further reduced in the operation of the flying object 200. It becomes possible to do.

Further, in the above-mentioned example, an example in which the flying object 200 is controlled so that the expected moving range and the falling range do not overlap is described, but the second embodiment is not limited to this example. For example, the movement range estimation unit 104 represents the possibility that the flying object collides with the target at the time of falling, such as the existence probability of the target at the time of falling of the flying body, for each set area instead of the above-mentioned predicted moving range. A numerical value may be calculated. In this case, the control unit 103 controls the flying object 200 so as to move to the area where the value becomes the minimum value. According to this aspect, the risk of the flying object 200 falling to a place where more people exist can be reduced, and the flying object 200 can be optimally controlled.

[program]
The program according to the second embodiment may be any program that causes a computer to execute steps B1 to B5 shown in FIG. By installing this program on a computer and executing it, the flying object control device 120 and the flying object control method according to the second embodiment can be realized. In this case, the CPU (Central Processing Unit) of the computer functions as a detection unit 101, a drop range estimation unit 102, a control unit 103, and a movement range estimation unit 104, and performs processing.

Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the detection unit 101, the drop range estimation unit 102, the control unit 103, and the movement range estimation unit 104, respectively.

(Embodiment 3)
Next, the flying object control device, the flying object control method, and the program according to the third embodiment of the present invention will be described with reference to FIGS. 10 and 11.

[Device configuration]
First, the configuration of the flying object control device according to the third embodiment will be described with reference to FIG. FIG. 10 is a block diagram showing a configuration of an air vehicle control device according to a third embodiment of the present invention.

As shown in FIG. 10, the flying object control device 130 according to the third embodiment has a fall risk estimation unit in addition to the detection unit 101, the fall range estimation unit 102, and the control unit 103, which are also shown in FIG. 105 is further provided. The flying object control device 130 according to the third embodiment is different from the flying object control device 100 according to the first embodiment shown in FIG. 1 in this respect. Hereinafter, the differences from the first embodiment will be mainly described.

As shown in FIG. 10, the fall risk estimation unit 105 estimates the risk of the flying object 200 falling based on the current situation of the flying object 200. Specifically, the fall risk estimation unit 105 is, for example, the wind speed, the turbulence of the air flow, the remaining battery level, the reception state of the radio wave for remote control, the temperature of the motor 203, and the computer for constructing the flying object control device 130. The temperature, etc. is monitored every moment, and the value of each item (denoted as "monitoring information") is acquired.

Then, since each item to be monitored increases the risk of falling when the value exceeds a certain value, a threshold value is set in advance for each item. Therefore, the fall risk estimation unit 105 determines whether or not the value of each item to be monitored is equal to or greater than a predetermined threshold value indicating the fall risk.

As a result of the determination, if one or more items exceed the threshold value, the fall risk estimation unit 105 determines that the fall risk is increasing, and outputs to that effect to the control unit 103. On the other hand, as a result of the determination, if the threshold value is not exceeded for any of the items, the fall risk estimation unit 105 determines that the fall risk at the current time is low, and outputs to that effect to the control unit 103.

When the control unit 103 determines that the fall risk estimation unit 105 has a high risk of falling, the position of the target detected by the detection unit 101 does not overlap with the fall range estimated by the fall range estimation unit 102. , Control the flying object 200.

Further, when the control unit 103 determines that the fall risk estimation unit 105 has a low risk of falling, the position of the target detected by the detection unit 101 overlaps with the fall range estimated by the fall range estimation unit 102. Can also be forgiven. In this case, the control unit 103 does not cause the flying object 200 to perform the avoidance motion.

[Device operation]
Next, the operation of the flying object control device 130 according to the third embodiment of the present invention will be described with reference to FIG. FIG. 11 is a flow chart showing the operation of the flying object control device according to the third embodiment of the present invention. In the following description, FIG. 10 will be referred to as appropriate. Further, in the third embodiment, the flying object control method is implemented by operating the flying object control device 130. Therefore, the description of the flying object control method in the third embodiment will be replaced with the following description of the operation of the flying object control device 130.

As shown in FIG. 11, first, the detection unit 101 acquires the video data output by the image pickup apparatus 201 (step C1). Next, the detection unit 101 detects the target from the video data acquired in step C1 (step C2). Further, in step C2, the detection unit 101 specifies the position of the detected target on the ground plane. Next, the fall range estimation unit 102 estimates the fall range when the flying object 200 falls (step C3). Note that steps C1 to C3 are the same steps as steps A1 to A3 shown in FIG. 7, respectively.

Next, the fall risk estimation unit 105 estimates the risk of the flying object 200 falling based on the current situation of the flying object 200 (step C4). Specifically, in step C4, the fall risk estimation unit 105 determines whether or not the value of each item that is constantly monitored is equal to or higher than a predetermined threshold value. As a result of the determination, the fall risk estimation unit 105 determines that the risk of falling is increased if one or more items exceed the threshold value, and determines that the risk of falling is low otherwise. ..

After that, the control unit 103 confirms whether or not the risk is determined to be high in step C4, and if it is determined to be high, the control unit 103 controls the flying object 200 so that the target does not fall within the fall range. (Step C5).

[Effect of Embodiment 3]
According to the third embodiment as described above, when the risk of falling is small, the flying object 200 can fly over a person, so that the flying object 200 works in a densely populated place. It is possible to pass through the airspace. As a result, the work efficiency of the flying object 200 can be improved.

Further, as shown in FIG. 12, the flying object control device 130 in the third embodiment may include the movement range estimation unit 104 shown in FIG. 8 in the second embodiment. In this case, the control unit 103 controls the flying object 200 so that the fall range does not overlap with any of the target position and the expected movement range when it is determined in step C4 that the risk is high.

[program]
The program according to the third embodiment may be any program that causes the computer to execute steps C1 to C5 shown in FIG. By installing this program on a computer and executing it, the flying object control device 130 and the flying object control method according to the third embodiment can be realized. In this case, the CPU (Central Processing Unit) of the computer functions as a detection unit 101, a fall range estimation unit 102, a control unit 103, and a fall risk estimation unit 105 to perform processing.

Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the detection unit 101, the fall range estimation unit 102, the control unit 103, and the fall risk estimation unit 105, respectively.

(Physical configuration)
Here, a computer that realizes an air vehicle control device by executing the programs according to the first to third embodiments will be described with reference to FIG. FIG. 13 is a block diagram showing an example of a computer that realizes the flying object control device according to the first to third embodiments of the present invention.

The computer shown in FIG. 13 is an embedded computer mounted on the flying object. However, in the first to third embodiments, the computer may be a computer that performs data communication with the flying object from the ground. In this case, the flying object control device will be constructed in a place different from the flying object.

As shown in FIG. 13, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, and a communication interface 114. Each of these parts is connected to each other via a bus 115 so as to be capable of data communication.

The CPU 111 expands the programs (codes) of the present embodiment stored in the storage device 113 into the main memory 112 and executes them in a predetermined order to perform various operations.

Specific examples of the storage device 113 include a semiconductor storage device such as a flash memory. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). The communication interface 114 mediates data transmission between the CPU 111 and another computer.

The program according to this embodiment is provided in a state of being stored in a computer-readable recording medium 116. Specific examples of the recording medium 116 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic storage media such as flexible disks, and CD-ROMs (CD-ROMs). Examples include optical storage media such as Compact Disk Read Only Memory). The program in this embodiment may be provided via the communication interface 114.

The flying object control device in the present embodiment can also be realized by using the hardware corresponding to each part instead of the computer in which the program is installed. Further, the air vehicle control device may be partially realized by a program and the rest may be realized by hardware.

A part or all of the above-described embodiments can be expressed by the following descriptions (Appendix 1) to (Appendix 18), but the present invention is not limited to the following description.

(Appendix 1)
A detector that detects the target from the video data output by the image pickup device that captures the lower part of the aircraft.
A fall range estimation unit that estimates the fall range of the flying object,
A control unit that controls the flying object so that the detected position of the target and the fall range do not overlap with each other.
An air vehicle control device characterized by being provided with.

(Appendix 2)
The fall range estimation unit estimates the fall range based on the altitude of the flying object from the ground and wind information that specifies the direction and speed of the wind.
The flying object control device according to Appendix 1.

(Appendix 3)
The fall range estimation unit holds at least a fall range for a combination of altitude and wind speed from the ground as a database in advance, and estimates the fall range with reference to the database.
The flying object control device according to Appendix 2.

(Appendix 4)
Further provided with a movement range estimation unit that estimates the range in which the detected target may move on the ground from the start of the fall to the arrival at the ground as the expected movement range. ,
The control unit controls the flying object so that the estimated expected movement range and the fall range do not overlap.
The flying object control device according to Appendix 1.

(Appendix 5)
If the control unit cannot eliminate the overlap between the detected position of the target and the fall range even if the control unit controls the flying object to move in the horizontal direction, the control unit reduces the altitude of the flying object. Control the flying object so that the detected position of the target and the fall range do not overlap.
The flying object control device according to Appendix 1.

(Appendix 6)
Further provided with a fall risk estimation unit that estimates the risk of the flying object falling based on the situation of the flying object.
When the estimated danger exceeds a predetermined level, the control unit sets the flying object so that one of the detected target position and the predicted movement range does not overlap with the fall range. Control,
The flying object control device according to Appendix 4.

(Appendix 7)
(A) A step of detecting an object from the video data output by the image pickup device that captures the lower part of the flying object.
(B) A step for estimating the fall range of the flying object, and
(C) A step of controlling the flying object so that the detected position of the target and the falling range do not overlap with each other.
A flying object control method characterized by having.

(Appendix 8)
In step (b), the fall range is estimated based on the altitude of the flying object from the ground and the wind information that specifies the direction and speed of the wind.
The flying object control method according to Appendix 7.

(Appendix 9)
In step (b), the fall range for at least the combination of altitude and wind speed from the ground is stored in advance as a database, and the fall range is estimated with reference to the database.
The flying object control method according to Appendix 8.

(Appendix 10)
(D) Further, there is a step of estimating the range in which the detected target may move on the ground from the start of the fall to the arrival at the ground as the expected movement range. And
In step (c), the flying object is controlled so that the estimated expected movement range and the fall range do not overlap.
The flying object control method according to Appendix 7.

(Appendix 11)
In the step (c), if the overlap between the detected position of the target and the fall range cannot be eliminated even if the flying object is controlled to move in the horizontal direction, the altitude of the flying object is lowered. The flying object is controlled so that the detected position of the target and the falling range do not overlap with each other.
The flying object control method according to Appendix 7.

(Appendix 12)
(E) Further having a step of estimating the risk of the flying object falling based on the situation of the flying object.
In the step (c), when the estimated risk exceeds a predetermined level, the fall range is not overlapped with any of the detected target position and the expected movement range. Control the flying object,
The flying object control method according to Appendix 10.

(Appendix 13)
On the computer
(A) A step of detecting an object from the video data output by the image pickup device that captures the lower part of the flying object.
(B) A step for estimating the fall range of the flying object, and
(C) A step of controlling the flying object so that the detected position of the target and the falling range do not overlap with each other.
A program that executes.

(Appendix 14)
In step (b), the fall range is estimated based on the altitude of the flying object from the ground and the wind information that specifies the direction and speed of the wind.
The program described in Appendix 13.

(Appendix 15)
In step (b), the fall range for at least the combination of altitude and wind speed from the ground is stored in advance as a database, and the fall range is estimated with reference to the database.
The program described in Appendix 14.

(Appendix 16)
On the computer
(D) Further execution of the step of estimating the range in which the detected target may move on the ground from the start of the fall to the arrival at the ground as the expected movement range. Let me
In step (c), the flying object is controlled so that the estimated expected movement range and the fall range do not overlap.
The program described in Appendix 13.

(Appendix 17)
In the step (c), if the overlap between the detected position of the target and the fall range cannot be eliminated even if the flying object is controlled to move in the horizontal direction, the altitude of the flying object is lowered. The flying object is controlled so that the detected position of the target and the falling range do not overlap with each other.
The program described in Appendix 13.

(Appendix 18)
On the computer
(E) Based on the situation of the flying object, the step of estimating the risk of the flying object falling is further executed.
In the step (c), when the estimated risk exceeds a predetermined level, the fall range is not overlapped with any of the detected target position and the expected movement range. Control the flying object,
The program according to Appendix 16.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the structure and details of the present invention.

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2015-165850 filed on August 25, 2015, the entire disclosure of which is incorporated herein by reference.

As described above, according to the present invention, an air vehicle such as a UAV can be operated with a low risk even in an airspace where a person exists and in the vicinity thereof. The present invention is useful for various flying objects.

100 Flying object control device 101 Detection unit 102 Fall range estimation unit 103 Control unit 104 Movement range estimation unit 105 Fall risk estimation unit 110 Computer 111 CPU
112 Main memory 113 Storage device 114 Communication interface 115 Bus 116 Recording medium 200 Air vehicle 201 Imaging device 202 Motor drive unit 203 Motor 204 Point located directly under the air vehicle 205 Fall range circle 206 Target (person)
301 Direction obtained by projecting a virtual arrow indicating the direction of the wind on the ground (main axis direction)
302 Direction orthogonal to the main axis direction (sub-axis direction)
401, 402, 403 Fall range set for each combination of wind speed and altitude 501 Fall range that overlaps with the position of people 502 people 601 Fall range that does not overlap with the position of people 602

Claims (9)

A detector that detects an object using a machine learning method from the video data output by an image pickup device that has an infrared detection device and captures the lower part of the flying object.
A control unit that controls the air vehicle so that the position of the target detected by the detection unit and the landing range of the air vehicle do not overlap with each other.
An air vehicle control device characterized by being provided with. The flying object control device according to claim 1.
A fall range estimation unit that estimates the fall range based on the altitude of the flying object from the ground and wind information that specifies the direction and speed of the wind.
A flying object control device. The flying object control device according to claim 2.
The fall range estimation unit holds at least a fall range for a combination of altitude and wind speed from the ground as a database in advance, and estimates the fall range with reference to the database.
Aircraft control device. The flying object control device according to any one of claims 1 to 3.
The movement range estimation unit, which estimates the range in which the detected target may move on the ground between the time when the flying object starts falling and the time when it reaches the ground, is estimated as the expected movement range.
With more
The control unit controls the flying object so that the estimated expected movement range and the fall range do not overlap.
Aircraft control device. The flying object control device according to any one of claims 1 to 4.
The movement range estimation unit calculates the movement speed of the target detected by the detection unit.
Aircraft control device. The flying object control device according to any one of claims 1 to 5.
If the control unit cannot eliminate the overlap between the detected position of the target and the fall range even if the control unit controls the flying object to move in the horizontal direction, the control unit reduces the altitude of the flying object. Control the flying object so that the detected position of the target and the fall range do not overlap.
Aircraft control device. The flying object control device according to claim 4.
Further provided with a fall risk estimation unit that estimates the risk of the flying object falling based on the situation of the flying object.
When the estimated danger exceeds a predetermined level, the control unit sets the flying object so that one of the detected target position and the predicted movement range does not overlap with the fall range. Control,
Aircraft control device. (A) A step of detecting an object by using a machine learning method from the video data output by an imaging device that captures the lower part of the flying object.
(B) A step of controlling the flying object so that the position of the target and the landing range of the flying object do not overlap with each other.
A flying object control method characterized by having. On the computer
(A) A step of detecting an object by using a machine learning method from the video data output by an imaging device that captures the lower part of the flying object.
(B) A step of controlling the flying object so that the position of the target and the landing range of the flying object do not overlap with each other.
A program that executes.

Images