JP6680498B2 - Autonomous flying vehicle, target tracking method - Google Patents

Description

  The present invention relates to an autonomous flying vehicle and a target tracking method.

  Conventionally, there are small autonomous flying robots represented by drones. For example, in Patent Document 1, there is one that autonomously tracks a moving target so that the target stays within the sensor field of view of the unmanned aerial vehicle (for example, Patent Document 1).

  In Patent Document 1 described above, it is possible to autonomously track a moving target object by capturing it with a camera or the like. However, since no consideration is given to the case where the target is lost during tracking, in such a case, the target cannot be tracked.

  In view of such a point, even if the intruder is lost, there is one that can detect the lost intruder by raising the flight altitude and overlooking the surroundings (for example, Patent Document 2).

JP, 2009-173263, A JP, 2014-149621, A

  In Patent Document 2 described above, the intruder that has been lost can be rediscovered by raising the flight altitude and looking around. However, in a situation where the flight altitude cannot be raised and the bird's-eye view cannot be obtained (for example, a situation in which there is a shield in the sky such as an underpass or a tunnel), tracking cannot be continued.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an autonomous flying vehicle and a target tracking method capable of tracking a target even in a situation in which a bird's eye view cannot be taken.

  In order to solve the above-mentioned problems, an autonomous flying vehicle according to the present invention is an autonomous flying vehicle capable of autonomous flight, and includes an imaging unit, a first image showing a predetermined target, and the imaging unit. A flight control unit that recognizes the target based on a second image captured by, and that tracks the target by flying while maintaining a predetermined distance between the target and the recognized target; And a search unit that searches for a route near the unrecognizable position based on map information when the target cannot be recognized, wherein the flight control unit is based on the route searched by the search unit. And continues the tracking of the target.

  The present invention can also be understood as a target tracking method performed by the above-mentioned autonomous flying vehicle.

  According to the present invention, it is possible to track a target even in a situation where it is not possible to look down.

It is a figure showing the example of composition of the autonomous flight control system to which the autonomous flight mobile object concerning the present invention is applied. It is a block diagram which shows the functional structure of the main-body part which an autonomous flying robot has. It is a figure which shows the example of the data which a memory | storage part memorize | stores. It is a figure which shows the example of the data which a memory | storage part memorize | stores (server). It is a figure which shows the example of environmental information. It is a sequence diagram which shows the process procedure of a tracking flight process. It is a figure which shows the example of a shortest path.

  Embodiments of an autonomous flying vehicle and a target tracking method according to the present invention will be described below in detail with reference to the accompanying drawings. In the following, when a robot, which is an autonomous flying vehicle, tracks a pedestrian, which is a target object, even if the pedestrian loses sight of the pedestrian, the map data stored in advance is used to determine the route of the pedestrian. Predict, proactively rediscover the pedestrian and continue tracking. With such tracking, it is possible to continue tracking the target even in a situation where the bird's eye cannot be overlooked. Not only pedestrians but also vehicles such as bicycles and automobiles can be similarly applied.

  FIG. 1 is a diagram showing a functional configuration of an autonomous flight system 1000 to which an autonomous flying vehicle and a target tracking method according to the present invention are applied. As shown in FIG. 1, the autonomous flight system 1000 includes an autonomous flying robot 100 and a server 200, which are connected via a wireless communication network N1. The wireless communication network N1 is, for example, a communication network using a frequency band licensed for the autonomous flying robot 100 to fly. The communication network N2 is, for example, the Internet line network. In the figure, a target T, which is a target, moves on the ground G in the direction of an arrow A, and an autonomous flying robot 100 flies above and behind the target G, tracking the target T while imaging an imaging range R.

  The autonomous flying robot 100 is a small moving body such as a drone that can fly autonomously. As shown in FIG. 1, the autonomous flying robot 100 has a plurality of rotors 101, a motor 102, a main body 103, a skid 104, and an imaging unit 105. In accordance with an instruction from the main body 103, the autonomous flying robot 100 receives the driving force from the motor 102 and rotates the rotor 101 to fly. The skid 104 is used as a foot of the autonomous flying robot 100 during takeoff and landing. The image capturing unit 105 is an image capturing device that captures an image of the target, and is configured by, for example, a CCD (Charge Coupled Device) camera.

  FIG. 2 is a block diagram showing a functional configuration of the main body 103 of the autonomous flying robot 100. As shown in FIG. 2, the main body 103 includes a storage unit 1031, a positioning unit 1032, a search unit 1033, a flight control unit 1034, a communication unit 1035, and a control unit 1036. . Actually, in addition to these units, for example, the acceleration sensor for recognizing the moving speed, the ultrasonic sensor for recognizing the altitude, the compass for recognizing the direction of itself, the battery, etc., the autonomous flying robot 100. Has the functions that are normally provided.

  The storage unit 1031 includes a general storage medium such as a memory, and stores data regarding the autonomous flying robot 100.

  FIG. 3 is a diagram illustrating an example of data stored in the storage unit. As illustrated in FIG. 3, the storage unit 101 stores image pickup information 10311 and map information 10312.

  Imaging information 10311 is image data captured by the imaging unit 105. The imaging information 10311 is, for example, an image of a screen image of the target, which is a bird's eye view of the front of the autonomous flying robot 100 flying.

  The map information 10312 is general map data used in a navigation system or the like.

  The positioning unit 1032 is, for example, a GPS (Global Positioning System) module, measures the current position of the autonomous flying robot 100, and outputs current position information including the date and time and the position. This example is described on the assumption that the current position information is updated at any time, but if the storage capacity of the storage unit 1031 is allowed, data for a certain past period is accumulated in the autonomous flying robot 100. Good.

  The search unit 1033 reads the map information 10312 and searches for a route to the target. Further, when the tracking target is lost, the search unit 1033 reads the map information 10312 and searches the shortest route among the routes in the direction in which the target has moved among the routes on the map near the lost location. To do. Various conventionally known techniques can be applied to the search method. The position of the target is transmitted from the server 200.

  When the search unit 1033 searches for a route, the flight control unit 1034 drives the motor 102 to control the rotation of the rotor 101 and starts autonomous flight to the target. When the autonomous flight starts, the flight control unit 1034 controls the rotation speed of the rotor 101 so that the ultrasonic sensor achieves a predetermined altitude (for example, 3 meters). In addition, when the flight control unit 1034 starts autonomous flight, the distance to the target is a constant distance (for example, the distance between the target and the target is a predetermined size in the screen image of the image data captured by the imaging unit 105) (for example, The number of rotations of the rotor 101 is controlled so as to fly in a rear position (2 meters away from the target). By such control, the autonomous flying robot 100 can perform tracking from the ground to the end of the flight while keeping a certain distance from the ground to the target at a certain altitude.

  The communication unit 1035 is a processing unit that wirelessly communicates with the server 200 via the wireless communication network N1 to transmit and receive various types of information according to a predetermined standard.

  The control unit 1036 controls the operation of each of the above units included in the autonomous flying robot 100. The specific operation of the autonomous flying robot 100 will be described later using a sequence diagram. Subsequently, returning to FIG. 1, the server 200 will be described.

  The server 200 is a general computer such as a PC (Personal Computer). The server 300 is a server that monitors tracking of a target by the autonomous flying robot 100.

  As shown in FIG. 1, the server 200 includes a storage unit 201, a monitoring unit 202, a first communication unit 203, a second communication unit 204, and a control unit 205.

  The storage unit 301 is a general storage device such as an HDD (Hard Disk Drive), and stores information related to tracking by the autonomous flying robot 100.

  FIG. 4 is a diagram illustrating an example of data stored in the storage unit 201. As illustrated in FIG. 4, the storage unit 301 stores current position information 2011 of the autonomous flying robot 100, imaging information 2012 captured by the autonomous flying robot 100, and environmental information 3013 around the tracked route. The current position information 2011 stores the current position information indicating the current date and time and the current position measured by the positioning unit 1032 of the autonomous flying robot 100. The imaging information 2012 stores the imaging information 10311 imaged by the imaging unit 105 of the autonomous flying robot 100. The environment information 2013 is information about the environment around the tracked route, such as operation information, road information, and disaster information.

  FIG. 5 is a diagram showing an example of the environment information 2013. As shown in FIG. 5, the environment information 3013 stores the date and time when an event that changes the environment occurs and the event information indicating the content of the event in association with each other. FIG. 5 shows that, for example, as of 12:30:00 on September 1, 2015, the vicinity of the ΔΔ street xx intersection in Shinagawa ward is closed due to construction work. As described above, the environment information 3013 stores information indicating an event that affects movement on the route tracking the target, such as road construction information around the route being tracked. For example, the monitoring unit 202 accesses the site providing the road traffic information, the website of the municipality, or the like through the communication network N2, reads the information, and outputs the information as the environmental information 2013. The latest situation is stored by storing or updating the stored information at predetermined intervals.

  In addition to the above examples, when the target is moved by the vehicle, in addition to these pieces of information, traffic congestion information (for example, the date and time when the event occurred, the highway name, and the event information are associated with each other). Information) may be included. Subsequently, returning to FIG. 1, the monitoring unit 202 will be described.

  The monitoring unit 202 is a processing unit that monitors the autonomous flying robot 100 that is flying on a path that tracks a target. The monitoring unit 202 reads the current position information and the imaging information 10311 received from the autonomous flying robot 100, and checks that the autonomous flying robot 100 is currently flying on the route tracking the target. In addition, the monitoring unit 302 accesses the site providing the road traffic information, the website of the municipality, or the like via the communication network N2 in order to acquire the environmental information 2013, and acquires the acquired information in the environmental information 2013. Record.

  The first communication unit 203 is a processing unit that wirelessly communicates with the autonomous flying robot 100 via the wireless communication network N1 to transmit and receive various types of information according to a predetermined standard.

  The second communication unit 204 is a processing unit that accesses a site that provides the road traffic information, a homepage of a municipality, or the like via the communication network N2 according to a predetermined standard.

  The control unit 205 controls the operation of each of the above units included in the server 200. The specific operation of the server 200 will be described later using a sequence diagram. Next, the processing performed by this system will be described.

  FIG. 6 is a sequence diagram showing a processing procedure of processing (tracking flight processing) performed in this system. As shown in FIG. 6, in the tracking flight process, first, an input device (for example, a keyboard) (not shown) included in the server 200 is used by a system administrator to identify the target position and the target (target information). ) Is received (step S601), and the monitoring unit 202 transmits the input target information to the autonomous flying robot 100 (step S602). The target information includes, for example, coordinate information indicating the current position of the target and a face image of the target.

  Upon receiving the target information from the server 200, the autonomous flying robot 100 starts positioning of the current position and imaging by the imaging unit 105, outputs current position information and imaging information (step S603), and outputs these information to the server 200. (Step S604). Thereafter, these pieces of information are performed in real time until the end of this processing.

  When the monitoring unit 202 of the server 200 receives the current position information and the imaging information from the autonomous flying mobile robot 100, the monitoring unit 202 records the information in the storage unit 201 (step S605).

  The flight control unit 1034 of the autonomous flying robot 100 drives the motor 102 to control the rotation of the rotor 101 and starts autonomous flight to the position of the target indicated by the target information (step S606). For example, the flight control unit 1034, after flying to near the current position of the target indicated as the target information, selects the target from the face image of the target indicated in the target information and the image captured by the image capturing unit 105. The face image similar to the face image of is recognized, and the person including the recognized face image is set as the target. After that, the autonomous flying robot 100 transmits the current position information and the imaging information to the server 200 at any time.

  When the flight control unit 1034 arrives at the target after starting the flight, the flight control unit 1034 transmits a message to that effect to the server 200 (step S607). When the input device of the server 200 receives the information indicating that the target has arrived at the target from the autonomous flying robot 100, the input of the tracking instruction for the target is accepted (step S608), and the monitoring unit 202 sends the tracking instruction to the autonomous flying robot. It is transmitted to 100 (step S609).

  When the flight control unit 1034 of the autonomous flying robot 100 receives the tracking instruction from the server 200, it starts tracking the target. The flight control unit 1034 controls the number of rotations of the rotor 101 so as to fly while keeping the distance to the target (steps S610 and S611). For example, the flight control unit 1034 analyzes the image captured by the image capturing unit 105, determines the size of the target image in the image, and flies so that the size is constant.

  In this way, by controlling the flight so that the size of the image of the target becomes constant, it is possible to fly while keeping the distance between the target actually moving and constant. In this example, the position where the flight control unit 1034 flies is positioned by image analysis. However, when the target has a mobile terminal, the radio wave between the mobile terminal and the autonomous flying robot 100 is set. The flight may be controlled so that the intensity is constant.

  The flight control unit 1034 determines whether or not the target is recognized by the positioning (step S612), and when it is determined that the target is recognized (step S612; Yes), the process returns to step S611, and the distance is set. Continue flight while maintaining.

  On the other hand, when the flight control unit 1034 determines that the target cannot be recognized (step S612; No), the search unit 1033 executes the route search (step S613). Then, the flight control unit 1034 determines whether or not the route can be searched (step S616), and when determining that the route can be searched (step S616; Yes), returns to step S610 and continues the tracking.

  Specifically, the search unit 1033 reads the map information 10312 stored in the storage unit 1031 and searches the shortest route of the routes in the moving direction of the target among the routes on the map near the lost position. Then, the route is estimated to be a route along which the target moves. The shortest route is, for example, as shown in FIG. 7, assuming that the target T moves in the traveling direction A, when the autonomous flying robot 100 loses sight of the target T at the point P immediately before the branch point, the search unit. Reference numeral 1033 is R1 searched from the routes R1, R2, and R3 to the confluence of the routes. In this way, the flight control unit 1034 has the shortest route with the smallest travel distance among the plurality of routes from the point where the route in the traveling direction searched by the search unit 1033 branches to the point where the branched route merges. To explore.

  Then, the flight control unit 1034 increases the flight speed, moves ahead of the shortest route searched until the target T moves to the confluence point, and moves to a position near the confluence point (for example, the autonomous position shown in FIG. 7). Stand by at the position of the flying robot 100 ') and ambush. Then, the flight control unit 1034 analyzes the image captured by the image capturing unit 105 at that time, and determines whether or not the size of the image of the target occupied in the image is equal to or larger than a certain size. Then, when it is determined that the size of the image of the target is equal to or larger than a certain size, it is determined that the target T has moved to the vicinity of the confluence point, and the tracking of the target T (target T ′ in FIG. 7) is continued.

  When the tracking instruction is transmitted in step S609, the monitoring unit 202 of the server 200 accesses another system such as a site providing road traffic information or a homepage of a municipality via the communication network N2, and the environment described above is set. The information is acquired, recorded in the environment information 2013 (step S614), and transmitted to the autonomous flying robot 100 (step S615). At this time, the monitoring unit 202 refers to the acquired environment information and the current position information 2011 and the imaging information 2012 stored in the storage unit 310, and refers to the environment information indicating the event that affects the movement of the user (for example, the current environment information). An event indicating construction at the intersection name included in the route) is extracted and recorded in the environment information 2013. After that, the monitoring unit 202 updates the environment information 2013 each time the environment information is acquired, and also transmits the acquired latest environment information to the autonomous flying robot 100.

  Upon receiving the environment information from the server 200, the search unit 1033 of the autonomous flying robot 100 searches for a detour route that avoids the event indicated in the environment information, and then re-searches for the shortest route so that the countermeasure can be taken again. If yes (step S616; Yes), the process returns to step S610 to continue the tracking by the autonomous flying robot 100. The autonomous flying robot 100 repeats these steps each time it acquires environment information until the tracking is completed.

  On the other hand, when the flight control unit 1034 determines that the route could not be searched in step S616 (step S616; No), it determines that the target has lost sight of the target such as entering a building, and sends a tracking impossibility notification to the server 200 ( Step S617). Even if the target is lost as described above, the route up to which the target has been tracked and the position of the target at that time can be known, and hence the range of manual tracking of the target can be limited thereafter.

  When the input device of the server 200 receives the tracking impossible notification from the autonomous flying robot 100, the input of the tracking end instruction for the target is accepted (step S618), and the monitoring unit 202 issues the tracking end instruction to the autonomous flying robot 100. (Step S619). Upon receiving the tracking end instruction from the server 200, the autonomous flying robot 100 ends the target tracking flight and returns to the position where the tracking was started (step S620).

  As described above, in the present system, by executing the above processing, the target object can be tracked even in a situation where the bird's eye view cannot be performed.

  Each process performed by the autonomous flying robot 100 and the server 200 is actually realized by executing a program installed in the autonomous flying robot 100 and the server 200. The program may be provided by being pre-installed in a ROM or the like, or may be provided by being recorded in a recording medium in a file of an installable format or an executable format, or distributed.

1000 Autonomous flight system 100 Autonomous flight robot 101 Rotor 102 Motor 103 Main body 1031 Storage unit 10311 Imaging information 10311 Map information 1032 Positioning unit 1033 Searching unit 1034 Flight control unit 1035 Communication unit 1036 Control unit 104 Skid 105 Imaging unit 200 Server 201 Storage unit 2011 current position information 2012 imaging information 2013 environment information 202 monitoring unit 203 first communication unit 204 second communication unit 205 control unit N1 wireless communication network N2 communication network

Claims (4)

An autonomous flying vehicle capable of autonomous flight,
An imaging unit,
The target is recognized based on a first image indicating a predetermined target and a second image captured by the image capturing unit, and a flight is performed while maintaining a predetermined distance from the recognized target. A flight controller that tracks the target,
In the case where the flight control unit cannot recognize the target, a search unit that searches for a route near the position that cannot be recognized based on map information,
The search unit searches for the shortest route from the branch point of the searched route to the confluence point,
The flight control unit flies along the shortest route to the vicinity of the confluence, and waits at a position near the confluence to await the target based on the second image captured by ambushing. Determines whether or not it has moved to the vicinity of the confluence, if it is determined that the target has moved to the vicinity of the confluence , continue tracking the target,
An autonomous flying vehicle characterized by the above. The search unit acquires environmental information about the route from another system, extracts an event that affects movement on the route from the environmental information, and avoids the extracted event to avoid the route. To explore,
The autonomous flying vehicle according to claim 1 , wherein: A target tracking method using an autonomously flying mobile body capable of autonomous flight,
A recognition step of recognizing the target based on a first image showing a predetermined target and a second image captured by an image capturing section provided in the autonomous flying vehicle;
A tracking step of flying and tracking the target while maintaining a predetermined distance from the recognized target;
A search step of searching for a route in the vicinity of the unrecognizable position based on map information when the target cannot be recognized;
Continuing the tracking of the target based on the searched route ,
In the search step, the shortest route from the branch point of the searched route to the confluence point is searched,
In the continuation step, the target is flown along the shortest path to the vicinity of the confluence point, and the target is based on the first image and the second image captured by awaiting at a position near the confluence point. It is determined whether or not the target has moved to the vicinity of the confluence, and if it is determined that the target has moved to the vicinity of the confluence, the tracking of the target is continued,
A target tracking method characterized by the above. In the searching step, environmental information around the route is acquired from another system, an event affecting movement on the route is extracted from the environmental information, and the extracted event is avoided to avoid the route. To explore,
The target tracking method according to claim 3 , wherein:

Images