JP2019159624A - Unmanned aircraft, and watching method - Google Patents

Abstract

To provide a technique capable of inhibiting that an acting person who generates a hazardous event runs away, when the hazardous event (snatching, violence action, or the like) is generated to an object person, that is, to provide an unmanned aircraft and a watching method thereof.SOLUTION: A flight control part 108 tracks an object person, based on a position of the object person detected by an object person detection part 106. When detecting that a hazardous event is generated to the object person tracked by a hazardous event detection part 104, the flight control part 108 tracks a moving body detected after a moving body detection part 105 processes a frame image relating to a moving image imaged by a camera 6.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for watching an object person.

  Conventionally, there has been a technique for watching an object person with an unmanned aerial vehicle (see Patent Documents 1 and 2).

  In Patent Document 1, an information terminal device such as a smartphone carried by a subject (monitored person) transmits the position of the terminal acquired from a GPS or a mobile phone wireless base station to an unmanned aircraft (drone device). The unmanned aerial vehicle acquires the position of its own aircraft by GPS, tracks the target person by self-sustained flight based on the received position of the information terminal device and the acquired position of the own aircraft, and shoots with the mounted video camera In this configuration, the peripheral video of the target person is transmitted to a predetermined transmission destination.

  Patent Document 2 directly receives a beacon signal transmitted by a portable beacon transmission unit carried by a subject by an unmanned aircraft. The unmanned aerial vehicle is configured to track a target person by self-sustained flight based on the direction of arrival of a beacon signal and to transmit a peripheral image of the target person captured by a mounted video camera to a predetermined transmission destination.

JP2015-207149A JP 2017-114270 A

  However, in Patent Documents 1 and 2 described above, by tracking a target person who possesses a terminal (an information terminal device or a portable beacon transmission unit), the target person can be watched for a dangerous event (snacking). , Violent acts, etc.) are not supposed to be prevented from being escaped by the actor who caused the dangerous event.

  An object of the present invention is to provide a technique capable of preventing an actor who has generated a dangerous event from escaping when a dangerous event (such as snatching or violent behavior) occurs in a subject. is there.

  The unmanned aerial vehicle of the present invention is configured as follows to achieve the above object.

  A moving body detection unit detects a moving body being imaged by processing a frame image related to a moving image captured by a mounted imaging device. In addition, the target person detection unit detects the position of the target person. The flight control unit performs first flight control for tracking the target person based on the position of the target person detected by the target person detection part. In addition, the event detection unit detects whether or not a dangerous event has occurred in the subject being tracked. When the flight control unit detects that a dangerous event has occurred in the subject tracked by the event detection unit, the flight detection unit processes a frame image of the moving image captured by the imaging device. Second flight control for tracking the detected moving object is performed.

  According to this configuration, when a dangerous event (such as snatching or violent behavior) occurs in the target person, tracking of the actor who generated the dangerous event is started. Specifically, when a dangerous event occurs in the target person, it is detected as a moving body that tracks an actor or a vehicle, motorcycle, etc. on which the actor gets on, and tracking of this moving body is started. Therefore, when a dangerous event occurs in the subject, it is possible to prevent the actor who generated the dangerous event from escaping.

  In addition, the second flight control by the flight control unit is, for example, an unmanned aerial vehicle such that the moving body is captured in a frame image related to a moving image captured by the imaging device in accordance with the movement of the tracked moving body. What is necessary is just to set it as the structure which controls the flight of a main body. Thereby, it is possible to suppress losing sight of the moving body in the tracking of the moving body that has caused the dangerous event to the subject.

  In addition, when the flight control unit starts the second flight control, an output unit may be provided that outputs a tracking request for the subject including position information where the dangerous event has occurred in the subject. If comprised in this way, when a dangerous event generate | occur | produces in an object person, the said object person's watching over can be handed over to another unmanned aircraft.

  Furthermore, a moving image storage unit that stores a moving image picked up by the image pickup device, and a period from when the flight control unit starts tracking the target person through the first flight control to when the tracking of the target person ends. An image deletion unit that deletes a moving image captured by the imaging apparatus during this period from the moving image storage unit if the event detection unit does not detect that a dangerous event has occurred in the subject being tracked It may be.

  In this way, a moving image of the subject captured while the subject is watching is not seen by others. Therefore, it is possible to watch the subject's safety and security without infringing on the subject's privacy.

  According to the present invention, when a dangerous event (such as snatching, violent behavior, etc.) occurs in the subject, it is possible to prevent the actor who generated the dangerous event from escaping.

It is a figure explaining the outline | summary of a watching system. It is a block diagram which shows the structure of the principal part of drone. It is a block diagram which shows the structure of the principal part of a portable terminal. It is a block diagram which shows the structure of the principal part of a management apparatus. It is a flowchart which shows the monitoring operation | movement of the subject in a drone.

  Embodiments of the present invention will be described below.

<1. Application example>
FIG. 1 is a diagram for explaining an overview of a monitoring system according to an embodiment of the present invention. This watching system includes a drone 1, a mobile terminal 2, and a management device 3. The drone 1 corresponds to the unmanned aircraft referred to in the present invention. The portable terminal 2 is possessed by the subject. The management device 3 manages one or a plurality of drones 1.

  The portable terminal 2 has a GPS function that acquires the position (latitude, longitude, altitude) of the terminal itself using GPS (Global Positioning System). The portable terminal 2 transmits position information indicating the position of the own terminal acquired by the GPS function to the drone 1. The mobile terminal 2 may be configured to transmit position information directly to the drone 1 or may be configured to transmit to the drone 1 via the management device 3.

  The drone 1 has a GPS function for acquiring the position (latitude, longitude, altitude) of its own device, similar to the mobile terminal 2. The drone 1 acquires the position of the own device by GPS, and tracks the subject by independent flight based on the position of the mobile terminal 2 based on the received position information and the acquired position of the own device. That is, the drone 1 performs a self-sustained flight that changes the position of the own aircraft according to a change in the position of the mobile terminal 2.

  Further, when the drone 1 is tracking the target person, the drone 1 captures a peripheral video of the target person with a video camera installed therein. The drone 1 processes the surrounding video of the subject using a video camera, and detects whether or not a dangerous event (such as snatching or violence) has occurred in the subject. When the drone 1 detects that a dangerous event has occurred in the target person, the drone 1 starts tracking the actor who has generated the dangerous event.

  The drone 1 and the mobile terminal 2 perform data communication with the management device 3 via the network 5 shown in FIG.

<2. Configuration example>
FIG. 2 is a block diagram showing the configuration of the main part of the drone. The drone 1 includes a control unit 11, a storage unit 12, a moving image input unit 13, a voice input unit 14, a communication unit 15, a sensing unit 16, and a flight drive unit 17.

  The control unit 11 controls the operation of each part of the drone 1 body. The control unit 11 includes an image processing unit 101, an audio processing unit 102, a sensing signal processing unit 103, a dangerous event detection unit 104, a moving object detection unit 105, a target person detection unit 106, and a moving image deletion unit 107. And a flight control unit 108. Details of the configuration of the control unit 11 will be described later.

  The storage unit 12 includes a moving image storage unit 111 and a tracking information storage unit 112. The moving image storage unit 111 stores a moving image captured by the camera 6 mounted on the drone 1 body. The camera 6 is a video camera and captures a moving image at a frame rate of about several tens of frames / sec (10 to 30 frames / sec). The camera 6 corresponds to the imaging device referred to in the present invention. The tracking information storage unit 112 stores information indicating the flight path of the drone 1 main body as tracking information. This tracking information is flight history information in which the date and time and the position (latitude, longitude, altitude) of the drone 1 main body are associated with each other. The tracking information may be information obtained by registering the location of the drone 1 at a constant time interval, or may be information obtained by registering the location of the drone 1 at irregular time intervals.

  A moving image captured by the camera 6 is input to the moving image input unit 13.

  The voice input unit 14 receives a voice signal related to a voice collected by a microphone 7 (hereinafter simply referred to as a microphone 7) mounted on the drone 1 body.

  The communication unit 15 performs data communication between the mobile terminal 2 and the management device 3. The communication unit 15 has a configuration corresponding to the output unit referred to in the present invention.

  The sensing unit 16 has various sensors. The sensing unit 16 includes a GPS sensor that measures the position (latitude, longitude, altitude) of the drone 1 body, a 3-axis acceleration sensor that measures the acceleration of the drone 1 body, and a 3-axis gyro that measures the angular velocity of the drone 1 body. Various types of sensors are included, such as sensors, barometric pressure sensors that measure atmospheric pressure, and radars that detect obstacles.

  The flight drive unit 17 drives the engine, propeller, and the like to fly the drone 1 body.

  Next, the control unit 11 includes an image processing unit 101, a sound processing unit 102, a sensing signal processing unit 103, a dangerous event detection unit 104, a moving object detection unit 105, a subject detection unit 106, a moving image deletion unit 107, and The flight control unit 108 will be described.

  The image processing unit 101 processes a frame image applied to the moving image input to the moving image input unit 13 and detects a person or an object (hereinafter collectively referred to as an object) captured in the frame image. . The image processing unit 101 detects, for example, an object captured in a frame image and identifies the type of the object by known pattern matching. In the detection of an object captured in a frame image and the identification of the type of the object, the image processing unit 101 performs edge extraction processing on the frame image using, for example, a known differential filter, and is captured. Detect the outer shape of the object.

  The voice processing unit 102 processes the voice signal input to the voice input unit 14 and detects screams, screams, and the like.

  The sensing signal processing unit 103 processes sensing signals of various sensors included in the sensing unit 16, and acquires the position of the drone 1 body, the acceleration of the drone 1 body, the angular velocity of the drone 1 body, the atmospheric pressure, the presence / absence of an obstacle, and the like.

  The dangerous event detection unit 104 analyzes a change in the behavior or posture of the target person, which is one of the objects detected by the image processing unit 101, and whether the target person has a dangerous event such as snatching or violent behavior. Detect if. The dangerous event detection unit 104 corresponds to the event detection unit referred to in the present invention.

  When the dangerous event detection unit 104 detects the occurrence of a dangerous event such as a violent action by the dangerous event detecting unit 104, the moving object detecting unit 104 or the vehicle on which the actor gets on is detected. Alternatively, a moving body such as a motorcycle is detected. Specifically, the moving object detection unit 105 acts on a moving object that exists near the target person among the objects detected by the image processing unit 101 when a dangerous event occurs in the target person. It is detected as a person (including a vehicle or a motorcycle on which the actor rides).

  The target person detection unit 106 detects a target person from the objects detected by the image processing unit 101. The target person detection unit 106 specifies a target person who owns the mobile terminal 2 based on the position of the mobile terminal 2 and the position of the drone 1 main body. The drone 1 acquires the position of the mobile terminal 2 based on the position information indicating the position of the own terminal transmitted from the mobile terminal 2.

  The moving image deletion unit 107 deletes the moving image captured by the camera 6 stored in the moving image storage unit 111.

  The flight control unit 108 inputs drive control signals such as an engine and a propeller to the flight drive unit 17. In other words, the flight drive unit 17 drives the engine, the propeller, and the like in accordance with instructions from the flight control unit 108 and causes the drone 1 body to fly. The flight control unit 108 causes the main body of the drone 1 to fly by the first flight control for tracking the subject who owns the mobile terminal 2 and the second flight control for tracking the actor who does not have the mobile terminal 2. .

  The control unit 11 of the drone 1 includes a hardware CPU, a memory, and other electronic circuits. The hardware CPU includes an image processing unit 101, an audio processing unit 102, a sensing signal processing unit 103, a dangerous event detection unit 104, a moving object detection unit 105, a subject detection unit 106, a moving image deletion unit 107, and a flight control unit 108. Works as. The memory also has an area for developing a program related to the flight control of the drone 1 and an area for temporarily storing data generated when the program is executed. The control unit 11 may be an LSI in which a hardware CPU, a memory, and the like are integrated.

  FIG. 3 is a block diagram illustrating a configuration of a main part of the mobile terminal. The mobile terminal 2 includes a control unit 21, a position detection unit 22, an operation unit 23, a display unit 24, and a communication unit 25. The control unit 21 controls the operation of each part of the mobile terminal 2 main body. The position detection unit 22 includes a GPS sensor and detects the position (latitude, longitude) of the mobile terminal 2 main body. The operation unit 23 performs an input operation on the main body of the mobile terminal 2. The operation unit 23 has an emergency call button. The display unit 24 displays a screen showing the state of the main body of the mobile terminal 2 at that time, an input confirmation screen corresponding to an input operation in the operation unit 23, and the like on the display. The communication unit 25 performs data communication between the drone 1 and the management device 3.

  The control unit 21 of the mobile terminal 2 includes a hardware CPU, a memory, and other electronic circuits. The memory stores a terminal ID for identifying the mobile terminal 2 main body. The control unit 21 may be an LSI in which a hardware CPU, a memory, and the like are integrated.

  FIG. 4 is a block diagram illustrating a configuration of a main part of the management apparatus. The management device 3 includes a control unit 31, a communication unit 32, and a drone management database 33 (drone management DB 33). The control unit 31 controls the operation of each unit of the management apparatus 3 main body. The communication unit 32 performs data communication between the drone 1 and the mobile terminal 2. In the drone management DB 33, for each drone 1 to be managed, watch reservation information that associates the watch start date and time of the target person by the drone 1, the start location, the terminal ID of the mobile terminal 2, and the like is registered.

  The control unit 31 of the management device 3 includes a hardware CPU, a memory, and other electronic circuits. The control unit 31 may be an LSI in which a hardware CPU, a memory, and the like are integrated.

<3. Example of operation>
FIG. 5 is a flowchart showing the watching operation of the subject regarding the drone according to this example. The management device 3 transmits a watch preparation start to the target drone 1 based on the watch reservation information registered in the drone management DB 33. The management device 3 transmits a watch preparation start to the drone 1 at a timing at which the target drone 1 can reach the start location slightly before the watch start date and time of the target person. This watch preparation start includes the watch start date and time of the subject, the start location, and the terminal ID of the mobile terminal 2.

  When the drone 1 receives the request for the start of watching preparation transmitted from the management device 3 (s1), the drone 1 moves to the watching start place of the subject (s2). In s2, the drone 1 moves to the target watching start location of the subject by self-sustained flight.

  When the drone 1 moves to the monitoring start location of the target person, the position of the target person (in this example, the position of the mobile terminal 2) is detected (s3), and the main body of the drone 1 is moved to a position corresponding to the position of the target person. The first flight control is performed (s4). In s3, the position of the subject is detected by receiving the location information indicating the location of the terminal transmitted by the portable terminal 2 possessed by the subject. The drone 1 may receive the position information of the mobile terminal 2 directly from the mobile terminal 2 or may receive it via the management device 3. That is, the mobile terminal 2 may transmit position information indicating the position of its own terminal to the drone 1 or to the management device 3. Thus, the drone 1 detects the position of the mobile terminal 2 as the position of the subject person.

  Further, in s4, the flight control unit 108 controls the flight driving unit 17 so that the drone 1 main body is positioned, for example, just above the number m (2 to 4 m) of the detected position of the subject. Further, the flight control unit 108 controls the flight so that the main body of the drone 1 does not collide with an obstacle or the like based on the processing result obtained by processing the sensing signals of various sensors belonging to the sensing unit 16 by the sensing signal processing unit 103.

  The drone 1 starts capturing a moving image by the camera 6 with the start of the first flight control. Although not particularly illustrated, the drone 1 has a camera control unit that controls the angle and imaging magnification of the camera 6 so that the subject is within the imaging area of the camera 6 during the first flight control. The moving image captured by the camera 6 is stored in the moving image storage unit 111.

  The drone 1 repeats the processes related to s3 and s4 until it is detected that a dangerous event has occurred in the target person (s5) or until the monitoring of the target person is detected (s6).

  In s5, the dangerous event detection unit 104 analyzes a change in the behavior or posture of the subject from the moving image captured by the camera 6, and detects whether or not a dangerous event has occurred in the subject. Moreover, in s6, when the communication part 15 receives the command concerning the completion | finish of watching of the object person from the portable terminal 2 or the management apparatus 3, it determines with it being watching end.

  When it is detected that a dangerous event has occurred in the target person, the drone 1 notifies the management apparatus 3 that the dangerous event has occurred in the target person (s7). In s7, the drone 1 transmits to the management device 3 the location (latitude, longitude) where the dangerous event occurred in the target person, the time of occurrence, a moving image at the time of occurrence, and the like.

  Further, the drone 1 detects an actor who has caused a dangerous event in the target person (s8). In s8, when the moving object detection unit 105 detects that the dangerous event has occurred in the target person in the dangerous event detection unit 104, the moving object (actor or actor who exists near the target person). ) Is detected. The drone 1 starts the second flight control for tracking the moving object detected in s8 (s9). In the second flight control, the flight control unit 108 controls the flight driving unit 17 so that the moving body being tracked is positioned at the center of the frame image related to the moving image captured by the camera 6. In the second flight control, the flight control unit 108 does not collide the obstacle 1 with the obstacle based on the processing results obtained by processing the sensing signals of various sensors belonging to the sensing unit 16 by the sensing signal processing unit 103. To control the flight.

  The drone 1 repeats the processes related to s8 and s9 until it is determined that the tracking of the moving body that is the actor is completed (s10). The drone 1 determines that the tracking of the actor is completed in s10 when the management device 3 is notified of the termination of the tracking of the actor or loses sight of the actor.

  When the drone 1 determines the end of the tracking of the actor in s10, the drone 1 returns to s1.

  If the drone 1 determines that the watching is over at s6, the drone 1 deletes the moving image captured by the camera 6 in the current watching of the subject (s11), and returns to s1.

  Thus, in this example, when a dangerous event occurs in the target person, the drone 1 tracks the actor (mobile body) that caused the target person to generate the dangerous event. Therefore, it is possible to prevent an actor who has caused a dangerous event (such as snatching or violent behavior) from escaping.

  In addition, when the monitoring of the target person is terminated without causing a dangerous event, the moving image captured during the current monitoring is deleted, so that the target person captured during the monitoring of the target person is affected. Moving images are not seen by others. Therefore, it is possible to watch over the subject without infringing on the subject's privacy.

  Further, the target person may be able to notify the drone 1 that a dangerous event has occurred by operating an emergency call button provided on the operation unit 23 of the mobile terminal 2. That is, the dangerous event detection unit 104 also detects that a dangerous event has occurred in the target person even when the occurrence notification of the dangerous event is received from the mobile terminal 2. If comprised in this way, when the subject enters a dangerous state, the drone 1 detects that a dangerous event has occurred by operating an emergency call button provided on the operation unit 23 of the portable terminal 2. Can be made.

  Furthermore, the dangerous event detection unit 104 may also detect that a dangerous event has occurred in the subject when detecting a scream or cry in the sound collected by the microphone 7. If comprised in this way, when a subject will be in a dangerous state, the drone 1 can detect that the dangerous event generate | occur | produced by raising a loud voice.

  Further, when the drone 1 tracks the actor in s8 and s9, the drone 1 stores the tracking information in the tracking information storage unit 112, and periodically manages the tracking information stored in the tracking information storage unit 112. 3 may be transmitted. Thereby, on the management apparatus 3 side, the whereabouts of the actor who generated the dangerous event in the target person can be confirmed.

  In addition, when the management device 3 receives the notification of the occurrence of the dangerous event from the drone 1, the management device 3 may make a request for rescue of the target person to the fire department or the like according to the dangerous event that has occurred, or request to secure the actor You may go to a police station.

  In addition, in s7, the risk event occurrence notification notified to the management apparatus 3 includes the target person's tracking request, and the management apparatus 3 that has received this notification causes another drone 1 to track the target person. Also good. In this way, the target person temporarily stops watching by the drone 1 due to the occurrence of the dangerous event, but the watching is resumed by another drone 1. Therefore, even after a dangerous event occurs, the target person can be watched appropriately. The subject can be tracked by the first flight control described above.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

Furthermore, the correspondence between the configuration according to the present invention and the configuration according to the above-described embodiment can be described as in the following supplementary notes.
<Appendix>
In an unmanned aerial vehicle (1) equipped with an imaging device (2) for capturing a moving image and flying unattended,
A moving body detection unit (105) that detects a moving body that is captured by processing a frame image of a moving image captured by the imaging device (2);
A target person detection unit (106) for detecting the position of the target person;
A flight control unit (108) for performing first flight control for tracking the target person based on the position of the target person detected by the target person detection unit (106);
An event detector (104) that detects whether a dangerous event has occurred in the subject being tracked, and
When the flight control unit (108) detects that a dangerous event has occurred in the subject tracked by the event detection unit (104), the moving body detection unit (105) causes the imaging device (2 The unmanned aerial vehicle (1) that performs the second flight control for tracking the moving body detected by processing the frame image of the moving image captured by (1).

DESCRIPTION OF SYMBOLS 1 ... Drone 2 ... Portable terminal 3 ... Management apparatus 5 ... Network 6 ... Camera 7 ... Microphone (microphone)
DESCRIPTION OF SYMBOLS 11 ... Control unit 12 ... Memory | storage unit 13 ... Moving image input part 14 ... Audio | voice input part 15 ... Communication part 16 ... Sensing unit 17 ... Flight drive part 101 ... Image processing part 102 ... Audio | voice processing part 103 ... Sensing signal processing part 104 ... Hazardous event detection unit 105 ... moving body detection unit 106 ... subject detection unit 107 ... moving image deletion unit 108 ... flight control unit 111 ... moving image storage unit 112 ... tracking information storage unit

Claims (5)

In an unmanned aerial vehicle that is equipped with an imaging device that captures moving images and flies unattended,
A moving body detection unit that detects a moving body that is captured by processing a frame image of a moving image captured by the imaging device;
A subject detection unit for detecting the position of the subject,
A flight control unit that performs first flight control for tracking the target person based on the position of the target person detected by the target person detection unit;
An event detector that detects whether a dangerous event has occurred in the subject being tracked, and
When the flight control unit detects that a dangerous event has occurred in the subject tracked by the event detection unit, the moving body detection unit displays a frame image related to a moving image captured by the imaging device. An unmanned aerial vehicle that performs second flight control to track a moving object that has been processed and detected.   In the second flight control by the flight control unit, the unmanned aerial vehicle is configured such that the moving body is captured in a frame image of a moving image captured by the imaging device according to the movement of the moving body being tracked. The unmanned aerial vehicle according to claim 1, wherein the unmanned aerial vehicle controls flight of the main body.   The output part which outputs the tracking request | requirement of the said subject including the positional information on which the dangerous event generate | occur | produced to the said subject when the said flight control part started the said 2nd flight control was provided in Claim 1 or 2 The unmanned aircraft described. A moving image storage unit for storing a moving image captured by the imaging device;
There is a risk to the subject being tracked by the event detector between the time when the flight control unit starts tracking the subject by the first flight control and the time when the tracking of the subject is finished. The image deletion unit according to any one of claims 1 to 3, further comprising: an image deletion unit that deletes a moving image captured by the imaging apparatus during this period from the moving image storage unit if it is not detected that an event has occurred. Unmanned aerial vehicle. An imaging device that captures a moving image is mounted, and is a watching method by an unmanned aircraft that flies unattended,
A moving body detection unit detects a moving body that is captured by processing a frame image applied to a moving image captured by the imaging device,
The target person detection unit detects the position of the target person,
The flight control unit performs first flight control for tracking the target person based on the position of the target person detected by the target person detection part. Detect if it occurred,
Further, when the flight control unit detects that a dangerous event has occurred in the subject tracked by the event detection unit, the moving body detection unit takes a frame related to a moving image captured by the imaging device. A monitoring method that tracks moving objects detected by processing images.

Images