JP6172958B2 - Shooting system - Google Patents

Description

  The present invention relates to an imaging system for imaging a surveillance area using a flying robot having an imaging function, and more particularly to an imaging system for imaging a blind spot for an emergency responder arriving at a surveillance area with a flying robot.

  2. Description of the Related Art Conventionally, there has been proposed a monitoring system in which various sensors are installed in a monitoring area around a building and its surroundings, and when the sensor detects the moving robot moves to a location where an abnormality has been detected and images the monitoring area. For example, Patent Document 1 discloses a monitoring system in which a ground-traveling mobile robot moves to a place where an abnormality occurs and images an abnormal situation based on signals from a sensor that detects a fire or a sensor that detects an intruder. It is disclosed. Such a monitoring system may monitor a wide monitoring area such as a factory or a shopping mall where a parking lot is provided. In the system of Patent Document 1, the mobile robot travels to a place where the sensor detects an abnormality, and performs imaging of the place where the abnormality has occurred, notification to the monitoring center, and the like.

JP 2009-181270 A

  By the way, when an abnormality occurs in the monitoring area, there is a form of a monitoring service in which an emergency coordinator is rushed to the monitoring area from the supervisor of the monitoring center that has received the report and an appropriate countermeasure is taken.

In such a monitoring service, in particular, emergency if you want to monitor more monitoring area of the blind spot for the response personnel, the emergency from the response personnel to take a place that can not be seen in the shadow of the building at the mobile robot, emergency image By presenting it to the counselor, more effective responses can be made.

  However, in the monitoring system described in Patent Document 1, the mobile robot remains in the role of quickly photographing the place where the abnormality has occurred and presenting the image to the monitoring center. The shooting location was not determined in consideration of the position. For this reason, it is not easy to capture the blind spots of emergency response personnel who have arrived in the surveillance area with a mobile robot.

  Accordingly, an object of the present invention is to realize an imaging system that captures a blind spot for an emergency response person who has arrived at a monitoring area with a mobile robot and presents the image to the emergency response person.

In order to achieve such an object, the present invention provides a flying robot having a photographing function, a portable terminal having a display for displaying an image photographed by the flying robot, and a control for controlling the flight of the flying robot to a target position. The control device includes a storage unit for storing a monitoring space map representing a monitoring area including an obstacle such as a building, and a current position of the mobile terminal on the monitoring space map. Using the monitoring space map and the position of the counselor to extract the position of the counselor as a position, an invisible area that is physically behind the obstacles and is physically invisible is extracted from the monitoring space map. An invisible area extracting means for performing imaging, and a target position setting means for setting a position where the flying robot can image a predetermined position in the invisible area in the monitoring space map as a target position. To provide a system.

  As a result, the present invention enables the emergency response personnel who have arrived at the monitoring area to monitor the monitoring area while confirming the state of the monitoring area that is a blind spot for themselves.

  It is preferable that the counselor position calculating unit sequentially calculates the counselor position, the invisible region extracting unit sequentially extracts the invisible region, and the target position setting unit sequentially sets the target position.

  As a result, the present invention makes it possible to confirm the state of the blind spot monitoring area that changes with the movement even when the emergency responder monitors while moving in the monitoring area.

  The monitoring space map includes information on the opening position of the opening where a person can enter and leave the building, and the target position setting means sets the opening position in the invisible region as a target position where the flying robot can shoot. Is preferred.

  As a result, the present invention makes it possible to pinch and shoot a bandit trying to escape from the building with the emergency response staff and the flying robot when the bandit is located in the building.

  In addition, when there are a plurality of opening positions in the invisible region, the target position setting means preferably sets a position where the flying robot can photograph the most opening positions as the target position.

  Further, the target position setting means preferably sets the position farthest from the coordinator position as the target position when there are a plurality of positions that satisfy the target position condition.

  Thereby, even when monitoring a building with a large area, the present invention makes it possible to efficiently pinch and shoot a bandit trying to escape from the building with an emergency response person and a flying robot.

  According to the present invention, it is possible to monitor the monitoring area while confirming the state of the monitoring area where the emergency agent who arrives at the monitoring area becomes a blind spot for himself.

Overall configuration diagram of the shooting system Functional block diagram of a flying robot Functional block diagram of the operator terminal Diagram explaining the image of the operator terminal Functional block diagram of security equipment Functional block diagram of the robot control module Diagram showing the detection area of the laser sensor Figure showing an extraction image of the invisible region The figure explaining the flight image of the flying robot after sensor detection Illustration explaining the flight image of the flying robot when the emergency response staff arrives Flow of flight control processing when photographing the detection location of the sensor Flow of flight control processing when shooting blind spots for emergency responders

  Hereinafter, an embodiment of a photographing system according to the present invention will be described.

  FIG. 1 is a diagram schematically showing the overall configuration of the photographing system 1. The imaging system 1 is a center installed in a monitoring center connected to a security device 2, a flying robot 3, a laser sensor 4, an in-building sensor 5, and a counselor terminal 6 installed in a monitoring area E via a network. The apparatus 7 is configured. The center device 7 is connected to the security device 2 via the IP network, receives an image captured by the flying robot 3 from the security device 2, a detection signal from the sensor 5 in the building, and the like, and displays it on the monitor. The monitor looks at this monitor and grasps the status of the monitoring area E and takes an appropriate response. Although the network is described as an IP network, the network is not limited to this as long as there is no problem in image transmission / reception such as a general public line network or a mobile phone network.

  The flying robot 3 flies while photographing based on a wireless flight control signal from the security device 2, and transmits the photographed image to the security device 2. FIG. 2 is a diagram showing functional blocks of the flying robot 3.

  The flying robot 3 includes an antenna 31 for performing wireless communication with the security device 2, four rotors 32 for flying such as ascending / descending / turning direction / advancing, a motor for providing driving force to the rotor 32, and the like. The rotor drive unit 33, an altitude sensor 34 that measures the current altitude of the flying robot 3 by projecting and receiving laser light vertically below, and a measurement that measures the surrounding situation of the flying robot 3 by projecting and receiving laser light horizontally and around it. A distance sensor 35, a camera 36 that captures a color image of the front of the flying robot 3, an illumination 37 that is an LED illumination that lights when the surroundings are dark and assists the camera 36, and a robot that controls the entire flying robot 3. The power supply 39 which is a lithium polymer battery for supplying power to each unit of the control unit 38 and the flying robot 3 is configured.

  In addition, the robot controller 38 is a robot communication control unit 381 that controls wireless communication with the security device 2 via the antenna 31, starts / ends shooting of the camera 36, and acquires images captured by the camera 36 to control robot communication. The communication control means 381 using the camera control means 382 for performing processing such as transmission from the means 381 to the security device 2, the altitude information measured by the distance measuring sensor 35 and the altitude sensor 34, and the distance data between the surrounding object and the own device as scan data. Flight control for controlling the rotor driving unit 33 to fly to the target position by controlling the rotor drive unit 33 based on the flight control signal from the security device 2. The unit 384 is constituted.

  Next, the agent terminal 6 will be described. The coordinator terminal 6 is a portable terminal possessed by an emergency coordinator who has received a request for dispatch to the monitoring area E from the monitoring center. The coordinator terminal 6 receives information such as an image taken by the flying robot 3 by wireless communication from the security device 2 and the current position (flying position) of the flying robot 3. The coordinator terminal 6 transmits the current position information of the terminal to the security device 2. FIG. 3 is a diagram illustrating functional blocks of the coordinator terminal 6. FIG. 4 is a diagram showing an image of the coordinator terminal 6.

  The counselor terminal 6 includes a communication unit 61 including a GPS receiving unit 611 that receives a GPS signal from a GPS satellite (not shown) and a security device communication unit 612 for performing wireless communication with the security device 2, A terminal control unit 62 that controls the terminal 6, a terminal storage unit 63 composed of peripheral components such as a ROM / RAM storing programs and various data and parameters necessary for processing of the coordinator terminal 6, and a liquid crystal display The display unit 64 is an information display device such as an information display device, and the operation unit 65 is an information input device such as a keyboard or a touch panel. Although not shown, a power supply unit that supplies power to each unit is also provided.

  Here, information stored in the terminal storage unit 63 will be described. The monitoring space map 631 is information that represents the monitoring area E in three dimensions, and is map information that represents the monitoring space from the ground to a height required for the flight of the flying robot 3. The monitoring space map 631 includes three-dimensional information of buildings in the monitoring area E in advance, three-dimensional information of openings such as windows and doors that allow people to enter and exit the building, and their coordinates (opening positions), and information in the monitoring area E. 3D information of an object existing in the monitoring area E, such as the position of the laser sensor 4 installed in is stored. A monitoring space map 241 is also stored in the storage unit 24 of the security device 2 described later. The monitoring space map 631 stored in the terminal storage unit 63 and the monitoring space map 241 stored in the storage unit 24 may be the same information, or include information necessary for processing executed using each of them. If there is a difference, the information amount may be different. The monitoring space map 241 will be described later. The terminal storage unit 63 stores various programs other than the monitoring space map 631 for realizing the function of the agent terminal 6.

  Next, the terminal control unit 62 will be described. The terminal control unit 62 reads out a software module (not shown) in the terminal storage unit 63 and performs each process by the CPU or the like.

  The terminal position calculation means 621 calculates the current position information (latitude / longitude / altitude information) of the coordinator terminal 6 from the GPS signal received by the GPS reception means 611 and calculates it to the security device 2 via the security device communication means 612. The current location information is transmitted as the terminal location. Here, the terminal position calculation unit 621 calculates the terminal position for the GPS signal received by the GPS reception unit 611 at a predetermined interval (for example, every several tens of milliseconds) while the power of the coordinator terminal 6 is ON. And transmitted to the security device 2. In the present embodiment, the calculation of the terminal position using the GPS signal is described, but the present invention is not limited to this. If it is possible to specify the current position information of the coordinator terminal 6, the terminal position may be calculated using another radio positioning method, IC tag, image processing technology, or the like.

  When the display / operation control unit 622 receives the information (robo information) about the flying robot 3 including the image taken by the flying robot 3 and the flight position of the flying robot transmitted from the security device 2 at the security device communication unit 612. The robot information is displayed on the display unit 64 using the monitoring space map 631 and the like stored in the terminal storage unit 63, so that the emergency agent who is the user of the agent terminal 6 can grasp the robot information. In addition, when the coordinator position transmitted from the security device 2 is received, the coordinator position is displayed on the display unit 64 using the monitoring space map 631 stored in the terminal storage unit 63. In addition, various kinds of information necessary for the emergency agent to operate the agent terminal 6 are displayed on the display unit 64. The display / operation control unit 622 controls the coordinator terminal 6 based on an input signal from the operation unit 65. For example, the emergency response person operates the operation unit 65 to start / end the response person terminal 6, enlarge / reduce, and rotate the image displayed on the display unit 64.

  Here, an example of the display of the coordinator terminal 6 will be described with reference to FIG. FIG. 4 shows a display example of the display unit 64 when the emergency responder who has arrived at the monitoring area E monitors the outer periphery of the building in the monitoring area E and photographs the blind spot for the emergency responder with the flying robot 3. Is shown. The display unit 64 displays a structural diagram 642 that allows the bird's-eye view of the entire monitoring area E generated using the monitoring space map 631 and a current captured image 641 captured by the flying robot 3. Here, the captured image 641 is an image of a door installed on the side of the building located on the side opposite to the side of the building currently monitored by the emergency response staff. This door is normally a blind spot of the building from the current position of the emergency response person and cannot be seen, but the emergency response person confirms the captured image 641 displayed on the display unit 64 to see the state. Can be monitored.

  The display unit 64 also displays on the structure diagram 642 a coordinator icon 644 indicating the current position of the coordinator terminal 6 (the current position of the emergency coordinator) and a flying robot icon 643 indicating the current position of the flying robot 3. ing. Although not shown, the shooting direction of the flying robot 3 may be displayed. In this way, when the emergency response person looks at the display unit 64 of the response person terminal 6, the self position in the monitoring area E, the flight position of the flying robot 3 (the shooting position), and the state of the monitoring area that becomes a blind spot for itself. Can be grasped instantly. Note that the display shown in FIG. 4 is an example, and the display mode on the display unit 64 is not limited to this.

  Next, the security device 2 will be described in detail with reference to FIGS. 5, 6, 7, 8, 9, and 10. The security device 2 is installed inside the building B in the monitoring area E shown in FIG. The security device 2 includes a sensor 5 in the building installed at an appropriate location for detecting an intruder into the building B, a laser in the monitoring area E and outside the building B such as a parking lot. Each sensor 4 is connected.

  FIG. 5 is a diagram illustrating functional blocks of the security device 2. The security device 2 includes a security mode switching unit 21 that performs a switching operation between a security set state monitored by the monitoring center and a security release state that is not monitored by the monitoring center, and various types of sensors such as the laser sensor 4 and the in-building sensor 5. A sensor interface 22 that receives signals from the sensors, a flying robot communication unit 25 that communicates with the flying robot 3, images taken by the flying robot 3, abnormal signals detected by various sensors, etc. A ROM / ROM that stores programs, various data, parameters, and the like necessary for processing of the security device 2, a monitoring center communication unit 26 that performs communication via the communication center, a coordinator terminal communication unit 27 that performs communication with the coordinator terminal 6 Consists of a storage unit 24 composed of peripheral components such as a RAM, and a CPU, MPU, etc. for overall control of the security device 2 And a Bei controller 23.

  Here, information stored in the storage unit 24 will be described. First, the monitoring space map 241 is information representing the monitoring area E in three dimensions as described above in the description of the monitoring space map 631 stored in the terminal storage unit 63 of the coordinator terminal 6, and is from the ground. This is map information representing a monitoring space up to a height required for the flight of the flying robot 3. Referring to FIG. 9, in the present embodiment, the presence of a fence separating the monitoring area E from the outside, the building B, the installation position of the laser sensor 4, and an opening (such as a door or window) through which people can enter and exit ( The position (opening position) of the door a, the window e, the window f, and the window g) and the three-dimensional information inside the building B are registered. For example, the opening position of the door a on the monitoring space map 241 is (x1, y1, z1), the opening position of the window e is (x2, y2, z2), the opening position of the window f is (x3, y3, z3), The opening position of the window g is stored as (x4, y4, z4). In the present embodiment, the monitoring space map 241 and the monitoring space map 631 are the same information.

  The in-building sensor arrangement information 242 is position information in the monitoring space map 241 of the monitoring location of each in-building sensor 5. This is determined in advance by a security plan, and a position on the monitoring space map 241 is associated with each in-building sensor 5. The in-building sensor arrangement information 242 is information in which the in-building sensor 5 is associated with sensor attributes, a spatial region indicating a detection target, and the three-dimensional coordinates of the nearest opening position. The sensor attribute indicates the function of the in-building sensor 5, “door” is an attribute that the opening / closing of the door is detected by a magnet sensor, and “space sensor” is a thermal sensor that detects the heat generated by the human body in the area. The attribute “window sensor” for detecting the presence of a human body in a predetermined space such as an image sensor indicates an attribute for detecting opening / closing of a window or detecting destruction of a window. The space area indicates the monitoring block targeted by the in-building sensor 5. The opening position is used for calculating the target position of the flying robot 3 as a place where a bandit is likely to come out of the building B when the in-building sensor 5 detects it. With reference to FIG. 9, an example of the sensor arrangement information 242 in the building will be described. For example, the sensor 5a is a sensor that monitors the opening and closing of the door, and the intrusion into the monitoring block B1 is targeted for detection. The opening position (x1, y1, z1) on the monitoring space map 241 is associated with the three-dimensional coordinates of the nearest opening position.

  The laser sensor parameter 243 is information including the correspondence between the position of the laser sensor 4 in the monitoring space map 241 and the position in the detection area of the laser sensor 4 and the position on the monitoring space map 241. It is a parameter for converting the determined position into a position on the monitoring space map 241.

  The various parameters 244 are various parameters such as the IP address of the center device 7 necessary for the security device 2 to monitor the monitoring area E and data for communication with the flying robot 3. In addition to these, the storage unit 24 stores various programs for realizing the functions of the security device 2.

  Next, the security control unit 23 will be described in detail. The security control unit 23 reads a software module (not shown) in the storage unit 24 and performs each process by the CPU or the like.

The laser sensor analysis module 231 is software for analyzing the signal of the laser sensor 4 input from the sensor interface 22. Specifically, the search signal which is the result of the laser sensor 4 scanning the detection area with laser light is analyzed in time series. If there is no new approaching object or the like in the detection area, the search signal of the laser sensor 4 input in time series does not change, and the analysis result indicates that there is no moving object. On the other hand, if there is a new approaching object or the like in the detection area, the search signal of the laser sensor 4 changes, and the position in the detection area where the change has occurred is obtained by analysis. Further, using the laser sensor parameter 243 of the storage unit 24, the position is converted into a position on the monitoring space map 241 to calculate the position / size / movement direction of the approaching object, and the approaching object is tracked on the monitoring space map 241. . Further, when the approaching object stops, there is no change in the signal thereafter, so it can be determined that the object such as the automobile being tracked is parked.
The analysis result of the laser sensor analysis module 231 is output to an abnormality determination module 232 and a robot control module 233 described later.

  Here, the laser sensor 4 is installed outdoors, and monitors the approach to the parking lot of the monitoring area E and the surroundings of the building B. FIG. 7 is a diagram showing a detection area of the laser sensor 4. As shown in the figure, the laser sensor 4-1 is installed from the upper left of the monitoring area E as the detection area in the direction of the building B, and the laser sensor 4-2 is detected from the lower right of the monitoring area E in the direction of the building B. It is installed so that.

  The laser sensor 4 transmits a search signal that is a laser beam in a radial manner so as to scan a preset detection area, and receives the search signal that is reflected back to the object in the detection area. Then, the distance to the object is calculated from the time difference between transmission and reception, and the direction in which the search signal is transmitted and the calculated distance are obtained.

  And the laser sensor 4 transmits the result of the scanning unit which scanned the detection area with the predetermined period to the security apparatus 2. Thereby, the laser sensor analysis module 231 of the security device 2 can perform the object placement situation in the monitoring area E, the presence / absence of a person, the tracking of a car, and the like. In this embodiment, since the purpose is to monitor the approach of a vehicle or a person traveling on the ground, scanning is performed in one step in the horizontal direction. However, depending on the purpose of monitoring, scanning in a plurality of steps may be performed in the vertical direction. It may be.

  Returning to FIG. 5, the abnormality determination module 232 receives a security set / release signal from the security mode switching unit 21, a signal from the building sensor 5, and the laser sensor 4, and whether or not an abnormality has occurred in the monitoring area E. Determine whether. When receiving the security set signal from the security mode switching unit 21, the abnormality determination module 232 sets the monitoring area E to the security set mode that warns, and when receiving the security release signal, sets the monitoring area E to the security release mode that is not warning. In the security release mode, no special processing is performed even if a detection signal from the in-building sensor 5 or the laser sensor 4 is received. On the other hand, in the security set mode, when a detection signal from the in-building sensor 5 or the laser sensor 4 is received, it is determined that an abnormality has occurred, and the monitoring center communication unit 26 notifies the center device 6 of the abnormality. Along with the abnormality report, activation control of the flying robot 3 is executed for the robot control module 233. And the process which transmits the image which the flying robot 3 received from the flying robot communication part 25 image | photographed from the monitoring center communication part 26 to the center apparatus 6 is continued until an abnormal state is cancelled | released. Although there are various methods for canceling the abnormal state, the description thereof is omitted because it is not relevant to the present invention.

  When the robot control module 233 receives the activation signal of the flying robot 3 from the abnormality determination module 232, the robot control module 233 performs flight control of the flying robot 3 from the flying robot communication unit 25.

  Here, the robot control module 233 will be described with reference to FIG. FIG. 6 is a functional block diagram of the robot control module 233. The robot control module 233 has a monitoring space for an invisible area, which is an area that is not visible to the counselor, and cooperator position calculating means i for sequentially calculating the counselor position that is the current position of the counselor on the monitoring space map 241. The invisible area extracting means B extracted from the map 241; the target position setting means C for determining the target position that the flying robot 3 should reach; and the flight path for reaching the target position set by the target position setting means C A flight path calculation means D for performing, a robot control means E for generating and transmitting a flight control signal to the flying robot 3 so as to fly along the flight path calculated by the flight path calculation means D, and a monitoring space map of the flying robot 3 It is comprised from the flight position calculation means which calculates the flight position which is the current position information on 241 sequentially. Details of the target position setting means C will be described later.

  The counselor position calculation means a calculates the current position in the monitoring area E of the emergency coordinator who possesses the counselor terminal 6. Specifically, for the terminal positions (latitude / longitude / altitude information) sequentially received from the counselor terminal communication unit 27, the position on the monitoring space map 241 is sequentially calculated as the counselor position, and the invisible area extracting means b sequentially. Output. Here, when the coping person position is calculated by the coping person position calculating means (a), it means that the emergency coping person has arrived at the monitoring area E. In addition, when the coping person terminal is not calculated, it means that the emergency coping person has not arrived at the monitoring area E or the emergency coping person who has once arrived in the monitoring area E has gone out of the monitoring area E. .

The invisible area extraction means B uses the cooperator position calculated by the cooperator position calculation means a and the monitoring space map 241 so that the blind area for the emergency cooperator in the monitoring area E, that is, the shadow behind the building, can be visually checked. Extract impossible areas as invisible areas. Here, an example of a specific extraction method will be described with reference to FIG. FIG. 8 is a diagram showing an extraction image of the invisible region, and corresponds to FIG. FIG. 8 is a view of the monitoring area E viewed from above. When the counselor position calculated by the coordinator position calculation means (a) is 8, the invisible area extracted using the monitoring space map 241 is the area D shown in black. Assuming that the emergency responder can view 360 degrees centering on the three-dimensional coordinates of the responder position, the invisible area includes the three-dimensional information of the building B included in the monitoring space map 241 and the presence of a flaw. In consideration of physical constraints, an area beyond the obstacle in the direction in which an obstacle such as a building exists is extracted as an invisible area, that is, an invisible area. In FIG. 8, the area beyond the wall surface of the building 8 in the vicinity of the emergency agent is extracted as the invisible region D with the agent position 8 as the center. Note that the invisible area extraction processing by the invisible area extracting means b is not limited to this method.

  The flight position calculation means receives the scan data acquired by the scanning means 383 of the flying robot 3 and calculates a flight position by calculating a place where the scan data matches the monitoring space map 241. Since the flying robot 3 is located at a predetermined landing standby place until the start of activation, that position is the flying position. In the present embodiment, the current position is calculated based on the scan data. However, the present invention is not limited to this, and the flying robot 3 is provided with a GPS signal receiving function, and the current position is calculated based on the GPS signal. Also good.

  Returning to FIG. 5, the image processing module 234 processes the image captured by the flying robot 3 received from the flying robot communication unit 25. The image processing module 234 determines whether a person is shown in the image captured by the flying robot 3. That is, an image captured by the flying robot 3 is captured, and the presence / absence of a person is determined using a learning discriminator that learns images of various persons from the captured image. For example, a window frame that forms an opening is recognized, and a learning discriminator that has learned the silhouette of the face and shoulder is applied to an image shown in the window frame. That is, if there is an image having a texture similar to the silhouette of the face and shoulder in the window frame, it is determined that there is a person.

  The coordinator terminal communication unit 27 sequentially transmits the image taken by the flying robot 3 received from the flying robot communication unit 25 and the flight position calculated by the flight position calculation means to the counselor terminal 6 as robot information. The coordinator position calculated in the coordinator position calculating means (a) is transmitted to the coordinator terminal 6. The coordinator position may be calculated by the counselor terminal 6 instead of being transmitted from the coordinator terminal communication unit 27.

  Next, returning to FIG. 6, the details of the target position setting means C of the robot control module 233 will be described. The target position setting means C switches the target position setting mode using the cooperator position calculated by the cooperator position calculating means a and the analysis result of the laser sensor 4. The target position setting means C changes the target position setting process in accordance with the target position setting mode. The setting mode includes a single monitoring mode in which the target position is set based on the detection results of the laser sensor 4 and the in-building sensor 5, and a blind spot monitoring mode in which the target position is set based on the extracted invisible region. The single monitoring mode is set when the coping worker position is not calculated. The blind spot monitoring mode is set when the coping person position is calculated. Here, even when the coping worker position is calculated, when a bandit appears from the building B, the blind spot monitoring mode may be switched to the single monitoring mode. In this case, it is only necessary to switch from the blind spot monitoring mode to the single monitoring mode when the laser sensor 4 detects and the detected position is different from the coordinator position and the flight position. In addition, the coping person may operate the operation unit of the coping person terminal 6 to switch the mode. In this case, the security device 2 may receive a mode transition signal from the coordinator terminal 6.

  The setting of the target position in the single monitoring mode will be described. The target position setting means C automatically sets the target position of the flying robot 3 based on the analysis result of the laser sensor 4 and the detection result of the in-building sensor 5. For example, when a laser is detected by the laser sensor analysis module 231, an altitude of about 5 m above the position on the vehicle monitoring space map 241 that is the approaching object calculated by the laser sensor analysis module 231 is set as the target position. Here, about 5 m is a height that allows the flying robot 3 to photograph the entire automobile. Further, when the in-building sensor 5 detects an intruder, the target position setting means C refers to the in-building sensor arrangement information 242 and sets a position where the corresponding opening position can be photographed as the target position. For example, when the in-building sensor 5f detects the opening position (x3, y3, z3) in consideration of obstacles such as a wall and a wall of the building B by the monitoring space map 241, the position where the front can be photographed as much as possible is set as the target position. And

The setting of the target position in the blind spot monitoring mode will be described. The target position setting means C determines the target position based on the invisible area extracted by the invisible area extracting means b. Specifically, the target position setting means C sets a position where the position in the invisible region on the monitoring space map 241 can be photographed as the target position. Since the invisible region changes according to the position of the counselor, the flying robot 3 performs the flight while changing the target position according to the cooperator position calculated sequentially. The target position is set, for example, by sequentially setting a position where a position within the invisible region and a predetermined distance from the wall surface of the building B (for example, a width of 1 m and a height of 1 m) can be photographed as a target position. You may make it image | photograph while moving. At this time, the direction of the building B may be photographed, or the traveling direction of the flying robot 3 may be photographed. Further, the target position may be set so that a predetermined ratio or more (for example, 50% or more) of the invisible region is included in the captured image in consideration of the angle of view of the camera 36 mounted on the flying robot 3. . Alternatively, the target position may be sequentially set so that the invisible region is included in a predetermined ratio or more (for example, 80% or more) of the captured images, and the image may be taken while moving. The method is not limited to these methods as long as the target position is set to a position where the extracted position in the invisible region can be photographed. In addition, the target position is not automatically set sequentially, but an invisible region is extracted by using the counselor position at the time of receiving the request signal based on a request signal from the outside of the coordinator terminal 6 or the like, and based on the invisible region The target position may be set. Thus, from the position of the emergency response personnel can shoot an invisible region in the shadow of a building B.

Further, the target position setting means C may determine the target position based on the invisible area extracted by the invisible area extracting means B and the information on the opening position included in the monitoring space map 241. In the present embodiment, this determination method will be described in detail. Specifically, a position where an opening position existing in the extracted invisible region can be photographed is set as a target position. Referring to FIG. 8, the building B shows four openings, a door a, a window e, a window f, and a window g. As described above in the description of the monitoring space map 241, the monitoring space map 241 includes Includes the position coordinates of each opening position. Since there are two openings in the invisible region D in FIG. 8, the door a and the window g, the flying robot 3 first sets a position where the opening position of the door a can be photographed as a target position, for a predetermined time. When shooting, the position where both the door a and the window g can be shot is set as a target position, and when shooting for a predetermined time, the opening position of the window g is set as the target position, and when shooting for a predetermined time, A position where both the door a and the window g can be photographed is set as a target position. By thus taking open position located within the invisible region D, from the current position 8 of the emergency response personnel can shoot an invisible region in the shadow of a building B. In particular, the opening position is a position where the bandit is likely to escape from inside the building, so the bandit is struck by the emergency responder 8 and the flying robot 3 to ensure that the bandit intends to escape. Can be monitored. When there is only one opening position located in the invisible region D, it is not necessary to move the opening position from a position where photographing is possible. When there are a plurality of opening positions in the invisible region, there are a plurality of positions that satisfy the target position condition. In this case, a position farthest from the coordinator position may be set as the target position. Specifically, for each of the positions where the opening position can be photographed, the distance to the cooperator position is calculated, and the position where the calculated distance is the longest is set as the target position. Here, the calculated distance may be a moving distance considering an obstacle included in the monitoring space map 241. As a result, the flying robot 3 preferentially monitors the region that takes time until the emergency response person moves, so that a bandit that escapes from the building can be pinched and shot.

  As another determination method, when there are a plurality of opening positions in the invisible region, the position where the largest number of opening positions can be photographed at one time is considered in consideration of the angle of view of the camera 36 mounted on the flying robot 3. You may make it set as a position. Referring to FIG. 8, since there are two opening positions of the door a and the window g in the invisible region D, a position where both the door a and the window g can be photographed at a time is set as a target position. This means that the most opening positions are set at positions where photographing can be performed. In FIG. 8, it is possible to photograph both the door a and the window g at a time by photographing so as to face the diagonal direction of the monitoring area E from the position of the upper right corner of the monitoring area E. Thereby, even if it is a building where many openings exist, it is possible to pinch and shoot a bandit trying to escape from the building more efficiently. In addition, when there are a plurality of positions at which the most opening positions can be photographed at once, a position farthest from the coping staff position may be set as the target position. In this case, it is only necessary to calculate the distance from the coordinator position for each of the positions where the largest number of opening positions can be photographed at once, and set the position where the calculated distance is the longest as the target position. Here, the calculated distance may be a moving distance considering an obstacle included in the monitoring space map 241. Thereby, the flying robot 3 preferentially monitors the blind spot area that takes time until the emergency response person moves, so that a bandit that escapes from the building can be pinched and shot.

  Next, an operation image of the imaging system 1 configured as described above will be described with reference to FIGS. 4, 8, 9, and 10. FIG. 9 shows a state when the bandits enter the building B after entering the parking lot in the monitoring area E using the automobile 9 during the security set mode. Since the emergency responder has not arrived at the time of FIG. 9, the target position setting mode is set to the single monitoring mode. First, as shown in FIG. 9A, when the automobile 9 enters, the security device 2 detects an abnormality based on a signal from the laser sensor 4. The security device 2 notifies the center device 7 of an abnormality when the abnormality occurs and starts controlling the flying robot 3. In addition, the bandits enter the building B through the window where the sensor 5f is installed after getting out of the automobile 9, and the sensor 5f detects at this timing. Here, since the flying robot 3 is controlled in the autonomous flight mode, the flying robot 3 flies from the landing standby place to the position of the automobile 9 and images the automobile 9 at a high altitude position above the automobile 9. Thereafter, as shown in FIG. 9 (b), the place where the bandits entered is photographed based on the detection result of the in-building sensor 5f.

  FIG. 10 shows a situation in which an emergency response staff 8 possessing a response staff terminal 6 that has rushed in response to an instruction to handle the monitoring area E from a monitoring center that has received an abnormality report appears in the monitoring area E. First, as shown in FIG. 10A, when the emergency response person 8 arrives in the monitoring area E, the security device 2 calculates the position of the response person based on the terminal position of the response person terminal 6. The security device 2 sets the target position setting mode to the blind spot monitoring mode with the arrival of the emergency responder 8. After that, as shown in FIG. 10B, the emergency agent moves to the vicinity of the window f and the window e and performs monitoring. The security device 2 extracts an invisible region as shown in FIG. 8 based on the calculated coordinator position, and can photograph the opening position of the door a which is the farthest position from the coordinator position among the opening positions in the invisible area D. Set the target position at the correct position. As a result, the flying robot 3 moves to the set target position. As shown in FIG. 4, an image of the door a that is a blind spot for the emergency response staff 8, the flight position of the flying robot 3, and the position of the response staff are displayed on the response staff terminal 6. The emergency responder 8 can simultaneously monitor the position where the building B becomes a blind spot by checking the displayed information. Moreover, since the emergency response person 8 and the flying robot 3 can monitor the opening part where the bandit which invaded in the building B may escape, it becomes possible to pinch and shoot the bandit.

  Next, processing for controlling the flying robot 3 by the security device 2 will be described with reference to FIGS. 11 and 12. FIG. 11 is a control flow when the target position setting means C is set to the single monitoring mode. FIG. 12 is a control flow when the target position setting means C is set to the blind spot monitoring mode. Although not shown in this flow, when the security device 2 detects an abnormality in the security device 2 based on the detection signal of the laser sensor 4 or the in-building sensor 5, it activates the flying robot 3 and sets the setting mode. Set to single monitoring mode. Further, when the abnormal state is cleared, the security device 2 sets the coordinates of the landing standby location stored in advance as the target position, and the flight route calculation unit d sets the landing standby location to the landing standby location. The flight control means E calculates the flight control signal of the flight robot 3 so that the flight robot 3 can fly the route calculated by the flight path calculation means D, and causes the flight robot 3 to land and fly. When the flying robot arrives at the landing standby place, the robot control means ho stops the flying robot 3.

  With reference to FIG. 11, the process for controlling the flying robot 3 when the target position setting means C is set to the single monitoring mode will be described. The flow in FIG. 11 starts when the single monitoring mode is set. As described above, the condition set in the single monitoring mode is switched when the coping worker position is not calculated, and when the laser sensor 4 detects and the detected position is different from the coping member position and the flight position. Although not shown, immediately after the single monitoring mode is set, a standby flight is performed on the spot and the process proceeds to S100. First, the detection result of the laser sensor 4 is acquired in S100. Here, for example, the target position setting means C displays the position of a high altitude (for example, an altitude of 5 m) 3.5 m away from the current position that is the center of gravity position of the automobile 9 analyzed by the laser sensor analysis module 231 on the monitoring space map 241. Is set to the target position. (S100, S101)

  Next, the laser sensor analysis module 231 tracks the position of the automobile 9 based on the signal from the laser sensor 4. (S102)

  Then, the flight path calculation means D calculates the flight path by the path search algorithm using the target position set in S101, the flight position of the flying robot 3, and the monitoring space map 241. If a flight position and a target position are set, the route search algorithm calculates a route that can be safely and shortly arrived in consideration of the arrangement state of the monitoring space map 241 and the size of the flying robot 3. (S103)

  As a result of the tracking, for example, it is determined whether or not the automobile 9 is moving. If the automobile 9 that is the tracking target is moving, the target position is changed and set in accordance with the movement of the automobile 9, and S204. Similarly, the flight path is calculated again. (S104, S105)

  When the automobile 9 is not moving, the robot controller ho calculates the flight control signal of the flying robot 3 so that the flying robot 3 can fly along the route calculated by the flight route calculator d. A specific flight control signal is the number of rotations of each of the four rotors 32 in the flying robot 3. Then, a flight control signal is transmitted from the flying robot communication unit 25 as a radio signal. When the flying robot 3 receives the flight control signal from the antenna 31, the flying robot 3 flies based on the received flight control signal. Specifically, the flight control signal received from the antenna 31 is input to the flight control means 384, and the number of rotations of each rotor 32 is individually controlled from the rotor drive unit 33 to fly. (S106)

  After the flight is started, these S104 to S106 are repeatedly processed, and shooting is performed while tracking the automobile 9. When the flying robot 3 arrives at the target position, the robot control means ho calculates a flight control signal of the flying robot 3 and transmits it to the flying robot 3 so that the flying robot 3 can perform standby flight at the target position. When the flight robot 3 receives the flight control signal related to the standby flight from the antenna 31, the flying robot 3 performs the standby flight based on the received flight control signal. (S107, S108)

  Here, although not described in the flow shown in FIG. 11, when the flying robot 3 first receives the flight control signal, the camera control means 382 activates the camera 36 and transmits the captured image to the security device 2. Start. Further, the scanning unit 383 activates the distance measuring sensor 35 and the altitude sensor 34 and starts transmitting scan data to the security device 2.

  In S109, it is confirmed whether a new sensor has been detected. If no new sensor has been detected, the standby flight in S108 is continued. If a new sensor is detected, the detection information is acquired in S100, a target position is set based on the detection information, and S102 to S107 are executed for the set target position. For example, a case where the newly detected sensor is the in-building sensor 5f will be described as an example. A position where the opening position corresponding to the in-building sensor 5f detected from the in-building sensor arrangement information 242 can be photographed is set as the target position. To do. In this case, the target position is set to a position on the monitoring space map 241 where the altitude is “z5” about 3 m before the opening position. These values are merely examples, and are set in consideration of the constraints on the monitoring space map 241 such as the performance of the camera 36, the mounting parameters, and the distance from the bag. In addition, since the opening position does not move like an automobile, the processing related to tracking in S102, S104, and S105 may not be executed. (S109)

  Further, a new sensor may be detected when the image processing module 234 recognizes that a bandit has been photographed in S109. Although not described in detail, in this case, as described in the example of tracking the automobile 9, the position of the bandit captured by the laser sensor analysis module 231 is set as the target position in S100 and S101, and the bandage is detected in S102. What is necessary is just to make it track as a tracking object. In this case, when the arrival at the target position in S107 and the shooting of the bandit is completed, it is possible to move to a position where the bandit is taken in a bird's-eye view and fly in standby in S108.

  In the present embodiment, the flying robot 3 is controlled by the security control unit 23 of the security device 2, but all or part of the functions of the security device 2 may be appropriately mounted on the flying robot 3. Good.

  The flow of FIG. 11 ends when the setting mode of the target position setting means C shifts to the blind spot monitoring mode, and after the shift to the blind spot monitoring mode, the process is performed according to the flow of FIG.

  Next, a process for controlling the flying robot 3 when the target position setting means C is set to the blind spot monitoring mode will be described with reference to FIG. The flow in FIG. 12 starts when the blind spot monitoring mode is set. As described above, the condition set in the single monitoring mode is switched when the coping person position is calculated and when the detection position of the laser sensor 4 is the same as the coping person position and the flight position. Although not shown, immediately after the single monitoring mode is set, a standby flight is performed on the spot, and the process proceeds to S200. First, the target position setting means C extracts an invisible region from the monitoring space map 241 based on the cooperator position calculated by the cooperator position calculating means a. (S200)

  If an invisible region is extracted, the process proceeds to S203. Then, using the monitoring space map 241, it is determined whether or not an opening position exists in the extracted invisible region. If there are opening positions in the invisible region, the process proceeds to S204, where it is determined whether there are a plurality of opening positions in the invisible region. If there are a plurality of opening positions in the invisible region, the target position is set to a position where the opening position with the longest distance from the coping member position can be photographed in S207. If there is only one opening position in the invisible region, a position where the opening position can be photographed is set as a target position in S207. When the invisible region is not extracted, and when the opening position does not exist in the invisible region, the target position setting unit C sets a position where the coping member position can be photographed as a target position in S207. Here, when the opening position does not exist in the invisible area, the target position may be set to a position where the position in the invisible area can be photographed. For example, as described above in the description of the target position setting means C, the target position may be set so as to move on the outer periphery of the building B. (S201, S202, S203, S204, S205, S206)

  In the flow of FIG. 12, in S205 and S207, the target position is set to a position at which the opening position having the longest distance from the coordinator position among the opening positions existing in the invisible region can be photographed. However, when there are a plurality of opening positions in the invisible region, a position where the most opening positions can be photographed at one time may be set as the target position. Although not shown, in this case, S205 in FIG. 12 may be changed to a process of “calculating positions where the most aperture positions can be photographed at one time”. In addition, when there are a plurality of positions at which the largest number of opening positions can be photographed at one time, a position having the longest distance from the coping staff position may be set as the target position. In this case, S205 in FIG. 12 is changed to a process of “calculating positions where the most opening positions can be photographed at one time”, and “the most opening positions can be photographed at one time” between S205 and S207 in FIG. To determine whether there is more than one position ", and if there are more than one, set the longest distance from the cooperator position as the target position, and if there is only one, set that position as the target position “Yes” processing should be added.

  Thereafter, the flight path calculation means D calculates a flight path by a path search algorithm using the target position set in step S207, the flight position of the flying robot 3, and the monitoring space map 241. If a flight position and a target position are set, the route search algorithm calculates a route that can be safely and shortly arrived in consideration of the arrangement state of the monitoring space map 241 and the size of the flying robot 3. (S208)

  Then, the robot controller ho calculates the flight control signal of the flying robot 3 so that the flying robot 3 can fly along the route calculated by the flight route calculator d. Specific flight control is the same as that described in S106 of FIG. (S209)

  Thereafter, when the flying robot 3 arrives at the target position, a standby flight is performed in S212. Thereafter, it is confirmed whether or not the counselor position sequentially calculated by the counselor position calculating means (a) is changed, and if it has changed, the process after the setting of the invisible region is performed again in S200. Even when the flying robot 3 has not arrived at the target position, it is checked whether or not the counselor position sequentially calculated by the cooperator position calculating means a is changed, and if changed, it is not visible again in S200. Perform the processing after setting the area. Even when the process proceeds to S200 again, the flight control process to the initially set target position in S209 is continued, so even if the counselor position has changed, there is no change in the newly set target position. The flying robot 3 goes to the initial target position. (S210, S211, S212, S213)

DESCRIPTION OF SYMBOLS 1 ... Imaging system 2 ... Security apparatus 3 ... Flying robot 4 ... Laser sensor 5 ... In-building sensor 6 ... Dealer terminal

Claims (5)

An imaging system comprising: a flying robot having a shooting function; a portable terminal including a display unit that displays the captured image; and a control device that controls the flight of the flying robot to a target position. ,
The controller is
A storage unit for storing a monitoring space map representing a monitoring area including an obstacle such as a building;
Cooperator position calculating means for calculating the current position of the mobile terminal on the monitoring space map as a cooperator position;
An invisible area extracting means for extracting from the monitoring space map an invisible area that is behind the obstacle and is physically invisible from the cooperating person position using the monitoring space map and the coping person position ; ,
Target position setting means for setting, as the target position, a position where the flying robot can photograph a predetermined position in the invisible region in the monitoring space map;
An imaging system comprising:
The coordinator position calculating means sequentially calculates the cooperator position,
The invisible region extracting means sequentially extracts the invisible region;
The imaging system according to claim 1, wherein the target position setting unit sequentially sets the target position. The monitoring space map includes information on an opening position of an opening through which a person can enter and exit the building,
The imaging system according to claim 1, wherein the target position setting unit sets a position where the flying robot can image the opening position in the invisible region as the target position.   The target position setting means sets, as the target position, a position at which the flying robot can photograph the largest number of the opening positions when there are a plurality of the opening positions in the invisible region. The shooting system described in 1. 5. The target position setting means, when there are a plurality of positions that satisfy the condition of the target position, sets a position farthest from the coping staff position as the target position. Shooting system.