JP2016177640A - Video monitoring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video monitoring system which properly performs flight control of a flying body by accurately estimating a location and a posture of the flying body even when the monitoring area is an indoor space.SOLUTION: The video monitoring system includes: a flying body 1 with a camera and an inclination sensor mounted; a stationary camera 2 installed in a monitoring area; parameter storage means 3 for storing space information on the monitoring area, location estimation means 4 for estimating location information on the flying body on the basis of image information imaged by the stationary camera and the space information; posture estimation means 5 for estimating posture information on the flying body on the basis of photographic images by the camera mounted in the flying body, inclination information detected by the inclination sensor and the location information on the flying body; and a flight movement control means 6 for controlling flight movement of the flying body on the basis of the location information on the flying body, the posture information on the flying body and the space information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video surveillance system including a camera installed in a surveillance area and a flying object for assisting the surveillance function of the camera.

  As equipment for monitoring public facilities, retail stores, factories, and the like, video surveillance systems including cameras and recorders (video recording devices) are widely used. In video surveillance systems, it is common for humans to visually monitor video captured by a camera, but long-term monitoring by humans is a burden on the mind and body. Thus, as a technique for saving human monitoring, a human action recognition and tracking technique using an image recognition technique is disclosed (for example, see Patent Document 1).

  Furthermore, in order to efficiently monitor a wide area, a video monitoring technique that assists a monitoring function using a mobile robot has been proposed. Patent Document 2 discloses a flying object (flying robot) having a function of photographing with a camera while following a monitored object (intruder).

JP 2008-26974 A JP 2014-149621 A

Edited by Masatoshi Okutomi, “Digital Image Processing”, CG-ARTS Association, 2004, pp. 252-255 Z. Zhang, "A flexible new technique for camera calibration", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22, No.11, pp.1330-1334, 2000.

  In a video surveillance system, a single camera can monitor a wide range, but the monitoring range is limited. Further, the smaller the distance from the camera to the monitoring target, the smaller the monitoring target appears. For example, when a person is far away from the camera, the presence of the person can be grasped from the photographed image of the camera, but the details of the person's face and behavior cannot be grasped. Therefore, in order to cover the entire monitoring area, it is necessary to install a number of cameras corresponding to the size of the monitoring area. Therefore, there is a problem that it is very expensive to introduce a video surveillance system having a large number of cameras.

  When the monitoring area is wide, by introducing a flying object as disclosed in Patent Document 2, an image can be taken close to the monitoring target. In this case, in order for the flying object to recognize the surrounding environment and determine the movement path, a process (self-position estimation process) for estimating its current position (self-position) and posture is required. Therefore, when the flying object cannot detect its own position and posture, the function as the flying object (tracking or monitoring an intruding person) cannot be continued, such as flying away from the monitoring area. .

  Further, when the flying object flies outdoors, the self-position can be estimated by using a GPS (global positioning system) receiver. However, since GPS radio waves cannot be received indoors, there is a problem that self-position cannot be estimated and monitoring cannot be continued.

  As for the attitude of the flying object, the inclination with respect to the horizontal plane can be estimated by using, for example, an acceleration sensor. As for the rotation angle (azimuth) around the vertically upward axis, the absolute direction can be known by using a magnetic direction sensor when outdoors. However, since the accuracy of the magnetic direction sensor is greatly lowered due to the influence of structures such as buildings and roofs indoors, there is a problem that the self-position cannot be estimated and monitoring cannot be continued.

  Furthermore, the indoor structure is simple, and windows, ceiling panels, and fixtures are often regularly arranged, and the scenery and structure may be similar at different positions. Therefore, there is a problem that an erroneous estimation result is obtained when self-position estimation based on image data is performed. Furthermore, although Patent Document 2 performs self-position estimation using a laser scanner, there is a problem that an expensive sensor is required to obtain high estimation accuracy.

  Precise position information can be transmitted to the flying object by installing a means for transmitting position information such as beacons and optical markers in the monitoring area, but a large number of these devices are installed in the monitoring area. However, there is a problem that the cost is very high for performing the regular maintenance.

  The object of the present invention is to solve the above problems and provide a video surveillance system for accurately controlling the flying object by accurately estimating the position and posture of the flying object even if the monitoring area is an indoor space. There is to do.

  An image monitoring system according to the present invention includes a flying object equipped with a camera and a tilt sensor, a fixed camera installed in the monitoring area, parameter storage means storing spatial information of the monitoring area, and image information captured by the fixed camera. And position estimation means for estimating the position information of the flying object based on the spatial information, a captured image of a camera mounted on the flying object, inclination information detected by the inclination sensor, and position information of the flying object. Based on attitude estimation means for estimating the attitude information of the flying object, flight information control means for controlling the flying action of the flying object based on the position information of the flying object, the attitude information of the flying object, and the spatial information; Is provided.

  According to the video surveillance system of the present invention, the position of the flying object is estimated from the image information of the fixed camera installed on the monitoring area side, and at the same time, the attitude of the flying object from the image information and tilt information of the camera mounted on the flying object. Is estimated. Therefore, even in indoor spaces where it was difficult to estimate position and orientation in the past, the position and orientation of the flying object can be accurately estimated with a simple configuration, and the flying operation of the flying object can be controlled appropriately. Can do.

It is a block diagram which shows the structure of the video surveillance system which concerns on the 1st Embodiment of this invention. It is a schematic diagram which shows the structure of the video monitoring system of FIG. It is a schematic diagram which shows the structure of the flying body 1 of FIG. It is a schematic diagram of the fixed camera 2 of FIG. It is a figure for demonstrating modeling of the monitoring area 101 which the flying body 1 of FIG. 1 monitors. 4 is a schematic diagram of an image 221 captured by a fixed camera 2. FIG. It is a schematic diagram of the image 121 image | photographed with the camera 12 with which the flying body 1 is provided. It is the schematic diagram which looked at the monitoring area 101 of FIG. 5 from the top. It is a block diagram which shows the structure of the video surveillance system which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the video surveillance system which concerns on the 3rd Embodiment of this invention.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In the following embodiments, the same components are denoted by the same reference numerals, and description thereof is omitted.

First embodiment.
FIG. 1 is a block diagram showing a configuration of a video surveillance system according to the first embodiment of the present invention. The video monitoring system of FIG. 1 includes a flying object 1, a plurality of fixed cameras 2, a parameter storage unit 3, a position estimation unit 4, a posture estimation unit 5, and a flight motion control unit 6. . Here, each component is provided with communication means, and various information or control signals are transmitted and received by wired or wireless communication between the components. In FIG. 1, each component is separated from each other. However, a plurality of components may be modularized in a single housing, or each component may be a vehicle 1 or a fixed camera 2. It may be built in the housing.

  FIG. 2 is a schematic diagram showing the configuration of the video monitoring system of FIG. Here, an example of monitoring the monitoring area 101 is shown. In FIG. 2, three fixed cameras 2A, 2B, and 2C are installed in the monitoring area 101, and the flying object 1 is photographing the person 102 in the monitoring area. The operation of each component of the video surveillance system in FIG. 1 will be described below with reference to FIG. In FIG. 2, three fixed cameras 2 </ b> A, 2 </ b> B, and 2 </ b> C are installed in the monitoring area 101, but two or more cameras 2 may be provided in the monitoring area 102. In this case, the number of installed fixed cameras 2, the installation location, and the posture (orientation) are adjusted so that at least two of the fixed cameras 2 can always capture the image of the flying object 1.

  FIG. 3 is a schematic diagram showing the configuration of the flying object 1 of FIG. Here, the flying object 1 includes a mechanism necessary for movement by flight, a camera 12, an inclination sensor that detects the inclination of the flying object 1 with respect to the horizontal plane, and a rechargeable battery. Driven by supply. In FIG. 3, the flying object 1 includes a main body 10, four rotors 11, a camera 12, and a light source 13. The rotor 11, the camera 12, and the light source 13 are attached to the main body 10. The flying object 1 is a quad-copter type flying object that flies by the rotation of four rotors 11. By changing the number of rotations of the four rotors 11, the flight speed, altitude, direction, and attitude of the flying object 1 can be controlled. The tilt sensor is used to measure the tilt of the flying object 1 with respect to the horizontal plane with respect to the horizontal plane. For example, an acceleration sensor is used. The flying object 1 is not limited to the quadcopter type, and any flying mechanism can be used. In addition to the configuration in which the flying object 1 is driven by power supply from a built-in rechargeable battery, the flying object 1 may be configured to receive and drive power from a wired power cable connected to the flying object 1.

  The light source 13 is mounted on the flying object 1 in order to make it easier to detect the position of the flying object 1 from the captured image taken by the fixed camera 2. With this configuration, even when the distance between the flying object 1 and the fixed camera 2 is long, the flying object 1 can be easily detected from the captured image of the fixed camera 2.

  Here, it is preferable to arrange the light source 13 in the housing of the flying object 1 so that the fixed camera 2 can photograph the light source 13 regardless of the attitude of the flying object 1. Furthermore, in order to improve the accuracy with which the position estimation unit 4 described later estimates the position of the flying object 1, it is desirable to arrange the light source 13 as close as possible to the optical center of the camera 12 provided in the flying object 1. In the block diagram shown in FIG. 1, the video monitoring system is configured to include one flying object 1, but may be configured to include a plurality of flying objects 1.

  FIG. 4 is a schematic diagram of the fixed camera 2 of FIG. In FIG. 2, the fixed camera 2 (2 </ b> A to 2 </ b> C) includes a camera body 20, a mounting leg 21, and a light source 22. With this configuration, even when the distance between the flying object 1 and the fixed camera 2 is long, the fixed camera 2 can be easily detected from the captured images of the camera 12 included in the flying object 1. Here, the attachment leg 21 and the light source 22 are attached to the camera body 20. The fixed camera 2 is fixedly installed in the monitoring area 101 by attaching the mounting leg 21 to the ceiling or wall of the monitoring area 101 and adjusting the posture of the camera body 20. In addition, the light source 22 is preferably disposed at a position as close as possible to the optical center of the fixed camera 2 and is disposed so as to be visible from any direction around the fixed camera 2. Further, the light source 22 may be a flash light source. With this configuration, even when the distance between the flying object 1 and the fixed camera 2 is long, the fixed camera is selected from the images captured by the camera 12 included in the flying object 1. It becomes easier to detect the position of 2.

  The flying object 1 flies over the monitoring area 101, captures an image of the monitoring area 101 with the camera 12, and outputs the captured image information to the posture estimation unit 5. The flying object 1 detects inclination information with respect to the horizontal plane of the flying object 1 using an inclination sensor, and outputs the detected inclination information with respect to the horizontal plane of the flying object 1 to the attitude estimation unit 5.

  A plurality of fixed cameras 2 are installed in the monitoring area 101, and the situation of the monitoring area 101 is photographed to acquire image information and output it to the position estimation unit 4. Further, the image information for estimating the position of the flying object 1 flying in the monitoring area 101 is acquired, and the acquired image information is transmitted to the position estimating unit 4.

  The parameter storage unit 3 includes spatial information necessary for estimation processing (described later) of the position estimation unit 4 and attitude estimation unit 5 and control of flight operation of the flying object 1 (described later) executed by the flight operation control unit 6; The camera parameters of each camera 12 and the fixed camera 2 are stored. Details of the camera parameters will be described later. Here, the spatial information includes a flight plan (flight schedule information such as a flight schedule and a flight course) of the flying object 1.

  The position estimation unit 4 estimates the three-dimensional position of the flying object 1 in the monitoring area 101 based on the image information and spatial information captured by the fixed camera 2, and uses the estimated three-dimensional position information as the flying object position information. Output to the posture estimation unit 5.

  The posture estimation unit 5 includes the image information acquired by the camera 12 of the flying object 1, the flying object position information input from the position estimation unit 4, and the inclination information with respect to the horizontal plane of the flying object 1 obtained from the inclination sensor. Based on this, the attitude information of the flying object 1 is estimated, and the estimated attitude of the flying object 1 is output to the flight operation control unit 6 as the flying object attitude information. Here, the posture estimation unit 5 estimates the posture using the fact that the fixed camera 2 is reflected in the captured image captured by the camera 12 of the flying object 1. The fixed camera 2 is provided with a light source 22 in order to detect the position of the fixed camera 2 from the captured image.

  The flight operation control unit 6 compares the flying object position information estimated by the position estimation unit 4 and the flying object posture information estimated by the posture estimation unit 5 with the flight path information stored in the parameter storage unit 3 in advance. Then, based on the comparison result, flight control information for controlling the flight operation of the flying object 1 is generated and transmitted to the flying object 1. Here, the flight operation control unit 6 calculates an actual flight path based on the position information of the flying object 1 and the attitude information of the flying object 1, and calculates the calculated flight path and the flight path information of the airplane 1. In comparison, the flight operation of the flying object 1 is controlled based on the comparison result. Here, when it is determined that the position and posture of the flying object 1 are out of the flight path stored in advance, the flight operation control unit 6 generates flight control information to return to the flight path stored in advance. The flight operation of the flying object 1 is controlled.

  As described above, in the video surveillance system of FIG. 1, the position estimation unit 4 estimates the position of the flying object 1, the attitude estimation unit 5 estimates the attitude of the flying object 1, and the flight motion control unit 6 performs the flight. The operation control is repeated. With this configuration, even if the monitoring area 101 is an indoor space, the position and orientation of the flying object 1 can be estimated with high accuracy and the flying operation of the flying object 1 can be controlled with a simple structure.

  Next, the modeling of the space of the monitoring area 101 will be described.

FIG. 5 is a diagram for explaining the modeling of the monitoring area 101 monitored by the aircraft 1 of FIG. In FIG. 5, the X _W , Y _W , Z _W axis and the origin (0, 0, 0) orthogonal to each other are determined for the three-dimensional space of the monitoring area 101, and a three-dimensional coordinate system is defined. One unit of each X _W , Y _W , and Z _W axis corresponds to, for example, 1 centimeter in real space. This coordinate system is called the world coordinate system.

As shown in the figure, the floor surface 101a of the monitoring area 101 is assumed to be a plane parallel to the horizontal plane, and a plane formed by two orthogonal X _W and Y _W axes is made to coincide with the floor surface 101a. Further, the _{Z W} axis vertical upward axis to the floor surface _101a, X _W, Y W, the world coordinate system is defined by _{Z W-axis.}

  Next, a plurality of fixed cameras 2 are installed in the monitoring area 101. Here, a model that describes the correspondence between the three-dimensional coordinate system of the real space and the two-dimensional coordinate system of the fixed camera 2 is introduced. An outline of the most general perspective projection model is shown as this model. Details of the perspective projection model are disclosed in Non-Patent Document 1, for example. Here, a description will be given based on a simple perspective projection model in which lens distortion is ignored. Note that the video surveillance system according to the present invention can be used for a camera that cannot be modeled by a perspective projection model by introducing a model corresponding to the camera.

  First, the three-dimensional X, Y, and Z axes that have the optical center of the camera as the origin, the Z axis coincides with the optical axis of the camera, and the X axis and Y axis are positioned parallel to the horizontal and vertical directions of the image, respectively. Consider a coordinate system. This three-dimensional coordinate system is called a camera coordinate system. At this time, the relationship between the world coordinate system and the camera coordinate system can be expressed by the following equation.

但し、

However,

Here, the matrix R is a 3 × 3 rotation matrix, the vector t is a three-dimensional translation vector, and the rotation matrix R and the vector t are collectively referred to as an external parameter of the camera. Here, if X = Y = Z = 0 is substituted into Expression (1), the optical center coordinates (X ₀ , Y ₀ , Z ₀ ) of the camera in the world coordinate system are obtained as follows.

  Next, image coordinates (u, v) are introduced as a coordinate system representing the position of the two-dimensional camera image. One unit of image coordinates corresponds to a pixel. It is assumed that the origin of the image coordinate system is determined at an appropriate position of the image, for example, the upper left of the image. When each coordinate system is defined in this way, the relationship between the image coordinate system and the camera coordinate system can be expressed by the following equation.

Here, f _u and f _v are focal lengths (units are pixels) of the camera, and (c _u , c _v ) are positions of intersections of the optical axis and the image plane (image center) in the image coordinate system. The constants are collectively referred to as camera internal parameters.

  Further, the relationship between the world coordinate system and the image coordinate system can be expressed by the following equations from the above-described equations (1) and (2).

  As described above, the three-dimensional coordinate system (world coordinate system) in the real space and the two-dimensional coordinate system (image coordinate system) of the camera are related by the parameter appearing on the right side of Equation (4). These external parameters and internal parameters are collectively referred to as camera parameters.

  Further, as a preparatory work of the video monitoring system in FIG. 1, calibration is executed for each of the fixed cameras 2 installed in the monitoring area 101. Calibration refers to the work of obtaining the camera parameters described above. The camera 12 of the flying object 1 is also calibrated in advance to obtain internal parameters. Here, the camera parameters acquired by the calibration are stored (stored) in the parameter storage unit 3.

  As the calibration method described above, for example, an existing method such as the Zhang method (see Non-Patent Document 2) can be used. For example, information defining the range in which the flying object 1 can fly in the monitoring area 101 in the world coordinate system, or the flight route information when the flying object 1 makes a round flight in the monitoring area 101 are stored in the parameter storage unit 3. May be.

  Next, a procedure for the position estimation unit 4 to estimate the position of the flying object 1 in the three-dimensional space will be described.

FIG. 6 is a schematic diagram of an image 221 captured by the fixed camera 2. Here, it is assumed that one of the plurality of fixed cameras 2 is focused and the flying object 1 is detected at the coordinates (u _A , v _A ) of the captured image. As described above, this can be easily realized by detecting light emitted from the light source 13 of the flying object 1, for example.

  The flying object 1 and the person 102 are in the field of view of the fixed camera 2 and are shown in the captured image 221. In FIG. 6, the picked-up image of a mode that the light source 13 with which the flying body 1 is light-emitting is shown typically.

From equation (3), the direction vector d in the camera coordinate system of the straight line connecting the optical center (X ₀ , Y ₀ , Z ₀ ) of the camera and the three-dimensional coordinates of the aircraft 1 can be expressed as follows: it can.

Multiplying by R ⁻¹ from the left of the vector d in equation (5), a straight direction vector d _W in the world coordinate system is obtained. Direction vector d _W is expressed by the following equation.

Here, the equation of a straight line that passes through the camera optical center (X ₀ , Y ₀ , Z ₀ ) and is parallel to the direction vector d _W is expressed by the following equation using the parameter t (t moves all real numbers): Can be represented.

  At a certain point in time, when the flying object 1 is detected from the images of a plurality of fixed cameras 2 photographed at the same time, the position estimation unit 4 first determines the optical center of the fixed camera 2 and the three-dimensional coordinates of the flying object 1 according to the procedure described above. Equation (7) of a straight line connecting The position of the flying object 1 in the world coordinate system can be obtained by obtaining this straight line for all the fixed cameras 2 and calculating the intersection of these straight lines.

  The light source 13 may be a flash light source so that the flying object 1 can be easily recognized from the captured image of the fixed camera 2. With this configuration, even when the distance between the flying object 1 and the fixed camera 2 is long, the flying object 1 can be easily detected from the captured image of the fixed camera 2. Furthermore, the lighting timing of the light source 13 may be synchronized with the imaging timing of the fixed camera 2 under the control of the position estimation unit 4. That is, the light source 13 emits light in synchronization with the timing when the fixed camera 2 captures an image. With this configuration, the accuracy of detecting the flying object 1 from the captured image of the fixed camera 2 is further improved.

  In addition, the video surveillance system may be composed of a plurality of flying objects 1, and in that case, the lighting timing of the light source 13 provided in the flying object 1 is set so that the position of each flying object 1 can be estimated independently. You may control so that it may shift.

  The position of the flying object 1 in the world coordinate system estimated by the above procedure is assumed to be the same as the coordinates of the optical center P of the camera 12 included in the flying object 1, and will be described below.

  Subsequently, a procedure in which the posture estimation unit 5 estimates the posture of the flying object 1 will be shown.

The attitude estimation unit 5 can obtain the rotation angle about the X _W and Y _W axes parallel to the horizontal plane from the tilt information from the flying object 1. Here, a procedure for estimating the rotation angle about the remaining Z _W-axis (the optical axis and the X _W axis and forms angle of the camera 12).

  FIG. 7 is a schematic diagram of an image 121 taken by the camera 12 included in the flying object 1. In FIG. 7, the fixed camera 2 and the person 102 are in the field of view of the camera 12, and each is shown in the captured image 121. FIG. 7 schematically shows a captured image in which the light source 22 included in the fixed camera 2 emits light.

  FIG. 8 is a schematic view of the monitoring area 101 of FIG. 5 as viewed from above. Here, the optical center of the camera 12 of the flying object 1 is P, the intersection of the optical axis of the camera 12 and the image plane is c, and the focal length is f. Here, each intersection point c and focal length are parameters obtained by the above-described camera calibration corresponding to the image center and focal length of the camera 12, respectively.

  Further, it is assumed that the fixed camera installed at the point Q in FIG. The position (world coordinates) of the fixed camera 2 is acquired as an external parameter of the camera by the above-described camera calibration. It is assumed that the posture estimation unit 5 can determine which of the plurality of fixed cameras 2 is the fixed camera 2 shown in the image of the camera 12. This can be realized, for example, by synchronizing the lighting timing of the light source 22 included in the fixed camera 2 with the timing when the camera 12 captures an image. With this configuration, the accuracy of detecting the fixed camera 2 from the captured image of the camera 12 included in the flying object 1 is improved, and the plurality of fixed cameras 2 can be easily identified.

  Further, u is a projection point on the image plane of the fixed camera installed at the point Q. The point u on the captured image of the camera 12 is obtained by detecting light emitted from the light source 22 included in the fixed camera 2 from the image as described above.

  Here, an angle φ formed by a straight line connecting the optical center P of the camera 12 included in the flying object 1 and the fixed camera Q and the optical axis of the camera 12 included in the flying object 1 is expressed by the following equation with reference to FIG. Can be expressed as

Therefore, if the projection point u of the point Q on the image is obtained, the angle φ can be obtained using the known camera parameters c and f. Here, the coordinates of the optical center P of the camera 12 are obtained by the position estimation unit 4. Also, the coordinates of the fixed camera Q can be obtained from the equation (2) by executing the above-described camera calibration, assuming that the optical center coordinates of the fixed camera and the camera position are approximately the same. Therefore, it is possible to determine the direction vector of the straight line PQ, from the angle φ obtained from the equation (8), the optical axis and the X _W axis and angle formed θ of the camera 12 in FIG. 8.

  By the above procedure, the posture estimation unit 5 can estimate the posture of the flying object 1, that is, the rotation angle for each of the three axes of the three-dimensional coordinate system. The light source 22 provided in the fixed camera 2 may be a flash light source. With this configuration, it becomes easier to detect the position of the fixed camera 2 from the image captured by the camera 12 of the flying object 1. Further, the posture estimation unit 5 may be controlled so that the timing when the camera 12 of the flying object 1 captures and the lighting timing of the light source 22 are synchronized.

  According to the video monitoring system according to the above embodiment, the position of the flying object 1 is estimated from the image information of the fixed camera 2 installed on the monitoring area 101 side, and the position estimation result and the camera 12 of the flying object 1 are used. The attitude of the flying object 1 can be estimated from the taken image and the inclination information of the flying object 1 with respect to the horizontal plane. Therefore, it is possible to estimate the position and posture of the flying object 1 with high accuracy even in an indoor environment where GPS, a magnetic bearing sensor, or the like cannot be used. Therefore, it is possible to control an appropriate operation for the flying object 1. Further, since it is not necessary to install a means for transmitting position information in the monitoring area 101, it is not necessary to perform maintenance on the means. Therefore, it is possible to reduce the operating cost of the video surveillance system.

Second embodiment.
FIG. 9 is a block diagram showing a configuration of a video surveillance system according to the second embodiment of the present invention. Compared to the video monitoring system of FIG. 1, the video monitoring system of FIG. 9 includes a flight motion control unit 6A instead of the flight motion control unit 6, and a moving object recognition unit 7 that recognizes a moving object that has entered the monitoring area 101. Is further provided.

  The moving object recognition unit 7 recognizes the moving object in the monitoring area 101 based on the captured image of the fixed camera 2. The moving object recognizing unit 7 generates position information of the recognized moving object based on the spatial information stored in the parameter storage unit 3 and transmits the position information to the flight motion control unit 6A. Here, the moving object recognition unit 7 may be configured to recognize a moving object based on a captured image obtained by photographing the moving object with the camera 12 of the flying object 1.

  As a means for the moving object recognition unit 7 to detect a moving object in the monitoring area 101, an existing object detection method can be used. As an existing object detection method, for example, there is a face detection algorithm for detecting a human face. If the face of a single person can be detected simultaneously from a plurality of fixed cameras 2, the position estimation unit 4 determines the three-dimensional position of the face in the world coordinate system in the same manner as the procedure for estimating the position of the flying object 1 in the three-dimensional space. Can be sought. However, the face detection algorithm is generally limited in the size of the face that can be detected, and can be applied only when a human face is captured in a certain area (for example, about 500 pixels or more) on the image.

  Another example of a method for detecting a moving object is a visual volume intersection method. This is a method of photographing an object with a plurality of cameras arranged around the object, and estimating a range in the space where the object can exist based on the silhouette shape of the object. The silhouette shape of the object can be extracted by, for example, a background difference algorithm.

  Compared with the flight operation control unit 6 in FIG. 1, the flight operation control unit 6 </ b> A is further controlled to guide the flying object 1 to the position of the moving object based on the position information of the moving object from the moving object recognition unit 7. Flight control information to be generated is transmitted to the aircraft 1.

  Here, the moving object is a target to be monitored by the video monitoring system according to the present embodiment, for example, a person who has entered the monitoring area 101. When the distance between the fixed camera 2 installed in the monitoring area 101 and the moving object is far away, even if it is detected from the captured image of the fixed camera 2 that the object has entered, the type of the object cannot be identified. Therefore, a moving object that has entered the monitoring area 101 is recognized by the fixed camera 2, and the flying object 1 approaches the moving object and is photographed by the camera 12 by controlling the flight operation to guide the flying object 1 to that position. It becomes possible to do.

  Compared with the video monitoring system according to the first embodiment, the video monitoring system according to the present embodiment further controls the flight operation based on the position information of the moving object transmitted from the moving object recognition unit 7. Can do. With this configuration, since the position of an object such as a person moving in the monitoring area 101 can be grasped, it is possible to continue shooting the moving object while the flying object 1 follows the movement of the moving object. For example, a flight operation that goes around the front of the person is planned so that the camera 12 included in the flying object 1 can capture the face of the invading person, and flight control information is transmitted to the flying object 1.

  According to the video surveillance system according to the above embodiment, the position of the moving object that has entered the surveillance area 101 is further recognized and the vehicle 1 is moved as compared with the video surveillance system according to the first embodiment. You can get close to the object and take a picture. Therefore, it is possible to confirm (identify) detailed information about the intruding object. In addition, even when the intruding object is out of the field of view of the camera 12 included in the flying object 1, the moving object recognizing unit 7 can recognize the position of the intruding object based on the captured image of the fixed camera 2. It is possible to immediately change the flight operation so as to follow the movement of the camera and continue shooting with the camera.

Third embodiment.
The video monitoring system according to the present embodiment is characterized by further including a device (a coupling device 8 to be described later) for retaining the flying object 1 when the flight operation is not performed, as compared with the above-described embodiment. With this configuration, the camera 12 of the flying object 1 can perform the same operation as that of the fixed camera 2 while the flying object 1 is coupled to the coupling device 8.

  FIG. 10 is a block diagram showing a configuration of a video surveillance system according to the third embodiment of the present invention. Compared with the video monitoring system of FIG. 1, the video monitoring system of FIG. 10 includes a position estimation unit 4A instead of the position estimation unit 4, a posture estimation unit 5A instead of the posture estimation unit 5, and a coupling device 8. It is further provided with a feature. Here, the coupling device 8 causes the flying object 1 to stay on the ceiling or wall of the monitoring area 101.

  In FIG. 10, when the coupling device 1 is coupled to the flying object 1 by, for example, flight control of the flying object 1, information indicating that the coupling device 8 and the flying object 1 are coupled is generated to estimate the position. To the unit 4 and the posture estimation unit 5, respectively.

  In FIG. 10, one or more coupling devices 8 are fixedly attached to the ceiling or wall of the monitoring area 101 and have a coupling mechanism necessary for coupling to the flying object 1. For example, an arm or a joint mechanism is used as the coupling mechanism. Further, the coupling device 8 and the flying object 1 may be attracted by the magnetic force to support the flying object 1. Further, when the configuration of the flying object 1 is driven by a built-in battery, the coupling device 8 is provided with a connection terminal (contact terminal) so that it can be charged while the flying object 1 is coupled to the coupling device 8. A power feeding function or a non-contact power feeding function that can feed power via the connection terminal may be provided.

  When the position estimation unit 4A receives information indicating that the flying object 1 is coupled from the coupling device 8 as compared with the position estimation unit 4 of FIG. 1, the coupling device 8 and the flying object 1 are coupled. The position information at the time of performing may be output. Here, the position information is an attachment position of the coupling device 8 and is stored in the parameter storage unit 3 as position information.

  When the posture estimation unit 5A receives information indicating that the flying object 1 is coupled from the coupling device 8 as compared to the posture estimation unit 5 of FIG. 1, the coupling device 8 and the flying vehicle 1 are coupled. You may make it output the attitude | position information at the time of doing. Here, the posture information is posture information calculated based on the attachment position of the coupling device 8 and is calculated in advance and stored in the parameter storage unit 3.

  The operation of the video surveillance system according to the third embodiment configured as described above will be described below. Only differences from the video monitoring system according to the first embodiment will be described.

  When the flying object 1 flies over the monitoring area 101, the flight operation control unit 6 transmits flight start control information to the flying object 1. Then, the flying object 1 is separated from the coupling device 8 by the flight control of the flying object 1, for example, and starts flying in the monitoring area 101. When the flight is stopped or when the built-in battery of the flying object 1 is depleted, the flight operation control unit 6 outputs control information that causes the flying object 1 to fly toward the position of the coupling device 8. Close to 8 and join.

  Note that the coupling mechanism of the coupling device 8 is such that one or both of the position and the attitude of the flying object 1 are uniquely determined by coupling the flying object 1 and the coupling device 8 by, for example, flight control of the flying object 1. May be configured. With this configuration, it is possible to obtain in advance one or both of the position and posture of the flying object 1 coupled to the coupling device 8. At this time, when the position estimation unit 4A receives information indicating that the flying object 1 is in the combined state from the combining device 8, the position estimating unit 4A obtains the position estimation result of the flying object 1 based on the captured image of the fixed camera 2. Instead of outputting, position information when the combining device 8 and the flying object 1 that have been obtained in advance may be output may be output. Similarly, the posture estimation unit 5A receives a posture estimation result based on image information and tilt information transmitted from the flying object 1 when receiving information from the combining device 8 indicating that the flying object 1 is in a combined state. Instead of outputting the attitude information, the attitude information when the coupling device 8 obtained in advance and the aircraft 1 are coupled may be output.

  Furthermore, while the flying object 1 is coupled to the coupling device 8, the camera 12 included in the flying object 1 can have the same function as the fixed camera 2. In this case, calibration is performed in advance on the camera 12 included in the flying object 1 in a state where the flying object 1 is coupled to the coupling device 8, and the acquired parameters are stored in the parameter storage unit 3. Therefore, while the flying vehicle 1 is coupled to the coupling device 8, the camera 12 included in the flying vehicle 1 can be used in the same manner as the fixed camera 2.

  According to the video surveillance system according to the above embodiment, as compared with the video surveillance system according to the first embodiment, the built-in battery is charged by being coupled with the coupling device 8 while not performing the flight operation. It is possible to stay while. Further, while the flying object 1 is coupled to the coupling device 8, the same function as that of the fixed camera 2 is provided, so that the remaining amount of the built-in battery of the flying object 1 and the conditions such as the flight operation planned in advance are met. This makes it possible to determine flexible flight behavior.

  Further, the video surveillance system may include a plurality of flying bodies 1 and may be configured to alternately perform charging and flying operation by coupling with the coupling device 8 by the plurality of flying bodies 1. With this configuration, it becomes possible for another aircraft 1 to fly over the monitoring area 101 (cooperative operation) while the aircraft 1 with a small remaining amount of the built-in battery is coupled to the coupling device 8 and charging. Therefore, even if the battery of one flying object 1 runs out, it becomes possible to continuously monitor the monitoring area 101 by operating another flying object 1 in flight.

Modification 1
It is also possible to adopt a configuration in which the embodiments described above are appropriately combined. In that case, it is possible to obtain an effect obtained by combining the effects produced by the combined embodiment. For example, a video surveillance system including both the moving object recognition unit 7 and the coupling device 8 is configured by combining the second embodiment and the third embodiment of the present invention. Then, the flying object 1 charges the built-in battery while being coupled with the coupling device 8 during normal times, and when the moving object recognition unit 7 recognizes the moving object (intruder) in the monitoring area 101, the flying object 1 is coupled to the coupling device. It is possible to determine the flight operation so as to separate from 8 and fly to the position of the moving object.

  As described above in detail, according to the video surveillance system according to the present invention, even if the surveillance area is an indoor space, it is possible to accurately control the flight of the flying object by accurately estimating the position and posture of the flying object. It becomes.

  1-1 flying object, 2 fixed camera, 3 parameter storage unit, 4, 4A position estimation unit, 5, 5A attitude estimation unit, 6, 6A flight operation control unit, 7 moving object recognition unit, 8 coupling device.

Claims (12)

An aircraft equipped with a camera and tilt sensor;
A fixed camera installed in the surveillance area;
Parameter storage means storing spatial information of the monitoring area;
Position estimation means for estimating position information of the flying object based on image information captured by the fixed camera and the spatial information;
Attitude estimation means for estimating attitude information of the flying object based on a captured image of a camera mounted on the flying object, inclination information detected by the inclination sensor, and position information of the flying object;
A video surveillance system comprising flight operation control means for controlling the flight operation of the flying object based on the position information of the flying object, the attitude information of the flying object, and the spatial information.   The flight operation control means calculates an actual flight path based on the position information of the flying object and the attitude information of the flying object, and compares the calculated flight path with the flight path information of the airplane. The video surveillance system according to claim 1, wherein the flight operation of the flying object is controlled based on the comparison result.   The moving object recognition means which recognizes the object which moves in the said monitoring area based on the image information image | photographed with the said fixed camera, and estimates the positional information on the recognized moving object is provided. The video surveillance system described.   4. The video surveillance system according to claim 3, wherein the flight motion control means controls the flight motion of the flying object based on the position information of the moving object estimated by the moving object recognition means.   The video surveillance system according to claim 1, further comprising a coupling device that couples the flying object.   The video surveillance system according to claim 5, wherein the coupling device includes a coupling mechanism coupled to the flying object. The video surveillance system according to claim 1, wherein the flying object includes a first light source. The video surveillance system according to claim 7, wherein the first light source is a flash light source. The video surveillance system according to claim 7 or 8, wherein the first light source emits light in synchronization with a timing at which the fixed camera captures an image. The video surveillance system according to any one of claims 1 to 9, wherein the fixed camera includes a second light source. The video surveillance system according to claim 10, wherein the second light source is a flash light source. The video surveillance system according to claim 10 or 11, wherein the second light source emits light in synchronization with a timing at which a camera included in the flying object captures an image.

Images