JP6726043B2 - Object detection sensor and monitoring system - Google Patents

Description

The present invention relates to an object detection sensor that periodically scans a target area with a rangefinder to detect an object, and a monitoring system that controls a flying robot based on the detection result to monitor the target area.

Conventionally, there has been proposed a monitoring device that detects an intruding object such as a person or a vehicle into a monitoring space by an object detection sensor using a rangefinder. A range finder is a measuring device that measures the distance between two points in a non-contact manner. For example, a laser pulse is sent to a monitoring space, and the distance of a point (detection point) at which an object reflects the laser pulse It can be measured from the time from transmission to reception of reflected light. The object detection sensor scans the monitoring space by sequentially changing the laser pulse transmission direction, and generates distance measurement data representing the position of the detection point determined by the scanning angle (azimuth) and the distance. Incidentally, the scanning is performed one-dimensionally or two-dimensionally, and the distance measurement data obtained by the two-dimensional scanning is called a range image.

Further, background subtraction processing is known as a technique for detecting an intruding object based on distance measurement data obtained from the monitoring space. In this technique, the distance measurement data that is the background is compared with the input distance measurement data, and the area (change area) in which the detection point where the distance has changed by a predetermined value or more appears in the scanning range is obtained. Then, based on the size of the changed area and the distance information of the detection points in the changed area, it is determined whether or not the changed area is due to an intruding object that is a detection target.

In Patent Document 1 below, as a technique for improving the reliability of the determination of whether the change area obtained by the background difference is due to the detection target object or other object, the position of the center of gravity of the change area is obtained and there is a time difference. It is described that it is determined whether or not the object is a detection target object based on the presence or absence of movement of the center of gravity position between images and the movement direction.

The monitoring device takes a predetermined action when the target object is detected. In addition, the monitoring device may determine a motion state such as the presence or absence of movement of the target object, and take action according to the state.

JP-A-4-340178

If the determination of the motion state of the target object is performed based on the change of the center of gravity position of the distance measurement data in the change region, there is a problem that the detected target object is not appropriately dealt with due to the erroneous determination.

For example, when the detection target object is a vehicle, a part of the vehicle may be blocked by a person who has come down from the vehicle, or the laser light from the sensor may be specularly reflected on the vehicle and the reflected light may not return to the sensor. Then, the shape of the detection target object captured by the distance measurement data in the change area may differ from the shape of a normal vehicle. As a result, the monitoring device may erroneously recognize the detection point group, which is originally one vehicle, as a plurality of objects. In this case, if the presence or absence of the movement of the object is determined by the change of the center of gravity position of the ranging data of the change area, for example, the position of the center of gravity changes even when the vehicle is stopped, and the vehicle has moved. False positives can occur.

The present invention has been made to solve the above problems, and improves reliability in determining the motion state of a detection target object based on the position of the center of gravity with respect to the change region, and appropriately deals with the detection target object. It is an object of the present invention to provide an object detection sensor and a monitoring system that can be performed in the same manner.

(1) An object detection sensor according to the present invention includes a measuring unit that periodically scans a target area with a rangefinder to measure the position of a detection point of an object that appears in the target area, and the detection related to the position. Of the cluster of points, a monitoring target detection unit that detects a condition that satisfies a given condition as a monitoring target corresponding to the detection target object, a representative position calculation unit that calculates a representative position of the monitoring target, and the past The same object determination unit that determines the one corresponding to the same detection target object between the monitoring target and the current monitoring target, and the state of the detection target object based on the change in the representative position of the detection target object A state determination unit that makes a determination, wherein the state determination unit determines the state immediately before when the size of the current monitoring target of the detection target object has changed by a predetermined reference or more with respect to the past monitoring target. The determination result is maintained as the current state determination result.

(2) In the object detection sensor according to (1) above, the state determination unit updates the representative position a predetermined number of times when a change in size of the monitoring target that is equal to or larger than the predetermined reference occurs. The state determination result immediately before is maintained as the current state determination result until then, and then the state determination can be performed based on the representative position updated after the size change.

(3) In the object detection sensor according to the above (1) or (2), the state determination unit determines the state immediately before when the detection target object is estimated to have a portion hidden by another object. The result can be maintained as the current state determination result.

(4) A monitoring system according to the present invention includes the object detection sensor according to any one of (1) to (3) above, a flying robot having an imaging means, and a control unit for controlling the operation of the flying robot, and the target area In the monitoring system for monitoring, the control unit, when the object detection sensor detects the monitoring target corresponding to the detection target object, dispatches the flying robot toward the detection target object at a predetermined altitude, When the moving direction of the detection target object is calculated from the change in the representative position and the current state determination result for the detection target object is a stationary state, the detection is performed based on the representative position and the immediately preceding moving direction. When the position and direction of shooting the target object are determined, the flying robot is lowered to instruct to start the shooting operation, and the shooting operation is performed when the current state determination result changes from the stationary state in the middle of the shooting operation. Instruct to stop and rise. Further, the control unit may be installed in a monitoring sensor or the like as an independent device that integrally controls both the object detection sensor and the flying robot. Alternatively, the flying robot main body may be provided with a partial control function of the control unit.

ADVANTAGE OF THE INVENTION According to this invention, the reliability at the time of determining the motion state of a detection target object based on the gravity center position regarding a change area|region improves, and it becomes possible to cope with a detection target object appropriately.

It is a schematic diagram which shows the schematic structure of the monitoring system which concerns on embodiment of this invention. It is a block diagram showing a schematic structure of an object detection sensor concerning an embodiment of the present invention. It is a block diagram which shows the schematic structure of a flying robot. It is a block diagram showing a schematic structure of a flight control device. It is a schematic flow chart of the supervisory control processing of an object detection sensor. It is a schematic flowchart of a state determination process. It is a schematic diagram explaining a state determination process. It is a schematic diagram for demonstrating the method of detecting the partial occlusion by the other object with respect to a detection target object. It is a schematic flow diagram of a coping process in the flight control device.

Hereinafter, embodiments of the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings.

FIG. 1 is a schematic diagram showing a schematic configuration of a monitoring system 1 according to the embodiment. The monitoring system 1 includes an object detection sensor 2, a robot port 3, a flying robot 4, a flight control device 5, and a monitoring center 6.

When the object detection sensor 2 detects an abnormality (for example, a suspicious vehicle M that has entered the surveillance area E) generated in the surveillance area E, the object detection sensor 2 notifies the flight control device 5. Details of the object detection sensor 2 will be described later. When the flight control device 5 receives a notification of abnormality detection from the object detection sensor 2, the flight control device 5 gives a flight instruction to the flying robot 4 via the robot port 3 and outputs an abnormality signal to the monitoring center 6. The flight control device 5 will also be described in detail later.

The Roboport 3 is a waiting place for the flying robot 4, and is equipped with equipment for taking off and landing the flying robot 4 in response to an instruction from the flight control device 5. Further, a mechanism for accommodating the flying robot 4 in the port when the flying robot 4 lands is provided, and when the flying robot 4 is accommodated in the port, power is supplied to the flying robot 4 in contact or non-contact. Have a function. When the flight robot 4 has not received the flight instruction from the flight control device 5, the flight robot 4 is on standby in the Roboport 3. On the other hand, when the flight instruction is received, the flight robot 4 flies to the abnormal place in the monitoring area E, and A target location set around the abnormality occurrence location (for example, a peripheral area including a detection target object such as a suspicious vehicle M) is photographed. For example, the flying robot 4 autonomously flies toward the target position while avoiding the obstacle based on the three-dimensional geographical information in the monitoring area E stored in advance, and determines that there is no obstacle near the target position. Sometimes it descends for shooting. The detailed configuration of the flying robot 4 will be described later.

The monitoring center 6 is a facility equipped with a center device operated by a security company or the like. The center device is usually composed of one or more computers. The center device controls various devices, records the abnormal signal received from the flight control device 5, displays the abnormal information on the monitor, and the monitoring staff monitors the plurality of monitoring areas E. For example, the center device displays a captured image transmitted from the flying robot 4 via the flight control device 5 on a monitor, and a surveillance staff member monitors the surveillance area E with the image. In addition, the monitoring center 6 instructs the flight control device 5 to direct the flight robot 4 to an arbitrary position in the monitoring area E (flight route instruction, target position and speed information) based on the judgment and operation of the observer. Instructions, takeoff instructions, return instructions, climb instructions, etc.).

Next, the object detection sensor 2 will be further described. The object detection sensor 2 is installed corresponding to a warning area S set outdoors, and is installed, for example, on an outdoor wall surface of a building. The caution area S is a target area where the object detection sensor 2 detects an object. A plurality of warning areas S may be set in the monitoring area E, and in this case, the object detection sensor 2 is installed for each warning area S. The object detection sensor 2 performs a spatial scan at a predetermined measurement cycle while irradiating a preset warning area S with a search signal such as a laser beam, and receives reflected light from an object on the optical path to alert the user. The position of the object existing in the area S is detected.

FIG. 2 is a block diagram showing a schematic configuration of the object detection sensor 2. The object detection sensor 2 includes a communication unit 21, a detection unit 22, a storage unit 23, and a control unit 24.

The communication unit 21 communicates with the flight control device 5. Specifically, the communication unit 21 is connected to the flight control device 5 by wire or wirelessly, receives the security start signal and the security release signal output from the flight control device 5, and outputs the signal to the control unit 24. Further, when the control unit 24 determines that the detection target object is present in the warning area S, the communication unit 21 transmits a detection signal including its own address information to the flight control device 5 as a notification of abnormality detection.

The detection unit 22 (measuring unit) is basically a laser rangefinder that periodically scans the warning area S to measure the position of the detection point of the object appearing in the warning area S. It has a mirror 222, a scan control unit 223, a reflected light detection unit 224, and a distance measurement data generation unit 225, and performs a distance measurement operation by irradiating and receiving laser light while scanning the warning area S. Specifically, the detection unit 22 repeats scanning at a predetermined cycle (for example, 60 milliseconds) with respect to the angular range in the horizontal direction that faces the warning area S centering on the object detection sensor 2. The distance measurement by the detection unit 22 is performed by using a time-of-flight method (TOF method) at every predetermined angular step (for example, 0.25°) within a scanning angle range. The time required from emission to light reception is measured, and the distance to the object reflecting the laser is calculated from the time and the speed of light. The distance measurement data by the detection unit 22 is represented by a scanning angle (azimuth) and a distance, and the position of the detection point on the object with the object detection sensor 2 as the viewpoint is given by the distance measurement data.

The laser oscillating unit 221 is, for example, a laser light source of near infrared light having a wavelength of about 890 nanometers, receives the emission timing signal from the scanning control unit 223, and irradiates the scanning mirror 222 with laser light.

The scanning mirror 222 is a movable mirror that reflects laser light, and the scanning control unit 223 performs scanning angle control that controls the direction of the scanning mirror 222 according to the scanning direction of the detection unit 22. This scanning angle control is performed in synchronization with the emission timing control for the laser oscillator 221. In each of the sequentially set scanning directions, the scanning mirror 222 reflects the laser light from the laser oscillator 221 and emits it in the scanning direction, and at the same time, reflects and reflects the laser reflected light from the object existing in the scanning direction. The light is incident on the light detection unit 224.

For example, the scanning mirror 222 can rotate about a predetermined axis, and the scanning control unit 223 rotationally drives the scanning mirror 222 at a constant speed. As a result, the detection unit 22 performs one-dimensional scanning and sequentially irradiates the entire warning region S with laser pulses. When scanning the fan-shaped security area S, the scanning may be one-way scanning from one end to the other end of the scanning angle range, or may be reciprocal scanning between both ends. You can also do it.

Here, when a plane including a group of loci of laser light in the one-dimensional scanning of the laser beam by the detection unit 22 is referred to as a scanning plane, the object detection sensor 2 detects that the scanning plane is a horizontal plane, or the farther the surface is at a depression angle, the more the ground surface. It is installed so that it becomes a flat surface that approaches. Further, the object detection sensor 2 is installed so that the detection target object existing at an arbitrary position in the warning area S is irradiated with the laser light. Specifically, the installation height of the object detection sensor 2 is set so that the scanning plane may intersect the detection target object at an arbitrary position in the warning area S.

The reflected light detection unit 224 includes a light receiving element, is arranged in the vicinity of the laser oscillation unit 221, and the reflected light with respect to the laser pulse emitted from the laser oscillation unit 221 enters from the scanning mirror 222 and is detected by the light receiving element.

The distance measurement data generation unit 225 generates distance measurement data based on the scanning result of the warning area S by the detection unit 22 and outputs it to the control unit 24. As described above, the distance measurement data is data representing the position of the detection point, and is represented by a set of scanning angle and distance. The distance measurement data generation unit 225 acquires the data indicating the scan angle from the scan control unit 223. Further, the distance measurement data generation unit 225 acquires the laser pulse emission timing from the scan control unit 223, acquires the reflected light reception timing from the reflected light detection unit 224, and calculates the distance to the detection point by the TOF method. If the reflected light does not return within the predetermined time, it can be considered that there is no object within the distance that the laser light can irradiate, and the distance measurement data generation unit 225 sets the predetermined pseudo data ( For example, a distance value to the outer periphery of the warning area S, an appropriate value equal to or longer than the effective measurement distance by the laser light, or the like is set.

The storage unit 23 is a storage device including an HDD (Hard Disk Drive), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and stores a program for operating the object detection sensor 2 and various setting information. Remember. For example, the storage unit 23 stores the caution area information 231, the reference data 232, and the tracking information 233.

The security area information 231 is information indicating a security area S set according to security planning of the monitoring area E by a security company or the like as a range to be monitored by the object detection sensor 2, for example.

The reference data 232 is basically distance measurement data in a state where there is no moving object in the warning area S (reference time). As the reference data 232, for example, distance measurement data acquired by scanning the warning area S when the monitoring system 1 is activated can be used. The reference data 232 corresponds to background data used for background subtraction processing, and is comparison reference information used for extracting an object that appears in the warning area S by comparison with the current distance measurement data. The reference data 232 can be stored in the form of a table in which distances are associated with angles (directions), for example.

The tracking information 233 is information used for a tracking process of tracking an object newly appearing in the warning area S over a plurality of cycles. In the tracking information 233, the position and size in each scanning cycle are associated and stored for each object. Further, the tracking information 233 can store identification information of the object (vehicle, person, etc.), and in particular, information indicating whether the object is a detection target object. Further, since a plurality of objects may appear in the warning area S, an identifier (object ID) for distinguishing the objects is added. In this case, the tracking information 233 is stored in association with the object ID.

When a plurality of object detection sensors 2 are installed, the storage unit 23 stores the address information for identifying the object detection sensor 2 itself before the operation of the monitoring system 1 is started.

The controller 24 includes a drive controller 241, a reference data generator 242, and a monitor controller 243. Specifically, the control unit 24 is configured by a microcomputer including a CPU (Central Processing Unit), ROM, RAM and the like and its peripheral circuits. The microcomputer reads the program from the storage unit 23 and executes the program to drive the control unit. 241, a reference data generation unit 242, and a monitoring control unit 243.

The drive control unit 241 outputs a drive signal to the detection unit 22 when a security start signal is input from the flight control device 5 via the communication unit 21, and starts driving the detection unit 22. As a result, driving of the scanning mirror 222 by the scanning control unit 223 and irradiation of laser light by the laser oscillation unit 221 are started. On the other hand, the drive control unit 241 outputs a drive stop signal to the detection unit 22 when the security release signal is input from the flight control device 5, and stops the drive of the detection unit 22 at the end of scanning at that time. As a result, driving of the scanning mirror 222, irradiation of laser light, and the like are stopped.

The reference data generation unit 242 uses the distance measurement data acquired by the detection unit 22 to generate the reference data 232 stored in the storage unit 23. As described above, the reference data 232 is distance measurement data in a state where there is no moving object in the warning area S (reference time). For example, when the control unit 24 receives a security start signal from the flight control device 5, the reference data generation unit 242 stores the distance measurement data output in the first scanning in the storage unit 23 as the reference data 232. In addition, the user may separately cause the object detection sensor 2 to perform the reference data acquisition operation prior to the start of security. The user or the like can execute the reference data acquisition operation after confirming that there is no moving object in the warning area S. For example, existing data such as plants and outer walls existing in the warning area S from the beginning are fetched as the reference data 232 by the reference data generation unit 242.

The monitoring control unit 243 groups the detection point groups of the objects appearing in the warning area S to generate clusters of which the positions are close to each other, and detects one of the clusters that satisfies a predetermined condition. It has a function as a monitoring target detection unit that detects a monitoring target corresponding to the target object.

Specifically, the monitoring controller 243 compares the current distance measurement data with the reference data and extracts the detection point of the object appearing in the warning area S. That is, the difference between the distance value for each scanning angle obtained from the current distance measurement data and the distance value for each angle stored in the reference data is calculated for each corresponding angle, and the current value at the angle at which the distance value has changed is calculated. It is determined that the detection point of the distance measurement data is the detection point of the object appearing in the warning area S (which will be referred to as a foreground detection point) instead of the background object. In the foreground detection point group, the monitoring control unit 243 classifies the foreground detection point groups having adjacent angles and having distance values equal to or smaller than a threshold value as common points on the same object surface.

The monitoring control unit 243 determines whether the cluster of the foreground detection points has a characteristic that the cluster obtained from the detection target object has, and when the cluster has the characteristic, the cluster of the foreground detection point is set as the monitoring target. For example, the monitoring controller 243 makes the determination based on the feature amount calculated from the cluster. As will be described later, the monitoring system 1 determines the motion state of a specific type (specific target) of the detection target objects, and controls the shooting operation of the flying robot 4 according to the state. In this embodiment, the detection target object to be specified is a vehicle. The clusters corresponding to the vehicles are arranged in a substantially linear shape or a substantially L shape on the scanning plane, and the linear portion of the shape may have a length of several meters. Therefore, it is possible to detect a portion that can be approximated as a line segment in the arrangement of the detection point groups forming the cluster, and use the length of the approximate line segment as the above-described feature amount of the vehicle. Further, it is possible to detect the monitoring target of the detection target object other than the vehicle (for example, a person) based on the characteristics of the size and shape of the object recognized from the cluster. Alternatively, in addition to the above characteristics, the speed determined from the cluster, the reflection intensity of the distance measurement data, and the like may be used as the characteristics for determining the vehicle.

In addition, the monitoring control unit 243 determines a function as a representative position calculation unit that calculates the representative position of the monitoring target, and determines a corresponding target object between the past monitoring target and the current monitoring target. It has a function as the same object determination unit.

For example, the position of the center of gravity of the cluster can be used as the representative position. The tracking process for associating the past monitoring target with the current monitoring target can be performed based on the distance of the representative position and the similarity of the characteristics of the monitoring target. For example, for a vehicle that is a specific target, the distance between the representative positions of the cluster in the previous cycle and the cluster in the current cycle is within a threshold, and the size of the object represented by the length of the approximate line segment described above. If the variation of the height is within the threshold, it is determined that both clusters correspond to the same object, that is, the same vehicle. The result of the tracking process is recorded in the tracking information 233. Regarding the calculation of the position of the center of gravity, when the cluster has a substantially L-shape as shown in FIG. 7A, which will be described later, the three end points of the L-shape and one bending point are set as the vertices. The center-of-gravity position of the triangle may be calculated, or the center-of-gravity position of the rectangle or parallelogram having the L-shaped two sides (point P ₀ shown in FIG. 7A) may be calculated. Further, when the schematic shape of the cluster is a line segment as shown in FIG. 7B, the midpoint of the line segment (point P ₁ shown in FIG. 7B) can be calculated as the position of the center of gravity. .. The method of calculating the position of the center of gravity is not limited to this, and it is also possible to use a position other than the position of the center of gravity as the representative position.

Furthermore, the monitoring control unit 243 has a function as a state determination unit that determines the state of the detection target object based on the change in the representative position of the detection target object. For example, the monitoring control unit 243 makes a state determination regarding the motion state of the specific target that is the detection target object. Specifically, the monitoring controller 243 determines whether the vehicle is moving or stopped. The status determination result is stored in the tracking information 233. Regarding the state determination, not only whether the object is moving or stopped, but the moving direction, speed, acceleration, etc. of the detection target object may be determined. In either case, it is possible to make a determination from the change in the representative position in a predetermined period.

Here, the state determination unit does not perform the state determination based on the representative position for the detection target object when the size of the current monitoring target for the detection target object changes by a predetermined reference or more with respect to the past monitoring target, The previous state determination result is maintained as the current state determination result (holding operation during change).

Further, after starting the state determination holding operation, the state determination unit maintains the immediately previous state determination result as the current state determination result until the representative position is updated a predetermined number of times, and thereafter is updated after the size change. The status is determined based on the representative position (hold release operation).

Furthermore, the state determination unit does not perform the state determination based on the representative position when it is estimated that the detection target object has a portion hidden by another object, and uses the immediately previous state determination result as the current state determination result. Maintain (holding operation at the time of concealment).

Next, the flying robot 4 will be described. FIG. 3 is a block diagram showing a schematic configuration of the flying robot 4. The flying robot 4 includes a rotor 41, a rotor driving unit 42, an antenna 43, an altitude sensor 44, a photographing unit 45, a storage unit 46, a power supply 47, and a robot control unit 48.

The rotor 41 is a rotary blade, and a plurality of (for example, four) rotors are provided, and the rotor 41 is driven by a rotor drive unit 42 to rotate. The rotor drive unit 42 drives the rotor 41 to rotate under the control of the robot control unit 48. The flying robot 4 flies with the lift force generated by the rotation of the rotor 41, and by controlling the rotation of the rotor 41, the flying robot 4 can be raised, lowered, translated, or changed in direction.

The antenna 43 is used for wireless communication with the flight control device 5 by low power wireless communication, Wi-Fi, or the like.

The altitude sensor 44 is a sensor that measures the current altitude of the flying robot 4, and operates under the control of the robot controller 48. For example, the altitude sensor 44 is composed of an atmospheric pressure sensor, a laser range finder that emits laser light vertically downward from the body of the flying robot 4, and the like.

The image capturing unit 45 is configured by, for example, a camera using an image sensor, and captures an image of the surroundings of the flying robot 4 (for example, the front or the bottom).

The storage unit 46 temporarily stores the detected object information and the obstacle information transferred from the flight control device 5. In addition, the images captured by the image capturing unit 45 during the flight of the flying robot 4 are sequentially stored.

The power supply 47 is composed of, for example, a rechargeable battery such as a lithium polymer battery, and supplies necessary power to each part of the flying robot 4.

The robot controller 48 includes a photographing controller 48a, a rotor controller 48b, and an attitude controller 48c.

The photographing control means 48a performs live transmission to the flight control device 5 of the start and end of photographing of the photographing unit 45, control of the photographing angle of the photographing unit 45, and the image photographed by the photographing unit 45 based on an instruction from the flight control device 5. And so on.

The rotor control means 48b controls the operation of the rotor 41 via the rotor drive section 42. For example, the rotor control means 48b causes the flying robot 4 to fly toward the detection target object while avoiding obstacles according to the obstacle information received from the flight control device 5 and temporarily stored in the storage unit 46, The altitude and speed of the flying robot 4 are controlled so as to reach the target value instructed by the flight controller 5.

The attitude control means 48c controls the in-flight attitude of the flying robot 4 based on the flight state (direction, attitude, acceleration, etc.) of the flying robot 4, the current position, and the current altitude.

Next, the flight control device 5 will be described. The flight control device 5 is a device that controls the operation of the flying robot 4, and is installed in a predetermined location in the monitoring area E or in the vicinity of the monitoring area E. The flight control device 5 is connected to the object detection sensor 2 via, for example, a LAN (Local Area Network), determines the abnormality of the monitoring area E based on the detection signal of the object detection sensor 2, and sends an abnormality signal to the monitoring center 6. Along with outputting, a signal for giving a flight instruction to the flying robot 4, information on detected objects, and information on obstacles are transmitted.

FIG. 4 is a block diagram showing a schematic configuration of the flight control device 5. The flight control device 5 includes a communication unit 51, a storage unit 52, and a control unit 53.

The communication unit 51 performs wireless communication with the flying robot 4, for example, by low power wireless communication or Wi-Fi communication. For example, the communication unit 51 receives information such as position (latitude, longitude, altitude) and speed as flight state information from the flying robot 4 and passes the information to the control unit 53, and the control unit 53 receives the information according to the received information. Various control signals to be output are transmitted to the flying robot 4.

Further, when the flight control device 5 receives a flight instruction of the flying robot 4 from the monitoring center 6, the communication unit 51 transmits various control signals according to the flight instruction to the flying robot 4.

Furthermore, the communication unit 51 transmits the image captured by the image capturing unit 45 of the flying robot 4 to the monitoring center 6 via a virtual dedicated network (VPN) constructed on a wide area network (WAN) such as the Internet. Also, the detected object information is received from the object detection sensor 2 and input to the control unit 53.

The storage unit 52 is a storage device including, for example, a ROM and a RAM, and stores a program for operating the flight control device 5, various setting information, and the like. Further, in the storage unit 52, a flight area map in which the area in which the flying robot 4 flies is expressed in three dimensions of latitude, longitude, and altitude, monitoring area information that is various information regarding the monitoring area E, and communication with the flying robot 4 are stored. Data for execution, various parameters for controlling the flight of the flying robot 4, position information (latitude, longitude information) of the Roboport 3, type of the object detection sensor 2 in the monitoring area E, and installation position information (latitude, longitude information) ) Is stored.

The control unit 53 includes flight control means 53a, shooting control means 53b, and state confirmation means 53c. Specifically, the control unit 53 is composed of a microcomputer including a CPU and the like, and the microcomputer reads and executes the program from the storage unit 52 to function as the flight control unit 53a, the shooting control unit 53b, and the state confirmation unit 53c. To do.

When the object detection sensor 2 detects a monitoring target corresponding to the detection target object, the flight control means 53a dispatches the flying robot 4 to the detection target object. Specifically, the flight control means 53a gives a flight instruction to the flying robot 4 when an abnormality is detected by the object detection sensor 2. Further, the flight control means 53a acquires flight state information, position information, and altitude information from the flying robot 4, and transmits control signals (target position, speed, etc.) related to the flight of the flying robot 4 to the flying robot 4 for flight. Control the flight of the robot 4.

The shooting control means 53b controls shooting by the shooting unit 45 of the flying robot 4, and generates a shooting permission signal (shooting prohibition cancellation signal) or a shooting prohibition signal based on the current position (altitude) acquired from the flying robot 4. , To the flying robot 4 via the communication unit 51.

The state confirmation means 53c confirms the motion state of the detection target object based on the output of the object detection sensor 2. For example, the state confirmation unit 53c calculates the moving direction of the detection target object from the change in the representative position of the monitoring target corresponding to the detection target object. In addition, when the current state determination result of the detection target object by the object detection sensor 2 is the stationary state, the state confirmation unit 53c determines the position and direction at which the detection target object is imaged based on the representative position and the moving direction immediately before. After that, the photographing control means 53b is caused to transmit an instruction to start the photographing operation to the flying robot 4. On the other hand, when the current state determination result changes from the still state during the shooting operation, the status checking unit 53c causes the shooting control unit 53b to send a command to stop the shooting operation to the flying robot 4.

The monitoring of the monitoring area E by the object detection sensor 2 can be started or canceled by an instruction from the monitoring center 6 to the flight control device 5, and the flight control device 5 is provided with an operation unit operated by a user or the like. A user or the like can operate the operation unit to set the start or cancellation of the monitoring. The flight control device 5 receives an instruction/setting for starting or canceling the monitoring, and transmits a guard start signal or a guard release signal to the object detection sensor 2, respectively.

Next, the operation of the monitoring system 1 will be described. FIG. 5 is a schematic flowchart of the monitoring control processing of the object detection sensor 2. The processing is started when the object detection sensor 2 receives the security start signal, and a series of processing including steps S1 to S6 is repeated until the security release signal is received.

The distance measurement data generation unit 225 acquires distance measurement data from the warning area S (step S1). The monitoring control unit 243 performs background subtraction processing using the reference data 232 on the obtained distance measurement data to obtain a cluster of foreground detection points. Note that a plurality of clusters can be acquired in the background difference processing. The monitoring control unit 243 extracts, from the clusters, those whose characteristics such as size are monitoring targets that satisfy a predetermined condition of a detection target object such as a vehicle or a person (step S2).

When the monitoring target is extracted (in the case of “Yes” in S2), tracking processing is performed on the monitoring target using the tracking information 233 (step S3). That is, when it is determined that the monitoring target extracted in the current cycle is the same object as any of the clusters recorded in the tracking information 233 obtained in the previous cycle, the monitoring control unit 243 monitors the current cluster. Target information is additionally written, and when a cluster determined to be the same object is not recorded in the tracking information 233, it is recorded in the tracking information 233 as a newly appearing object.

Next, if the monitoring target is the specific target (“Yes” in step S4), the monitoring control unit 243 performs a state determination process (step S5), and outputs a detection signal of the monitoring target together with the determination result to the flight control device. 5 (step S6). On the other hand, if the monitoring target is not the specific target (“No” in step S4), the monitoring control unit 243 omits the state determination process (step S5) and transmits the detection signal of the monitoring target to the flight control device 5. Yes (step S6). In the present embodiment, the vehicle is the specific target, and if the monitoring target is the vehicle, the state determination process S5 is performed, and for example, the process is omitted for a non-specific target such as a person.

After the detection signal transmission (S6) is completed, and when the monitoring target is not extracted in step S2, the process returns to step S1.

FIG. 6 is a schematic flowchart of the state determination processing S5. The monitoring controller 243 determines whether or not the size of the monitoring target has changed by a predetermined reference value or more (step S51). When the change is detected (“Yes” in S51), the monitoring controller 243 resets the position information history of the monitoring target recorded in the tracking information 233 (step S52), and The hold operation at the time of change is performed, and the immediately previous state determination result regarding the motion state of the monitoring target is maintained as the current state determination result (step S55).

Further, the monitoring control unit 243 does not reach the predetermined number of representative position information stored in the tracking information 233 even when the change in the size of the monitoring target is less than the reference value (in the case of “No” in S51). If (Yes in step S53), the previous state determination result is maintained (S55).

The case where the representative position information in step S53 is less than the predetermined number corresponds to a case where it is shortly after the position information history is reset in step S52. When the number of times of measurement at the representative position is small, the influence of the measurement variation becomes large, and the reliability of the determination of the presence or absence of movement of the monitoring target becomes low. Therefore, in such a case, the state determination based on the representative position (step S56) is not performed, and the change hold operation is continued.

The number of scans during which the hold operation during change is continued is set so that the dispersion of the measurement values converges and the stationary monitoring target is not erroneously determined to be moving. Although the number of times depends on the accuracy of the object detection sensor 2, for example, it may be about several times. If the scanning cycle is 60 milliseconds, the representative position can be measured about 16 times per second, so it can be said that the suspension of the state determination about several times has a small effect on the real-time property of abnormality detection.

When the representative position information is obtained a predetermined number of times or more (“No” in step S53), the monitoring control unit 243 updates the representative position after the position information is reset due to the size change of the reference value or more. It is possible to determine the state based on the. That is, the hold release operation for releasing the hold of the state determination due to the size change is performed.

Even when the monitoring target appears in the warning area S shortly after, the same process as the above-described change hold operation and hold release operation can be performed. However, in this case, in step S55, the state in which there is no previous state determination result is maintained, and the first state determination result is obtained in the state determination based on the representative position after the hold releasing operation (step S56).

Further, as the processing of the monitoring control unit 243, even if the change in the size of the monitoring target is less than the reference value (in the case of “No” in S51), the detection target object has a portion hidden by another object. When it is estimated that this is the case (“Yes” in step S54), the state determination based on the representative position (S56) is not performed, and the immediately previous state determination result is maintained as the current state determination result (S55). It is possible to add a concealment hold operation to perform. When the detection target object is estimated to have a portion hidden by another object (in the case of “Yes” in S54), the monitoring control unit 243 causes the detection target object recorded in the tracking information 233 to be detected. The position information history of the monitoring target corresponding to the object may be reset.

When occlusion occurs, the motion state of the detection target object is determined based on only a part that can be measured by the object detection sensor 2, and the reliability of the state determination is basically low. Therefore, also in this case, the state determination based on the representative position (step S56) is not performed, and the hidden state holding operation is performed to output a more reliable previous state.

Therefore, in the present embodiment, the change in the size of the monitoring target is less than the reference value (in the case of “No” in S51), the hold release operation is performed (in the case of “No” in S53), and When partial occlusion of the monitoring target by another object has not occurred (“No” in step S54), the state determination based on the representative position (S56) is performed.

FIG. 7 is a schematic diagram for explaining the state determination processing S5, in which a cluster of foreground detection points to be monitored for the detection target object is represented by a solid line on the scanning plane. FIG. 7A shows an example in which a warning area S in which a stopped vehicle (automobile) is present is scanned by a laser beam from the object detection sensor 2, and the monitoring target 100 corresponding to the vehicle surface is detected in an L shape. The center of gravity P ₀ defined for the monitored object 100 is indicated by a cross. FIG. 7B shows an example of a state in which a person gets out of the vehicle of FIG. 7A, and the monitoring targets 101 and 102 correspond to a part of the monitoring target 100 of FIG. 7A. On the other hand, the monitoring target 103 corresponds to a person standing between the vehicle and the object detection sensor 2. Since a part of the monitoring target 100 is hidden by a person, the vehicle is detected by being divided into two monitoring targets 101 and 102.

For example, assume that FIG. 7A shows the state of the previous scanning period, and FIG. 7B shows the state of the current scanning period. The monitoring control unit 243 performs tracking between the monitoring target of FIG. 7A and the monitoring target of FIG. 7B based on the representative position and size of the monitoring target. The larger one of the close monitoring targets 101 and 102 is stored in the tracking information 233 in association with the monitoring target 100. At that time, the center of gravity P ₁ of the monitored object 101 is recorded as the representative position.

For example, the monitoring control unit 243 compares the sizes of the monitoring target 100 and the monitoring target 101 in the state determination processing S5, and the change in size between the previous scanning period and the current scanning period is equal to or greater than the reference value. If it is determined to be (Yes in S51), the representative position P ₀ of the monitored object 100 up to the previous scanning cycle recorded in the tracking information 233 is erased (S52), while the center of gravity P ₁ is changed. It is recorded in the tracking information 233 as a representative position. Although the representative position changes from P ₀ to P ₁ between the previous scanning period and the current scanning period, the monitoring control unit 243 does not determine that the motion state of the monitored vehicle is the moving state. Instead, the current state determination result is maintained in the same stationary state as the previous state determination result (S55).

FIG. 7B shows an example in which the size change of the monitored object corresponding to the vehicle occurs due to the occlusion by a person who is another object, but the size change may occur due to other causes. .. For example, it may occur due to a change in the reflection state. Specifically, when the door of the stopped vehicle is opened and the direction of the door is such that the diffuse reflection component of the laser light to the object detection sensor 2 becomes less than the detection lower limit, or the portion of the door where the laser light hits is This is the case when it causes specular reflection.

As shown in FIG. 7B, when a person who partially shields a stopped vehicle moves in front of the vehicle, the size or representative position of the monitored object corresponding to the vehicle changes. The size change at that time may be less than the reference value in step S51, but may be erroneously determined to be a moving state in the state determination processing. However, the monitor control unit 243 performs the above-described concealment holding operation to maintain the current state determination result in the stationary state and prevent an erroneous determination regardless of the size change in the concealed state.

FIG. 8 is a schematic diagram for explaining a method of detecting partial occlusion of the detection target object by another object, showing a detection point 112 on the scanning plane, a positional relationship of the scanning line L of the laser beam, and the like. There is. An object 110 indicated by a straight line represents a vehicle, and an object 111 indicated by an ellipse represents a person located in front of the vehicle when viewed from the object detection sensor 2, and detection points 112 are obtained on the respective surfaces.

The monitoring control unit 243 defines a cluster consisting of detection points whose positions are close to each other as a monitoring target, and thus, in the example of FIG. And a cluster of detection points 112d and a plurality of detection points arranged on the right side thereof are extracted as monitoring targets. Further, clusters corresponding to the object 111 and having the detection points 112b and 112c at both ends are extracted as monitoring targets.

The monitoring control unit 243 determines that the shield is generated when the two monitoring targets are adjacent to each other with respect to the scanning angle. Specifically, the scanning angle at which the scanning line L ₁ of the detection point 112 a located at the end of the object 110 to be monitored and the scanning line L ₂ of the detection point 112 b located at the end of the object 111 to be monitored are adjacent to each other. Since it has, it is determined that a part of the object 110 is shielded by the object 111 located in front of it. The same determination is made between the detection points 112d and 112c.

Next, a coping process in the flight control device 5 when the object detection sensor 2 detects the detection target object will be described. As described above, the monitoring system 1 determines the vehicle as a specific target among the detection target objects, determines the motion state of the vehicle, and controls the shooting operation of the flying robot 4 according to the state. Specifically, when the vehicle is stationary, the robot 4 is caused to perform the photographing operation, while when the vehicle is in the moving state, the photographing operation is not performed. When the vehicle is in the moving state, the flight control device 5 keeps the flying robot 4 at a relatively high flight altitude to track the vehicle, and when the vehicle stops, lowers the altitude of the flying robot 4 to obtain a suitable image of the vehicle. The shooting operation is instructed. At that time, the flight control device 5 can determine a position and a direction for photographing the vehicle and instruct the photographing. For example, the flight robot 4 is positioned in front of and behind the vehicle to photograph the license plate, It is possible to give an instruction to take a picture of a face. Specifically, when photographing a license plate of a vehicle, it is necessary to position the flying robot 4 on the front surface or the rear surface of the vehicle, but the direction of the front surface or the rear surface of the vehicle may be determined from the moving direction of the vehicle. it can. Therefore, the shooting direction can be determined from the moving direction immediately before the vehicle, that is, the moving direction obtained immediately before the vehicle stops. Regarding the shooting position, in consideration of the sudden start of the stopped vehicle, it is possible to safely set a position that is a predetermined distance (for example, 5 m) or more away from the representative position of the vehicle within the shooting range. It is possible to take a picture. When the vehicle starts moving after the vehicle stops and starts the photographing operation, the flight control device 5 causes the flying robot 4 to interrupt the photographing operation and move to the tracking operation from the sky.

FIG. 9 is a schematic flowchart of the coping process in the flight control device 5. As described above, the object detection sensor 2 transmits a detection signal to the flight control device 5 when detecting the detection target object. When the flight control device 5 waits for the detection signal (step S10) and acquires the object information of the detection target (detection target object) sent together with the detection signal (in the case of "Yes" in S10), the flight robot is based on the information. The target position for dispatching No. 4 and the flight route are set (step S11). For example, the detection signal from the object detection sensor 2 includes position information of the detected object in the warning area S, and when a plurality of object detection sensors 2 are installed, address information specifying the object detection sensor 2 is provided. Including. The flight control device 5 can set the target position from these information and the installation position information of the object detection sensor 2 stored in the storage unit 52. Further, the flight control device 5 can set a flight route by using a flight area map or the like stored in the storage unit 52.

The flight control device 5 instructs the set flight route to the flight robot 4 via the Roboport 3, and also transmits obstacle information existing on the flight route to the flight robot 4 (step S12) to instruct the flight robot 4 to take off. Is given (step S13).

The flying robot 4 is equipped with an altitude sensor 44, a GPS (Global Positioning System), and an inertial measurement unit (Inertial Measurement Unit: IMU) to grasp its own position while flying according to a set flight route. It is possible to inform the flight control device 5 of its own position. The flight control device 5 monitors whether the flying robot 4 is near the target position (step S14).

When the flying robot 4 reaches the vicinity of the target position (in the case of “Yes” in S14), the flight control device 5 outputs the latest state determination result output by the object detection sensor 2 for the monitoring target corresponding to the detection target object is still. It is confirmed whether or not it is in the state (step S15), and if it is stationary, the flying robot 4 is moved to the photographing operation (in the case of "Yes" in S15). That is, a descent instruction is given to the flying robot 4 (step S16), a photographing permission signal is transmitted, and photographing of the vehicle to be detected is started (step S18). For example, it is possible to end the coping process by the end of the shooting (in the case of “Yes” in step S20).

When the monitored object is in the moving state when it reaches the vicinity of the target position (“No” in S15), the flight control device 5 corrects the target position based on the detection signal of the object detection sensor 2, The robot 4 is caused to perform a tracking operation (step S21).

Note that the flight control device 5 monitors the state determination result from the object detection sensor 2 even after the instruction of the shooting operation, and when the motion state of the detection target vehicle changes from the stationary state to the moving state, the flight robot 4 captures the image. The operation is interrupted and the tracking operation is started. For example, when the moving state is set at the timing after the descending instruction (S16), the ascent is instructed without instructing the start of photographing (S18) (step S22). Further, when the moving state is reached at the timing after the transmission of the photographing permission signal (S18), the photographing prohibition signal is transmitted to interrupt the photographing (step S23), and an instruction to rise is given (step S22). Then, the target position is corrected (S21), and the flying robot 4 after the ascent is moved to the tracking operation.

Further, in the above description, the example in which the flying robot 4 operates in response to an instruction from the flight control device 5 has been described, but the present invention is not limited to this, and in the flying robot 4, a detection signal from the object detection sensor 2 or a detection target Upon receiving the object information and the like, the flying robot 4 itself calculates the moving direction of the detection target object, determines the position and direction for shooting the detection target object, and determines the start (descent)/stop (up) of shooting. You may do so.

The object detection sensor 2 of the monitoring system 1 according to the above-described embodiment is configured to use only the reliable center-of-gravity position information, without using the low-reliability center-of-gravity position information when determining the motion state of the monitoring target. .. Specifically, when it is determined that the size of the monitoring target during tracking has changed by a predetermined reference or more, the state determination based on the position of the center of gravity is not performed, and the immediately previous state determination result is maintained as the current state determination result. ..

According to the object detection sensor 2 of the above-described embodiment, another object may enter between the object detection sensor 2 and the object to be monitored to cause partial shielding, or specular reflection of laser light on the object to be monitored. Even if the position of the center of gravity of the monitoring target changes significantly due to the occurrence of a situation, for example, it is possible to prevent erroneous determination that the stopped vehicle that is the monitoring target is moving. Further, by using the detection result of the object detection sensor 2, it becomes possible to appropriately control the flying robot 4 in the monitoring system 1 that controls the flying robot 4.

In the above embodiment, the case where the stationary state is maintained has been described, but the determination result that the moving state is maintained may be maintained. For example, the state determination result is maintained when a vehicle moving in the warning area S passes an object such as a person located in front of the object detection sensor 2. At that time, the motion state of the vehicle (moving direction, speed, acceleration, etc.) calculated by the tracking process can be maintained.

Further, although the object detection sensor 2 performs one-dimensional scanning in the above-described embodiment, it may be one that performs two-dimensional scanning and acquires a range image.

Further, in the present embodiment, an example of determining whether the moving state or the stationary state is described as an example of the state determination, but the present invention is not limited to this. For example, in the state determination, when the moving direction of the detection target object is determined, the accuracy of determining the moving direction of the object is improved, so that the object can be accurately tracked and photographed. Further, in the state determination, when determining the speed of the detection target object, the accuracy of the determination of the speed of the object is improved, and for example, tracking is performed while keeping the distance between the flying robot and the detection target object appropriately. Will be possible. As described above, the present invention can be applied to various state determinations.

Further, in the present embodiment, an example in which the state determination based on the representative position is not performed when the size of the monitoring target changes by a predetermined reference or more has been described, but the present invention is not limited to this, and when the shape of the monitoring target changes Alternatively, the state determination based on the representative position may not be performed. Specifically, referring to FIG. 7, which shows a situation in which a person has exited from a stopped vehicle, with respect to the shape of the vehicle, the monitoring target in FIG. 7A shows a substantially L-shape, and FIG. Shows a substantially linear shape. In this way, even if the state determination based on the representative position is not performed when the shape of the monitored object changes, the same effect as that of the present embodiment can be obtained.

Further, in the present embodiment, when the size of the monitoring target has changed by a predetermined reference or more with respect to the past monitoring target, the state determination based on the representative position is not performed, but the present invention is not limited to this, and based on the representative position. Although the state determination is performed, the same effect as that of the present embodiment can be obtained even in the configuration in which the determination result is not adopted.

Further, in the present embodiment, when the size of the monitoring target has changed by a predetermined reference or more with respect to the past monitoring target, or when it is estimated that the detection target object has a portion hidden by another object Although the example of resetting the position information history of the monitoring target has been described, the present invention is not limited to this, and the position information history is held without being reset, but the configuration may not be used for the state determination.

1 Monitoring System, 2 Object Detection Sensor, 3 Roboport, 4 Flying Robot, 5 Flight Control Device, 6 Monitoring Center, 21 Communication Unit, 22 Detection Unit, 23 Storage Unit, 24 Control Unit, 41 Rotor, 42 Rotor Drive Unit, 43 Antenna, 44 Altitude sensor, 45 Imaging unit, 46 Storage unit, 47 Power supply, 48 Robo control unit, 48a Imaging control unit, 48b Rotor control unit, 48c Attitude control unit, 51 Communication unit, 52 Storage unit, 53 Control unit, 53a Flight control means, 53b shooting control means, 53c state confirmation means, 221 laser oscillation section, 222 scanning mirror, 223 scanning control section, 224 reflected light detection section, 225 distance measurement data generation section, 231 warning area information, 232 reference data, 233 tracking information, 241 drive control section, 242 reference data generation section, 243 monitoring control section.

Claims (4)

A scanning unit periodically scans a target area by a rangefinder, and a measurement unit that measures the position of a detection point for an object that appears in the target area,
Among the clusters of the detection points regarding the position, a monitoring target detection unit that detects a condition that satisfies a predetermined condition as a monitoring target corresponding to the detection target object,
A representative position calculation unit for calculating the representative position of the monitoring target,
An identical object determination unit that determines the same target object to be detected between the monitoring target in the past and the current monitoring target,
A state determination unit that determines the state of the detection target object based on a change in the representative position of the detection target object,
Equipped with
The state determination unit performs the state determination when the change of the size of the current monitoring target with respect to the past monitoring target for the detection target object is less than a predetermined reference, while the current determination for the detection target object is performed. If changes against the past monitored magnitude of the monitored is the predetermined criterion or more is not carried out the state determination, maintaining the state immediately before the determination result as the current state determination result,
An object detection sensor characterized by. The object detection sensor according to claim 1,
When a change in size of the monitoring target that is equal to or greater than the predetermined reference occurs, the state determination unit thereafter maintains the immediately previous state determination result as the current state determination result until the representative position is updated a predetermined number of times. Then, thereafter, the state determination is performed based on the representative position updated after the size change, the object detection sensor. The object detection sensor according to claim 1 or 2,
The state determination unit maintains the immediately previous state determination result as the current state determination result when it is estimated that the detection target object has a portion hidden by another object, Detection sensor. A monitoring system for monitoring the target area, comprising: the object detection sensor according to any one of claims 1 to 3; a flying robot having an imaging means; and a controller controlling the operation of the flying robot. There
The control unit is
When the object detection sensor detects the monitoring target corresponding to the detection target object, the flight robot is dispatched toward the detection target object at a predetermined altitude,
Calculate the moving direction of the detection target object from the change in the representative position,
When the current state determination result for the detection target object is a stationary state, a position and a direction for capturing the detection target object are determined based on the representative position and the moving direction immediately before, and the flying robot is lowered. To start the shooting operation,
If the current state determination result changes from the stationary state during the shooting operation, instructing the shooting operation to be stopped and to rise.
Monitoring system characterized by.

Images