JP7384601B2 - Monitoring system - Google Patents

Description

The present invention relates to a monitoring system that monitors intrusion into a predetermined area using an installed object sensor and an imaging device.

BACKGROUND ART Conventionally, a monitoring system has been put into practical use in which a fence sensor is installed on a fence surrounding the outer periphery of a facility, and when the fence sensor detects an abnormality, an alarm is issued and a supervisor is notified through a monitoring terminal device. Such surveillance systems help prevent intruders from entering critical facilities such as airports, power plants, and prisons. Fence sensors include those that use infrared sensors installed at the top of the fence to detect intruders attempting to pass above the fence, those that detect when an intruder applies force to the wire stretched above the fence, and those that are attached to the fence. There are cable-shaped vibration sensors (sensor cables) that detect vibrations when an intruder touches a fence.

In recent years, systems that use surveillance cameras to monitor fences have also been considered. For example, Patent Document 1 discloses an invention in which a plurality of landmarks are installed on the top or side of a fence, and an intruder is detected from the shaking state of the landmarks photographed by a surveillance camera.

Japanese Patent Application Publication No. 2016-152525

The following problems are expected with the conventional technology.
- There is a possibility of false alarms being triggered by small animals or natural phenomena (e.g. wind, earthquakes, etc.).
・When multiple fence sensors over a wide area issue alarms simultaneously due to the influence of natural phenomena, it takes time to confirm the alarms.
- If the timing at which many fence sensors over a wide area issue an alarm due to the influence of a natural phenomenon or the like overlaps with the timing at which a fence sensor that responds to an actual intruder issues an alarm, there is a possibility that the intruder may be missed.

The present invention has been made in view of the above-mentioned conventional circumstances, and aims to provide a technology that can reduce the workload of a supervisor in a surveillance system that monitors intrusions into a predetermined area. purpose.

In the present invention, in order to achieve the above object, a monitoring system is configured as follows.
That is, the monitoring system according to the present invention includes an installed object sensor attached to an installed object surrounding a predetermined area, an imaging device that photographs an area including the installed object, and an image pickup device that takes an image in response to an alarm from the installed object sensor. a server that analyzes a video shot by the device and detects an object that appears in the video; the server analyzes the positional relationship between the installation and the object in the video, and the area of the object in the video. It is characterized in that it is determined whether the object is an intruder based on the following.

Here, as a configuration example, if at least a part of the object is in the vicinity of a boundary line of the lower end or the upper end of the installation object in the video, and the area of the object is greater than or equal to a threshold, The object may be determined to be an intruder.

In this case, the server includes a moving object contour detecting section that detects the contour of the object in the video, and a moving object contour detecting section that determines whether the contour of the object detected by the moving object contour detecting section is in the vicinity of the boundary line. It may also include a boundary line determination section.

The server also includes a moving object area measuring unit that measures an area inside the outline of the object detected from the video, and an area comparison unit that compares the area of the object measured by the moving object area measuring unit with the threshold value. It may also include a section.

In one configuration example, the monitoring terminal device further includes a monitoring terminal device that notifies the alarm of the installed object sensor, and the monitoring terminal device changes the notification mode depending on the determination result of the server as to whether or not the object is an intruder. It is also possible to have a configuration that changes the .

According to the present invention, it is possible to reduce the workload of a supervisor in a surveillance system that monitors intrusions into a predetermined area.

1 is a diagram showing a schematic configuration example of a monitoring system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating how an imaging device is installed. It is a figure showing an example in the case of using a vibration sensor as a fence sensor. 2 is a diagram showing an example of a monitoring screen in the monitoring system of FIG. 1. FIG. 2 is a diagram illustrating an example of a processing flow by a server of the monitoring system in FIG. 1. FIG. It is a figure which shows the example in the case of using an infrared sensor as a fence sensor.

A monitoring system according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a schematic configuration example of a monitoring system according to an embodiment of the present invention. The monitoring system of this example includes a fence sensor 10, an imaging device 20, a recording device 30, a server 40, and a monitoring terminal device 60, which are connected via a network.

The fence sensor 10 is attached to a fence surrounding the outer periphery of a facility to be monitored, and outputs an alarm signal in response to detecting a predetermined abnormality. As illustrated in FIG. 3, in this example, a cable-shaped vibration sensor (sensor cable) is used as the fence sensor 10. That is, the fence sensor 10 of this example outputs an alarm signal in response to detecting vibration of the fence. Identification information (for example, sensor number) of the fence sensor 10 is added to the alarm signal. The number of fence sensors 10 installed is arbitrary, and the more fences there are, the more fence sensors 10 are installed.

The imaging device 20 photographs an area including the fence (eg, the fence and its surroundings) from inside the fence (ie, from the facility side). Since the fence has transparency, even if the imaging device 20 is placed inside the fence, it is possible to photograph the outside of the fence. As illustrated in FIG. 3, in this example, the imaging device 20 is installed so as to take an image from a position facing the fence surface and overlooking the fence (a position higher than the fence). The number of imaging devices 20 installed is arbitrary, and the more fences there are, the more imaging devices 20 are installed. It is also possible to install the imaging device at the same height as the fence, so as to take pictures from a position where the fence surface is viewed in the lateral (horizontal) direction, as illustrated in Figure 2. In this case, it is difficult to determine whether the detected object (for example, a human being) is located at a position P1 close to the fence or at a position P2 far from the fence based only on the image. Therefore, it is preferable to install the imaging device 20 at a position higher than the fence, as shown in FIG. 3, so that it can be easily determined whether the detected object is close to the fence.

In this example, in order to simplify the explanation, a configuration in which one imaging device 20 is installed in correspondence with one fence sensor 10 will be described as an example. Note that a plurality of imaging devices 20 may be installed in correspondence to one fence sensor 10, or one imaging device 20 may be installed in correspondence to a plurality of fence sensors 10.

The recording device 30 stores video data captured by the imaging device 20. Further, in response to an external request, the stored video data is read out and transmitted to the request source (for example, the monitoring terminal device 60).

The server 40 analyzes the video captured by the imaging device 20 in response to the alarm from the fence sensor 10, detects an object (moving object) that appears near the fence, and determines whether it is an intruder. The server 40 of this example includes a sensor alarm reception section 41, a video reception section 42, an object contour extraction section 43, a boundary line determination section 44, an object area measurement section 45, a camera/sensor registration section 46, It includes a fence boundary line registration section 47, an area threshold registration section 48, an area comparison section 49, and an alarm information transmission section 50.

The camera/sensor registration unit 46 stores data indicating a correspondence relationship between the fence sensor 10 and the imaging device 20 that is set in advance. That is, the relationship between the fence sensor 10 and the imaging device 20 whose photographing range includes the fence to which the fence sensor 10 is attached is stored.

The fence boundary line registration unit 47 stores data on a preset boundary line at the lower end of the fence (that is, a boundary line between the fence and the ground) B1. The boundary line B1 at the lower end of the fence is used to determine whether an object detected from the image captured by the imaging device 20 exists in the immediate vicinity of the fence. This is because an intruder attempting to cross the fence will shake the fence while standing on the ground near the fence, so in that case, it is assumed that the intruder's feet are near the boundary line at the bottom of the fence. It's based on that. The boundary line B1 at the lower end of the fence is set for each fence to which the fence sensor 10 is attached.

The area threshold registration unit 48 stores data of area thresholds set in advance. The area threshold is
This is a threshold related to the area of an object detected from the image captured by the imaging device 20, and is used to determine whether the detected object is a human (i.e., an intruder) or another object (e.g., a small animal). used.

The sensor alarm receiving section 41 receives an alarm signal output from the fence sensor 10. Since the alarm signal has the identification information of the fence sensor 10 that is the transmission source added, it is possible to specify which fence sensor 10 has detected the abnormality.

The video receiving unit 42 transmits a video request signal to the imaging device 20 corresponding to the fence sensor 10 that is the source of the alarm signal in response to the sensor alarm receiving unit 41 receiving the alarm signal. , acquires video data from the imaging device 20. The imaging device 20 corresponding to the fence sensor 10 that is the transmission source of the alarm signal can be specified by referring to the data registered in the camera/sensor registration section 46.

The object contour extraction section 43 analyzes the video data received by the video reception section 42 and extracts the contour of an object (detected object) appearing in the video. The contour of the detected object can be obtained by various known methods. For example, the contour of the detected object may be obtained by detecting edge components from a difference image between a reference image prepared in advance and a photographed video.

The boundary line determining unit 44 compares the contour of the detected object extracted by the object contour extracting unit 43 and the boundary line B1 of the lower end of the fence registered in the fence boundary line registration unit 47, and determines whether the detected object is right next to the fence. Determine whether there is one nearby. That is, if at least a part of the outline of the detected object is within a predetermined distance from the boundary line B1 (including cases where they overlap), it is determined that the detected object is present in the immediate vicinity of the fence, and if it is not In this case, it is determined that the detected object does not exist in the immediate vicinity of the fence. As the distance between the contour of the detected object and the boundary line B1, for example, the number of pixels between the lower end of the contour of the detected object and the boundary line B1 can be used.

The object area measurement unit 45 measures the area inside the contour of the detected object extracted by the object contour extraction unit 43 when the boundary line determination unit 44 determines that the detected object exists in the immediate vicinity of the fence. do. The area inside the contour can be measured by various known methods. For example, the number of pixels inside the contour may be used as the area inside the contour. Since the distance between the imaging device 20 and the fence is constant, the area inside the contour of the detected object measured by the object area measurement unit 45 represents the size of the detected object located at approximately the same position as the fence. Become.

The area comparison unit 49 compares the area inside the contour of the detected object measured by the object area measurement unit 45 with the area threshold registered in the area threshold registration unit 48, and determines whether the detected object is a human (that is, an intruder). ). In other words, if the area inside the contour is larger than the area threshold, it is determined that the detected object is likely to be a human, otherwise it is determined that the detected object is likely to be another object (for example, a small animal). do.

The alarm information transmitting section 50 transmits alarm information according to the determination results of the boundary line determining section 44 and the area comparing section 49 to the monitoring terminal device 60. Specifically, if the boundary line determination unit 44 determines that the detected object is not located in the immediate vicinity of the fence, alarm information (hereinafter referred to as Send alarm A). Furthermore, if the boundary line determination unit 44 determines that the detected object is located very close to the fence, it is determined that the detected object is likely to be a small animal, according to the determination result by the area comparison unit 49. Alert information indicating that the detected object is likely to be a human (hereinafter referred to as alert C) is transmitted. The alarm information includes the alarm type, identification information (for example, sensor number) of the fence sensor 10 that detected the abnormality, identification information (for example, camera number) of the imaging device corresponding to the fence sensor 10, and the like.

The monitoring terminal device 60 notifies the fence sensor 10 of an alarm based on the alarm information transmitted from the server 40. The monitoring terminal device 60 of this example includes a video display section 61, an alarm information receiving section 62, and an alarm information display section 63.

The alarm information receiving unit 62 receives alarm information transmitted from the server 40.
The alarm information display unit 63 displays the alarm information received by the alarm information receiving unit 62 on the monitor of the monitoring terminal device 60.
The video display unit 61 requests the imaging device 10 (or the recording device 30) to transmit a captured video in response to an operation by the user of the monitoring terminal device 60, and displays the received video on the monitor of the monitoring terminal device 60.

FIG. 4 shows an example of a monitoring screen displayed on the monitor of the monitoring terminal device 60.
The monitoring screen 70 shown in FIG. 4 includes a video display area 71, a playback control button 72, a map area 73, an alarm information display area 74, an alarm confirmation button 75, a buzzer sound stop button 76, and an alarm history display area. 77.

The video display area 71 is an area that displays the video received from the imaging device 10 (or the recording device 30). The user can switch which imaging device 10 should display the captured video by operating the map display area 73. Note that when alarm information is received from the server 40, the display automatically switches to displaying a video image taken by the imaging device 10 corresponding to the alarm information. Additionally, a frame surrounding the detected object is displayed so that the user can easily find the detected object in the video. Furthermore, the display mode is changed depending on the type of alarm information so that the user can easily distinguish the type of alarm information. For example, the color of the area around the video display area 71 is changed to green for alarm A, yellow for alarm B, and red for alarm C.

The playback control button 72 is an operation unit that accepts a playback operation for the video displayed in the video display area 71 from the user. By operating the playback control button 72, the user can control playback, pause, fast forward, rewind, etc. of the video in the video display area 71.

The map display area 73 is an area that displays a map of the facility to be monitored, and a camera icon is displayed at a position corresponding to the installation location of the imaging device 20. By selecting the camera icon at the position where the user wants to display the video, the video shot by the imaging device 10 corresponding to the camera icon is displayed in the video display area 71. Note that when alarm information is received from the server 40, the mode of the corresponding camera icon is changed so that the user can easily identify the imaging device 10 corresponding to the alarm information. For example, the color of the camera icon is changed to green for alarm A, yellow for alarm B, and red for alarm C.

The alarm information display area 74 is an area for displaying alarm information received from the server 40 and related information. The alarm information display area 74 displays, for example, the number and size of detected intruders, emergency instructions, and the like.

The alarm confirmation button 75 is a button for canceling the alarm. After confirming the alarm information received from the server 40 and the image captured by the imaging device 10, the user operates the alarm confirmation button 75 to cancel the alarm.

The buzzer sound stop button 76 is a button for stopping the buzzer sound that sounds depending on the type of alarm information received from the server 40. The output of the buzzer sound is started, for example, when alarm C (high possibility of intruder) is issued, and the output is stopped when the user operates the buzzer sound stop button 76.

The alarm history display area 77 is an area that displays the history of alarm information received so far. In the alarm history display area 77, items such as the date and time of the alarm, the detection location of the intruder, and the type of alarm are displayed in a list format. The display in the alarm history display area 77 can change the sort order (display order) according to the user's operation. Furthermore, by selecting one of the alarm information histories, the video at that time can be acquired from the recording device 30 and displayed in the video display area 71.

FIG. 5 shows an example of a processing flow by the server 40. The server 40 performs the following processing in response to the alarm from the fence sensor 10.
First, the sensor alarm receiving section 41 receives an alarm signal from the fence sensor 10 (step S11). Next, the video receiving unit 42 acquires video data from the imaging device 20 corresponding to the fence sensor 10 that is the transmission source of the alarm signal (step S12). Next, the object contour extraction unit 43 analyzes the video data and extracts the contour of the detected object in the video (step S13). Next, the boundary line determination unit 44 compares the outline of the detected object with the boundary line B1 at the lower end of the fence, and determines whether the detected object is present in the immediate vicinity of the fence (step S14).

If it is determined that the detected object is not present in the immediate vicinity of the fence (step S14: No), it is determined that there is a high possibility of a false alarm caused by a natural phenomenon (step S17), and the alarm information transmitter 50 issues an alarm A indicating this (step S18).
On the other hand, if it is determined that the detected object is present in the immediate vicinity of the fence (step S14: Yes), the object area measurement unit 45 measures the area inside the contour of the detected object (step S15). Next, the area comparison unit 49 compares the area inside the outline of the detected object with an area threshold value to determine whether the detected object is a human (intruder) or a small animal (step S16).

If the area inside the contour is not larger than the area threshold (step S16: No), it is determined that the detected object is likely to be a small animal (step S19), and the alarm information transmitter 50 sends an alarm B indicating that. An alarm is issued (step S20).
On the other hand, if the area inside the contour is larger than the area threshold (step S16: Yes), it is determined that the detected object is likely to be a human (step S21), and the alarm information transmitter 50 sends an alarm C is issued (step S22).

FIG. 6 shows another embodiment in which an infrared sensor is used instead of the sensor cable. In the example shown in FIG. 6, the infrared sensor 12 is installed at the top of the fence. This is effective when detecting objects passing above the fence. In this case, the boundary line B2 at the upper end of the fence may be set in advance and registered in the fence boundary line registration section 47. This makes it possible to determine the type of object to be detected near the top end of the fence using the same method as when using the boundary line B1 at the bottom end of the fence.

As described above, the monitoring system of this example includes the fence sensor 10 attached to a fence surrounding a predetermined area, the imaging device 20 that photographs the area including the fence, and the imaging device 20 that photographs the area including the fence. The server includes a server 40 that analyzes images shot by the camera and detects objects appearing in the images, and a monitoring terminal device 60 that notifies alarms from the fence sensor 10. Then, the server 40 determines whether the detected object is an intruder based on the positional relationship between the fence and the detected object in the video and the area of the detected object in the video, and displays the determination result on the monitoring terminal device 60. It is configured as follows.

In this way, in the surveillance system of this example, it is automatically determined whether or not the detected object is an intruder based on the positional relationship between the detected object and the fence in the video. (user)'s workload can be reduced. This makes it easier to confirm when a large number of fence sensors 10 over a wide area simultaneously issue alarms due to natural phenomena, or when an intruder is detected at the same timing, and it is expected that the time required for confirmation work will be shortened.

In addition, in the monitoring system of this example, the server 40 detects that there is at least a part of the detected object in the vicinity of the boundary line B1 at the lower end of the fence (or the boundary line B2 at the upper end of the fence) in the video, and the area of the detected object is a threshold value. In the above cases, the detected object is determined to be an intruder. In this way, it is possible to determine whether the detected object is an intruder using a simple method, thereby reducing the processing load on the server 40. Furthermore, the cost of constructing a monitoring system can be reduced.

Further, in the monitoring system of this example, the monitoring terminal device 60 is configured to change the notification mode depending on the determination result of the server 40 as to whether the detected object is an intruder. Therefore, the supervisor can easily and specifically determine whether the detected object is an intruder or not.

In the above explanation, the video of the fence sensor 10 when the alarm is issued is analyzed, but by analyzing the video within a predetermined period before and after the alarm is issued, more accurate judgment can be made. possible. In short, it is sufficient to be able to analyze the video captured at the scene when the fence sensor 10 issues an alarm due to an object such as a human or a small animal (video taken at the time when the detected object is near the fence).

Here, in the monitoring system of this example, a fence sensor attached to a fence is exemplified as an installed object sensor, but it does not necessarily have to be a fence sensor. In other words, the installed object sensor according to the present invention is attached to an installed object (for example, a net, a fence, a fence, a wall, etc.) installed at the boundary of a predetermined area such as a facility or site, and is capable of sensing a predetermined physical action. It is possible to use various types of sensors (eg, infrared sensors, tension sensors, vibration sensors, acceleration sensors, etc.) that issue alarms depending on the situation.

Further, in the monitoring system of this example, the imaging device 20 is provided inside the installation object, but it does not necessarily have to be inside the installation object, and may be installed on the installation object line or outside the installation object. In other words, the imaging device 20 may be installed in any location as long as the positional relationship between the detected object and the installed object, the outline and area of the detected object, etc. can be detected from the image.

Although the present invention has been described in detail above, it goes without saying that the present invention is not limited to the configuration described above, and may be realized by a configuration other than the above.
The present invention also includes, for example, a method or system for executing the processing according to the present invention, a program for realizing such a method or system by a computer having hardware resources such as a processor or memory, and a program for implementing such a program. It is also possible to provide it as a storage medium for storing information.

INDUSTRIAL APPLICATION This invention can be utilized for the monitoring system which monitors intrusion into a predetermined area using an installation sensor and an imaging device.

10: Fence sensor, 20: Imaging device, 30: Recording device, 40: Server, 41: Sensor alarm reception section, 42: Video reception section, 43: Object contour extraction section, 44: Boundary line determination section, 45: Object Area measurement unit, 46: Camera/sensor registration unit, 47: Fence boundary line registration unit, 48: Area threshold registration unit, 49: Area comparison unit, 50: Alarm information transmission unit, 60: Monitoring terminal device, 61: Video display part, 62: alarm information receiving part, 63: alarm information display part

Claims (2)

an installation sensor attached to an installation surrounding a predetermined area;
an imaging device that photographs an area including the installed object in a diagonally upward direction;
a server that analyzes an image captured by the imaging device in response to an alarm from the installed object sensor and detects an object appearing in the image,
The server is
an object contour detection unit that detects the contour of the object in the image;
Comparing the outline of the object detected by the object outline detection unit with the boundary line of the lower end or the upper end of the installation object, if at least a part of the outline of the object is within a predetermined distance from the boundary line; , a boundary line determination unit that determines that the object is present in the vicinity of the installation, and if not, determines that the object is not present in the vicinity of the installation;
an object area measuring unit that measures an area inside a contour of the object detected from the image when the boundary line determining unit determines that the object exists in the vicinity of the installation object;
The area of the object measured by the object area measurement unit is compared with a threshold value, and if the area of the object is equal to or larger than the threshold value, it is determined that the object is likely to be an intruder; otherwise, the object is determined to be an intruder. A monitoring system comprising: an area comparison unit that determines that the object is highly likely not to be an intruder. The monitoring system according to claim 1,
Further comprising a monitoring terminal device that notifies the alarm of the installed object sensor,
The monitoring system is characterized in that the monitoring terminal device changes a notification mode depending on a determination result of the server as to whether or not the object is an intruder.

Images