KR20200065733A - Intelligent security system and method for detecting abnormal situation - Google Patents

Abstract

According to the present invention, provided is an abnormal situation detecting method in an intelligent security monitoring system, which comprises the steps of: receiving collected monitoring data from fixed or mobile monitoring equipment in a monitoring area; determining whether the collected monitoring data corresponds to a normal situation based on previously learned normal situation data; setting the collected monitoring data as an abnormality candidate when the collected monitoring data does not correspond to the normal situation; and transmitting the abnormal situation filtered by the abnormality candidate to a manager terminal.

Description

Intelligent security system and its abnormality detection method {INTELLIGENT SECURITY SYSTEM AND METHOD FOR DETECTING ABNORMAL SITUATION}

The present invention relates to an intelligent security system and an abnormality detection method thereof.

Recently, an intelligent security monitoring system has been introduced to improve the performance of video surveillance equipment and signal processing systems.

However, in most cases, the built-in algorithm operates simply and repeatedly, so it has a disadvantage that it is difficult to reflect the monitoring characteristics of the region or various requirements of the user.

In addition, in the process of storing, transmitting, and processing vast amounts of surveillance data, a lot of costs and problems arise.

According to these problems, there is a need for an intelligent security monitoring system that reflects the monitoring characteristics of the region, actively reflects the user's requirements, increases flexibility in various monitoring sites, reduces costs, and can continuously grow.

According to an embodiment of the present invention, surveillance data collected from surveillance devices using indoor and outdoor fixed CCTVs, mobile robots, and drones are abbreviated and converted, and anomalies are inferred to select candidates for anomalies, and the selected candidates for anomalies are filtered. On the other hand, it provides an intelligent security monitoring system that can improve the ability to detect anomalies by learning the feedback from the administrator, and a method for detecting the anomalies.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an abnormality detection method in an intelligent security monitoring system according to a first aspect of the present invention comprises the steps of receiving monitoring data collected from fixed or mobile surveillance equipment in a surveillance area; Determining whether the collected monitoring data corresponds to a normal situation based on previously learned normal situation data; Setting the abnormality candidate as the collected monitoring data does not correspond to the normal situation; And transmitting the abnormality filtered by the abnormality candidate to the manager terminal.

The surveillance data may include one or more of image, video, audio, and sensing data of the detection sensor.

An abnormality detection method according to the present invention includes the steps of storing the monitoring data in a database to correspond to the monitoring area; And learning the normal situation data through statistical analysis of monitoring data stored in the database.

The abnormality detection method according to the present invention further includes the step of abbreviating and converting the collected surveillance data into the same type, and storing the surveillance data in a database to correspond to the surveillance region comprises: It can be stored in the database.

The learning of normal situation data through statistical analysis of the surveillance data stored in the database may include: extracting normal situation data based on the frequency of occurrence of the stored surveillance data; And learning the extracted normal situation data.

Determining whether the collected monitoring data corresponds to a normal situation based on the learned result, and setting the abnormality candidate as the abnormal situation when the collected monitoring data does not correspond to the normal situation includes: Monitoring data outside the preset range based on the center of the data may be set as the candidate for the abnormality.

The abnormality detection method according to the present invention comprises the steps of sampling normalized situation data extracted with a predetermined probability or more among the extracted normal situation data; And storing the sampled normal situation data in a normal situation DB of the database.

An abnormality detection method according to the present invention comprises the steps of receiving feedback from the administrator terminal that the transmitted abnormality is incorrectly detected; And correcting the feedback received abnormal situation to a normal situation and storing it in the normal situation DB of the database.

In the step of transmitting the abnormality filtered by the abnormality candidate to the administrator terminal, the normality filter for filtering the abnormality candidate may be learned based on the normality DB.

The method for detecting an abnormality according to the present invention comprises the steps of receiving feedback from the manager terminal that the transmitted abnormality is not falsely detected; And it may further include the step of storing the feedback received in the abnormal situation DB of the database.

In the step of transmitting the abnormality filtered by the abnormality candidate to the administrator terminal, the abnormality filter for filtering the abnormality candidate based on the abnormality DB may be learned.

In addition, the intelligent security monitoring system capable of detecting an abnormal situation according to the second aspect of the present invention receives monitoring data from fixed or mobile monitoring equipment in a monitoring area, and transmits and receives data to and from an administrator terminal. And a processor for executing a program stored in the memory and a memory in which a program for detecting an abnormal condition is stored, wherein the processor executes the program to monitor the collected data based on previously learned normal situation data. It is determined whether data corresponds to a normal situation, and if it does not correspond to the normal situation, an abnormal situation in which the candidate for the abnormal condition is filtered is transmitted to the manager terminal through the communication module after setting the candidate as an abnormal situation.

The processor may convert the collected monitoring data into the same type and store it in a database of the memory to correspond to the monitoring area, and learn the normal situation data through statistical analysis of the monitoring data stored in the database.

The processor may extract and learn normal situation data based on the frequency of occurrence of monitoring data stored in the database.

The processor may sample the extracted normal situation data at a predetermined probability or higher among the extracted normal situation data, and store the sampled data in the normal situation DB of the database.

The processor may set monitoring data outside a preset range based on the center of the sampled normal situation data as the candidate for the abnormal situation.

The processor receives feedback from the manager terminal through the communication module whether or not the abnormal situation is detected as a false positive, and when the feedback result indicates that the abnormal situation is a false positive, the processor corrects the received abnormal situation to a normal situation, and If the abnormality is not detected as a result of the feedback, the abnormality received in the feedback may be stored in the abnormality DB of the database.

According to any one of the above-described problem solving means of the present invention, it is possible to self-learn the appropriate situation suitable for the characteristics of the corresponding region by learning the data of the corresponding monitoring region in the abnormality inference step through the reduced transformation step of monitoring information.

In addition, the error occurrence rate may be continuously reduced by receiving feedback from the manager when the determination is an error through the abnormality filtering step.

In addition, by continuously storing and expanding feedback data for an abnormal situation and a normal situation, and updating a filter for filtering the abnormal condition based on this, there is an effect that it is possible to detect various abnormal conditions while having a low error rate.

1 is a block diagram of an intelligent security monitoring system according to an embodiment of the present invention.
2 is a flowchart of an abnormality detection method according to an embodiment of the present invention.
3 to 7 are diagrams for explaining an example in which the surveillance data is abbreviated and converted.
FIG. 8 is a diagram for explaining a process of performing statistical analysis on normal and abnormal conditions of the collected monitoring information.
9 is a view for explaining an example of improving the abnormality detection performance through feedback.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains may easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention, parts not related to the description are omitted.

When a part of the specification "includes" a certain component, it means that other components may be further included rather than excluding other components, unless otherwise stated.

1 is a block diagram of an intelligent security monitoring system 100 according to an embodiment of the present invention.

An intelligent security monitoring system capable of detecting an abnormal situation according to an embodiment of the present invention includes a communication module 110, a memory 120, and a processor 130.

The communication module 110 receives surveillance data from fixed or mobile surveillance equipment in the surveillance area. In addition, the communication module 110 transmits and receives data to and from the manager terminal. The communication module may include a wireless communication module. The wireless communication module may be implemented with wireless LAN (WLAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, WiFi wireless USB technology, and wireless HDMI technology.

At this time, in the monitoring equipment in one embodiment of the present invention, a fixed CCTV, a mobile robot, a mobile drone and various detection sensors such as infrared and ultrasonic sensors may correspond to this. Surveillance data collected from such surveillance equipment may include one or more of image, video, audio, and sensing data of a detection sensor.

A program for detecting an abnormal condition based on monitoring data is stored in the memory 120, and the processor 130 executes the program stored in the memory 120. At this time, the memory 120 is a non-volatile storage device and a volatile storage device that keeps stored information even when power is not supplied.

For example, the memory 120 is a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD), and a micro SD NAND flash memory such as cards, magnetic computer storage devices such as hard disk drives (HDDs), and optical disc drives such as CD-ROMs, DVD-ROMs, and the like. Can be.

As the processor 130 executes the program stored in the memory 120, it determines whether the collected monitoring data corresponds to the normal situation based on the previously learned normal situation data, and does not respond to the normal situation. If not, after setting the candidate as an abnormality candidate, the abnormality filtered by the abnormality candidate may be transmitted to the manager terminal through the communication module 110.

For reference, the components shown in FIG. 1 according to an embodiment of the present invention may be implemented in software or in a hardware form such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and perform predetermined roles. can do.

However,'components' are not limited to software or hardware, and each component may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors.

Thus, as an example, a component is a component, such as software components, object-oriented software components, class components, and task components, and processes, functions, attributes, procedures, subs. Includes routines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays and variables.

Components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

Hereinafter, an abnormality detection method performed in the intelligent security monitoring system 100 according to an embodiment of the present invention will be described in detail with reference to FIGS. 2 to 9.

2 is a flowchart of an abnormality detection method according to an embodiment of the present invention. 3 to 7 are diagrams for explaining an example in which the surveillance data is abbreviated and converted. FIG. 8 is a diagram for explaining a process of performing statistical analysis on normal and abnormal conditions of the collected monitoring information. 9 is a view for explaining an example of improving the abnormality detection performance through feedback.

The abnormality detection method according to an embodiment of the present invention is largely performed by a monitoring information collection step (S110), an abnormality reasoning step (S120), and an abnormality filtering and transmission step (S130).

First, in connection with the monitoring information collection step (S110), the communication module 110 of the intelligent security monitoring system 100 receives the monitoring data collected from fixed or mobile monitoring equipment in the monitoring area.

For example, in the monitoring information collection step (S110), various surveillance data including video, noise, and various sensors are collected in the surveillance area through surveillance equipment such as a fixed CCTV, mobile robot, and drone.

At this time, in the monitoring information collection step (S110), a step of abbreviating and converting the collected monitoring data into the same type as shown in FIG. 3 may be additionally performed (S115). That is, the monitoring data collected by each monitoring device may be reduced to the entire surveillance area as shown in FIG. 3 and transmitted to the intelligent security monitoring system 100.

Specifically, in the step of abbreviating and converting the monitoring data into the same type, the monitoring area is partitioned into a grid shape as shown in FIG. 4, and event occurrence information in the grid-shaped partition is accumulated. For example, in the case of a pedestrian, the number of people (density), moving speed, moving direction, and posture of a person (walking, standing, sitting, running, lying down, etc.) can be accumulated as event occurrence information.

And the probability of occurrence of the event in the area is calculated for the event.

Next, as shown in the lower left black portion of FIG. 5, it is possible to display the behavior observation information of the speed, direction, and density of the pedestrian. Probability, direction occurrence probability can be displayed. At this time, the probability of occurrence means that the darker the color of the displayed square, the higher the probability.

Next, it is possible to determine the past action history by listing the abbreviated information as shown in the right side of FIG.

Referring back to FIG. 3, in FIG. 3 (a) to (d), the black portion on the bottom left of each side is a reduced transformation of density, speed, and direction information of each pedestrian. In the case of (a) of FIG. 3, 5 persons, and in the case of (c) of FIG. 3, 4 persons indicate which action is being performed at which position, and the white part at the bottom right of each drawing indicates the probability of occurrence of the event It shows how much.

Referring to FIG. 6 as an example of another abbreviated transformation, observation values for a vehicle as well as a person may be expressed together. In the case of FIG. 6, the probability of occurrence is not separately indicated.

Referring to FIG. 7 as another reduction conversion example, an embodiment of the present invention may reduce and convert the monitoring information such as a structure and a temperature change of a corresponding monitoring area.

As described above, according to an embodiment of the present invention, information such as multi-modal monitoring information such as a person, a vehicle, a structure of a monitoring area, and temperature, and an occurrence probability may be expressed in a reduced grid map form.

Accordingly, one embodiment of the present invention can express a vast amount of multi-modal monitoring information (person, vehicle, temperature, structure, etc.) of the monitoring area in a consistent and condensed form. In particular, it is easy to store, compare, and process statistical information, such as the characteristics of occurrence of events in the surveillance area, to improve the performance of judging anomalies and to significantly reduce the cost of building a system.

Next, in the abnormality reasoning step (S120 ), it is inferred whether the collected monitoring data corresponds to the normal situation based on the previously learned normal situation data (S125 ).

In the abnormality reasoning step (S120), previously learned normal situation data is used to infer whether the collected monitoring data corresponds to a normal situation, that is, how far the monitoring information established at the monitoring point deviates from the normal situation. However, for this purpose, the monitoring data collected in the monitoring information collection step (S110) is stored in a database to correspond to the monitoring area (S121). At this time, the data stored in the database may be abbreviated conversion monitoring data.

In addition, the normal situation data may be generated by learning the normal situation data through statistical analysis of the monitoring data stored in the database (S123 ). At this time, an embodiment of the present invention is a statistical learning method, and can perform an unsupervised learng such as an autoencoder on abbreviated transformed monitoring data to infer an abnormal situation statistically deviating from a normal category. have.

Referring to FIG. 8, statistical analysis can extract normal situation data from the surveillance data based on the frequency of occurrence of the surveillance data stored in the database, and perform learning on the extracted normal situation data.

For example, as a result of accumulating and accumulating occurrence frequencies in FIG. 8, a monitoring data area having a normal distribution graph may be extracted as normal situation data to perform learning.

By performing statistical analysis on the reduced transformed monitoring data, it is possible to determine whether or not the currently occurring event is a statistically rare event.

In the case of FIG. 8, the normal situation monitoring data portion P1 represents a score calculation result of performing statistical analysis on an ordinary event, and the abnormal condition monitoring data portion P2 is a score calculation that performs statistical analysis on an arbitrarily directed abnormal situation. The results are shown.

As can be seen from the graph of FIG. 8, it is possible to determine whether or not to report to an administrator terminal later by setting an candidate for an abnormal situation for an event outside the normal range.

Based on the learned normal situation data, monitoring data outside a preset range based on the center of the normal situation data may be set as an abnormality candidate (S125).

Next, in the filtering and transmission of an abnormal situation (S130), filtering using a supervised learning method such as a convolutional neural network (CNN) is performed on the candidate for the abnormal situation (S131), and the normal situation is performed on a stricter basis. And report the abnormality to the administrator terminal.

At this time, when the intelligent security monitoring system 100 judges and reports the normal situation to the administrator terminal as an abnormal situation, the administrator terminal feeds back to the intelligent security monitoring system 100 that the abnormal situation is falsely detected (S132).

The intelligent security monitoring system 100, which is fed back from the administrator terminal that the report of the current abnormality is false, corrects the abnormality to a normal situation and stores it in the normal situation DB of the database (S133).

In this process, the intelligent security monitoring system 100 may automatically generate a similar case of an abnormal situation that is detected as a normal situation to expand the data of the normal situation DB (S134).

In addition, in the normal situation DB, normal situation data extracted with a predetermined probability or higher among the normal situation data extracted in the abnormality reasoning step S120 may be sampled and sampled normal situation data may be stored. That is, the data stored in the normal situation DB can be expanded by sampling the surveillance information determined to be normal with a high probability in the abnormality reasoning step S120 and storing it in the normal situation DB.

When the abnormality transmitted by the intelligent security monitoring system 100 to the manager terminal is correct, the manager terminal feedbacks that the abnormality is not falsely detected by the intelligent security monitoring system 100 (S132).

Then, the intelligent security monitoring system 100 fed back from the administrator terminal stores the current abnormal situation in the abnormal situation DB of the database (S135).

In this process, the intelligent security monitoring system 100 may automatically generate a case similar to the abnormal situation to expand the data of the abnormal situation DB (S136).

The generated normal situation DB and the abnormal situation DB may be used for filter learning to filter the normal situation and the abnormal situation (S137). And the learned normal situation and abnormal condition filter is applied to the above-described abnormal condition filtering process (S131).

In this process, the normal situation DB and the abnormal situation DB continuously receive feedback from the administrator terminal, and at the same time, through the data expansion process, the possibility of error detection is lowered and the applicability to various monitoring situations is increased, and these normal situation DB and abnormal situation DB Through this, the normal and abnormal filters are learned and updated.

Through this process, it can be seen through the feedback as shown in FIG. 9 that the erroneous detection of the system for the actual monitoring data is rapidly reduced over time. For reference, the case of FIG. 9 shows an analysis result between 2018.6.7. and 2018.6.12.

Meanwhile, in the above description, steps S110 to S137 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted if necessary, and the order between the steps may be changed. In addition, even if omitted, the content already described in FIG. 1 may be applied to the method for detecting an abnormality in FIGS. 2 to 9.

According to one embodiment of the present invention, the monitoring information is reduced through the conversion step to learn the data of the corresponding monitoring area in the abnormality reasoning step by itself, and it is possible to detect an abnormal situation suitable for the characteristics of the corresponding area.

In addition, the error occurrence rate may be continuously reduced by receiving feedback from the manager when the determination is an error through the abnormality filtering step.

In addition, by continuously storing and expanding feedback data for an abnormal situation and a normal situation, and updating a filter for filtering the abnormal condition based on this, there is an effect that it is possible to detect various abnormal conditions while having a low error rate.

Meanwhile, an embodiment of the present invention may be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The above description of the present invention is for illustration only, and a person having ordinary knowledge in the technical field to which the present invention pertains can understand that it can be easily modified to other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

100: intelligent security monitoring system
110: communication module
120: memory
130: processor

Claims (17)

In the abnormality detection method in the intelligent security monitoring system,
Receiving collected surveillance data from fixed or mobile surveillance equipment in the surveillance area;
Determining whether the collected monitoring data corresponds to a normal situation based on previously learned normal situation data;
Setting the abnormality candidate as the collected monitoring data does not correspond to the normal situation; And
And transmitting the abnormality filtered by filtering the abnormality candidate to an administrator terminal. According to claim 1,
The monitoring data includes an image, video, audio and one or more of the sensing data of the detection sensor abnormality detection method. According to claim 1,
Storing the monitoring data in a database to correspond to the monitoring area; And
And learning the normal situation data through statistical analysis of monitoring data stored in the database. According to claim 2,
Further comprising the step of abbreviating and converting the collected monitoring data to the same type,
The step of storing the monitoring data in a database to correspond to the monitoring area is to store the reduced-converted monitoring data in the database. According to claim 2,
The step of learning the normal situation data through statistical analysis of the monitoring data stored in the database,
Extracting normal situation data based on the frequency of occurrence of the stored monitoring data; And
And learning the extracted normal situation data. The method of claim 5,
The step of determining whether the collected monitoring data corresponds to a normal situation based on the learned result, and setting the candidate as an abnormal situation when the collected monitoring data does not correspond to the normal situation include:
A method of detecting an abnormality, wherein monitoring data outside a preset range is set as the abnormality candidate based on the center of the normal situation data. The method of claim 6,
Sampling the normal situation data extracted above a preset probability among the extracted normal condition data; And
And storing the sampled normal situation data in a normal situation DB of the database. According to claim 1,
Receiving feedback from the manager terminal that the transmitted abnormality is incorrectly detected; And
And correcting the feedback abnormal situation to a normal situation and storing it in a normal situation DB of the database. The method of claim 7 or 8,
The step of transmitting the abnormality filtered by the abnormality candidate to the manager terminal is:
An abnormality detection method for learning a normality filter for filtering the abnormality candidate based on the normality DB. According to claim 1,
Receiving feedback from the manager terminal that the transmitted abnormality is not misdetected; And
And storing the feedback abnormal situation in an abnormal situation DB of the database. The method of claim 10,
The step of transmitting the abnormality filtered by the abnormality candidate to the manager terminal is:
An abnormality detection method for learning an abnormality filter for filtering the abnormality candidate based on the abnormality DB. In the intelligent security monitoring system capable of detecting an abnormal situation,
A communication module that receives surveillance data from fixed or mobile surveillance equipment in the surveillance area and transmits and receives data to and from the manager terminal,
A memory in which a program for detecting an abnormal condition is stored based on the monitoring data, and
It includes a processor for executing a program stored in the memory,
As the processor executes the program, it determines whether or not the collected monitoring data corresponds to a normal situation based on previously learned normal situation data, and sets a candidate for an abnormal situation if it does not correspond to the normal situation. Intelligent security monitoring system that transmits the abnormality filtered after the candidates to the manager terminal through the communication module. The method of claim 12,
The processor converts the collected monitoring data into the same type and stores it in the database of the memory to correspond to the monitoring area, and learns the normal data through statistical analysis of the monitoring data stored in the database. Security monitoring system. The method of claim 13,
The processor is based on the frequency of occurrence of the surveillance data stored in the database, intelligent security monitoring system to learn by extracting the normal situation data. The method of claim 14,
The processor is to extract the normal situation data extracted above a predetermined probability among the extracted normal situation data, the intelligent security monitoring system that stores the sampled data in the normal situation DB of the database. The method of claim 15,
The processor is to set the monitoring data outside the predetermined range based on the center of the sampled normal situation data, the intelligent security monitoring system to set the candidate for the abnormal situation. The method of claim 12,
The processor receives feedback from the manager terminal through the communication module whether or not the transmitted abnormality is false detection,
When the abnormality is detected as a result of the feedback, the abnormality received by the feedback is corrected to a normal situation and stored in a normal situation DB of the database.
An intelligent security monitoring system for storing the feedback received abnormal situation in the abnormal DB of the database when the abnormal result is not detected as a result of the feedback.

Images