KR102152237B1 - Cctv central control system and method based on situation analysis - Google Patents

Abstract

A CCTV control method and system based on situation analysis are disclosed. According to the present invention, the CCTV control system based on situation analysis comprises: a first image analysis device; a second image analysis device; and an integrated control server. The first image analysis device controls the operation of a camera based on first metadata, and the second image analysis device determines whether or not an event has occurred using at least one of the first metadata and second metadata. According to the present invention, since the control is based on a movement of an object and a change in a situation, it is possible to probably reduce false information in event detection.

Description

CCTV control method and system based on situation analysis {CCTV CENTRAL CONTROL SYSTEM AND METHOD BASED ON SITUATION ANALYSIS}

The present invention relates to a CCTV control method and system based on situation analysis, and more particularly, to a CCTV control method and system based on object detection at the edge and situation analysis in the cloud.

Looking at the main use status of existing CCTV facilities installed in residential and business facilities, it is not possible to respond immediately at the time of event occurrence, and it is used to analyze the cause of the event and process the result through the image stored after the occurrence. Therefore, there were many cases where it was difficult to prevent accidents and crimes and take immediate measures with existing CCTV facilities.

The early days of intelligent CCTV were rule-based, but rule-based intelligent CCTV means detecting moving objects by detecting moving pixels in an image. However, rule-based intelligent CCTV could not distinguish between people and objects, and there was a difficulty in manually setting the detection level. In the rule-based intelligent CCTV, a notification should sound when a person passes by, but dozens of misinformations occur a day, such as when a dog or cat passes, when fallen leaves are blown by the wind, when a tree is blowing in the wind, or even when noise occurs. did. So, even with the intelligent CCTV installed, it was necessary to rely on visual control without triggering the alarm.

As one related technology, an intelligent collaboration server, a system, and a collaboration-based analysis method thereof are disclosed in the publication of registration number KR 10-2008503. According to this related technology, information analysis by various analysis techniques is practically impossible, and when the first image analysis information and the second image analysis information conflict with each other, there is no method of selecting an analysis method with high reliability. . In addition, the above related technology has a disadvantage in that it is limited to generating camera control information based on image analysis.

KR Registered Patent Publication No. 10-2008503 (published on August 1, 2019)

An object of the present invention is to solve the problem of the conventional intelligent CCTV control system, which had a lot of misinformation due to initial CCTV and object motion detection, which did not immediately respond.

An object of the present invention is to distinguish between a CCTV for surveillance installed in a residential or business facility with little movement of an object and a CCTV for traffic control installed at an intersection with many movements of an object, so that different systems are applied to the conventional CCTV control system. In order to solve the problems of the system, it is to provide an intelligent CCTV control system that can be applied to both CCTV installed in residential and business facilities and CCTV for traffic control.

An object of the present invention is to provide a CCTV control system that can utilize an artificial intelligence model whose performance can be updated through transfer learning at the edge and in the cloud.

The subject of the present invention is not limited to the problems mentioned above, and other problems and advantages of the present invention not mentioned will be more clearly understood by the following examples. In addition, it will be appreciated that the problems and advantages of the present invention can be realized by the means shown in the claims and combinations thereof.

A CCTV control system based on situation analysis according to an embodiment of the present invention includes at least one camera for collecting and transmitting a surveillance image; A first image analysis device for generating first metadata through a first image analysis based on a motion of an object detected from a surveillance image; A second image analysis device that generates second metadata through a second image analysis based on a context sensed in a surveillance image, in which the first metadata is used; And an integrated control server that displays the surveillance image through the integrated control viewer and notifies the occurrence of an event of a preset condition. In addition, the first video analysis device controls the operation of the camera based on the first metadata, the integrated control server controls the operation of the camera based on the second metadata, and the first metadata and the second metadata Using data, it is possible to determine whether an event has occurred.

In addition, the first image analysis apparatus may extract first metadata about the object by extracting an object from the surveillance image and matching the learned pattern to the extracted object through object classification based on an artificial intelligence model.

In addition, the second image analysis device recognizes the background by applying a deep learning model to the features of the background extracted from the surveillance image, and the situation inferred through the motion of the object in the background using the first metadata. Second metadata about the change can be extracted.

In addition, the first image analysis apparatus may detect a motion of a signal violation of a vehicle or a person using at least one of a color of a lighted signal and signal information received from a traffic signal controller.

In addition, according to claim 1, wherein the second image analysis device recognizes at least one of a child protection zone, a crosswalk, and an intersection by applying an artificial intelligence model trained by learning to a background extracted from the surveillance image, It is possible to determine whether or not the situation inferred through the motion of the vehicle and the person in the recognized background changes using the metadata.

In addition, as a method performed by the integrated control server, the CCTV control method based on situation analysis is a model and situation that determines the movement of an object based on training based on training using a training data set composed of images collected through CCTV cameras. Constructing a model for determining change; Receiving a surveillance video from the CCTV camera to notify the occurrence of an event; Receiving first metadata based on the first image analysis based on the motion of the object detected in the surveillance image by using the model for determining the motion of the object; Receiving second metadata based on the second image analysis based on the context detected in the surveillance image by using the first metadata and a model for determining the change of the situation; And displaying the surveillance image through the integrated control viewer and notifying the occurrence of an event of a preset condition. In addition, it may be determined whether an event occurs using at least one of the first metadata and the second metadata.

In addition, the CCTV control method based on situation analysis may be configured to further include controlling the operation of the camera. Further, the operation of the camera at the edge may be controlled based on the first metadata, and the operation of the camera may be controlled by the cloud based on the second metadata.

According to the present invention, since it is based on the movement of the object and the change in the situation, it is possible to probabilistically reduce the false information in event detection.

According to the present invention, one event detection algorithm can be applied to both CCTV for crime prevention and CCTV for traffic control.

According to the present invention, the control intelligence of CCTV can be gradually improved through transfer learning at the edge of the artificial intelligence model and in the cloud.

1 is a block diagram of a CCTV control system according to the prior art.
2 is an exemplary diagram of a network environment connected to a CCTV control system based on situation analysis according to an embodiment of the present invention.
3 is a block diagram of a CCTV control system based on situation analysis according to an embodiment of the present invention.
4 is a block diagram of an image analysis apparatus according to an embodiment of the present invention.
5 is a detailed block diagram of a CCTV control system based on situation analysis according to an embodiment of the present invention.
6 is a flowchart of a CCTV control method based on situation analysis according to an embodiment of the present invention.
7 is a flowchart of a CCTV control method based on situation analysis according to an embodiment of the present invention.
8 is a flowchart of a CCTV control method based on situation analysis according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all modifications included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various elements, but the elements are not limited by the terms. These terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

1 is a block diagram of a CCTV control system according to the prior art.

Referring to FIG. 1, an image analysis server performing a server-based image analysis method is illustrated. The video analysis server processes video streams transmitted from n cameras, and among the video streams of n channels, video streams related to events are intensively displayed through the video management server, and alarms are displayed. It works. The object detection technology performed by the video analysis server is based on machine learning, and can recognize and extract monitoring, dangerous situations, or unusual matters based on learned behavior patterns.

Server-based video analysis method is a centralized structure that performs intelligent video analysis by receiving video from a camera, and it is easy to apply to existing systems because it does not depend on cameras, but because the number of cameras that can be processed by one server is limited. There is a problem with scalability.

And the problem of server-based image analysis is that it is not the case of recognizing and extracting the same or similar situation or object as the learning pattern, but recognizing the occurrence of object change related to the change of unexpected situation, and excessive events occurring in object extraction. The false positive rate increases.

In addition, since the video analysis server analyzes and processes video information transmitted through a plurality of channels, a network and system load problem may occur due to excessive video data transmission and processing.

In addition to the server-based image analysis method, the edge-based image analysis method is a method that analyzes the image in a camera or an image encoder connected to the camera, and has the advantage of analyzing the image without distortion of the signal that occurs during the compression process to transmit the image. There is this.

However, since the computing power of the embedded device is limited, there is a problem that the image analysis technique with high complexity cannot be applied and it is difficult to upgrade the function later.

The situation-based CCTV control method and the control system 100 using the same according to an embodiment of the present invention are characterized by a hybrid image analysis method including both edge-based image analysis and server-based image analysis. That is, the situation-based CCTV control method is two methods to solve problems such as high network bandwidth usage, the number of channels that can be processed per server, and the limitation of application of high-level video analysis techniques with the device operation capability of the edge-based method. It corresponds to a hybrid-based method that combines the advantages of.

Context-based CCTV control system 100 detects the movement of an object at the edge in image analysis, detects a situation change based on the movement of the object, and determines whether an event occurs based on the movement of the object and the change of the situation. can do.

2 is an exemplary diagram of a network environment connected to a CCTV control system based on situation analysis according to an embodiment of the present invention.

Referring to FIG. 2, the CCTV control system 100 based on situation analysis is configured to include a camera 110, a first video analysis device 120, a second video analysis device 130, and an integrated control server 150. Can be.

The camera 110 may be installed in various places according to various uses of the CCTV control system 100. For example, the camera 100 may be installed at an intersection to collect traffic data, count passing vehicles, check the direction of the vehicle, and detect abnormal stops of the vehicle, or detect a caution situation in a specific area, detect an intruder in a specific area, or It may be installed on private or public buildings or streets for smoke detection, entrance to gate detection, and roaming tracking, or in school facilities or child protection zones to detect violence, child collision, and crosswalk safety accidents. .

The camera 110 may be configured to include at least one type of a camera to which an IP address has not been assigned and an IP camera to which an IP address has been assigned on a network. A camera to which an IP address is not assigned may be connected to a video encoding device to which an IP address is assigned. Accordingly, the IP camera or the image encoding device may transmit the encoded video streaming to the first image analysis device 120 and the second image analysis device 130.

The first image analysis device 120 may analyze an image based on the motion of the object. On the other hand, the second image analysis device 120 may analyze an image based on a change in situation. Here, the first image analysis device 120 may be configured to be included in the camera 110 in the form of an embedded system. In order to detect the occurrence of an event, the first image analysis device 120 performs a first image analysis necessary to determine the movement of an object related to the occurrence of the event, and the second image analysis device 130 performs a situation change related to the event occurrence. Image analysis necessary to determine

In addition, the first image analysis device 120 and the second image analysis device 130 store an artificial neural network as an artificial intelligence model, for example, an artificial intelligence model, so that an image can be analyzed using the artificial intelligence model. The artificial intelligence model used by is characterized by being trained separately by learning focusing on the movement of objects and learning focusing on changes in the situation. Here, the second image analysis device 130 may generate the second metadata by using the first metadata that is the result of image analysis by the first image analysis device 120.

The second image analysis apparatus 130 finally determines whether the movement of the object is connected to the occurrence of an event by using the first metadata and the second metadata.

The integrated control server 150 may display an alarm of occurrence of an event, and display a corresponding image through the integrated control viewer 160. In addition, the operation of the camera 110 may be controlled using the received second metadata. In addition, the integrated control server 150 may store various settings required for image analysis in the first image analysis device 120 and the second image analysis device 130.

The CCTV system 100 based on reimbursement analysis may be configured to additionally include an integrated control viewer 160, an image storage device (NVR) 170, a metadata storage device 180, and the like. That is, the integrated control viewer 160 may be directly connected to the integrated control server 150 or configured as a separate server, and the video streaming may be stored in the NVR 170 corresponding to a network video recorder. And, the first metadata generated by the first image analysis apparatus 120 and the second metadata generated by the second image analysis apparatus 130 may be stored in a separate metadata storage device 180.

The camera 110 is related to a PTZ camera, and operations for rotation change, tilt change, and magnification change may be controlled. The first image analysis apparatus 120 may be connected to the camera 110 in the edge region and directly control the operation of the camera 110 using first metadata, which is a result of image analysis focusing on object movement. On the other hand, the integrated control server 150 may control the operation of the camera 110 by using the second metadata that is a result of image analysis focusing on a change in situation. In addition, the camera 110 may be directly controlled by an administrator with the help of the integrated control viewer 160.

The network 190 is a wired and wireless network, for example, a local area network (LAN), a wide area network (WAN), an Internet, an intranet and an extranet, and a mobile network, for example. It may be any suitable communication network including cellular, 3G, LTE, 5G, WiFi networks, ad hoc networks, and combinations thereof.

Network 190 may include a connection of network elements such as hubs, bridges, routers, switches and gateways. Network 190 may include one or more connected networks, such as a multi-network environment, including a public network such as the Internet and a private network such as a secure corporate private network. Access to network 190 may be provided through one or more wired or wireless access networks.

3 is a block diagram of a CCTV control system based on situation analysis according to an embodiment of the present invention.

3, a camera constituting a CCTV control system 100 based on situation analysis, an edge intelligent video analysis device 120, a situation-based video analysis device 130, an integrated control server 150, an integrated control viewer ( 160), an image storage device 170, and a metadata storage device 180 are shown. Here, the first image analysis device may be referred to as an edge intelligent image analysis device, and the second image analysis device may be referred to as a context-based image analysis device.

Images are collected through the n cameras 110, and the collected images and camera information may be transmitted to the edge intelligent image analysis device 120 and the context-based image analysis device 130. The edge intelligent image analysis apparatus 120 may extract first metadata through image analysis focusing on the movement of an object, and control information included in the first metadata may be transmitted to the camera 110.

The edge intelligent image analysis device 120 analyzes the image received from the camera 110 according to the image analysis setting, focusing on determining the motion of the object. Through the first analysis, first metadata including information on time, camera, location, and color of the object related to the movement of the object may be transmitted to the context-based image analysis apparatus 130.

The context-based image analysis apparatus 130 may generate second metadata through a second image analysis focusing on determining a situation change while using the received first metadata. The second metadata may include information on the identity of the moved object, the type of the recognized background, the composition and recognition of the scene according to the movement of the object background, and the situation change according to the situation inference and analysis.

The first metadata and the second metadata may be stored in a separate metadata storage device 180 and managed by the integrated control server 150.

The integrated control server 150 may display an alarm using the result of image analysis, that is, information on event occurrence, and bookmark the event occurrence of the corresponding image. In addition, the integrated control server 150 may store the image received from the context-based image analysis device 130 in a separate image storage device 170 and manage it.

In addition, the integrated control server 150 may display a corresponding image together with image analysis information through the integrated control viewer 160. In addition, the integrated control server 150 may store environment settings required for image analysis in the edge intelligent image analysis device 120 and the context-based image analysis device 130.

The CCTV control system 100 based on situation analysis has a feature that image analysis focusing on object motion detection and image analysis focusing on detection of changes in the situation are performed by different video analysis devices. Storage, metadata storage, and image display functions may be performed by one or more servers corresponding to a computer.

The first image analysis device 120, the second image analysis device 130, and the integrated control server 150 may also be implemented in a computer form. Hereinafter, a description will be given of a second image analysis device that can be implemented as a computer representing various devices.

4 is a block diagram of an image analysis apparatus according to an embodiment of the present invention.

4, the second image analysis apparatus 130 includes a processor 131, a memory 138, an input device 139, a display device 140, a data bus 141, a transceiver 142, and a storage device. It can be configured to include (143).

The processor 131 may execute a program module stored in the memory 138. Through the execution of the program module, commands related to various instruction sets included in the program module are executed, and as a result, the step of forming a context-based CCTV control method according to an embodiment of the present invention is performed by the second image analysis device 130. Can be done by

The transceiver 142 may receive an image collected through the camera 110 and transmit the image to the integrated control server 150. In addition, the transceiver 142 may transmit the second metadata generated by the processor 131 to the integrated control server 150. The transceiver 142 may be implemented in the form of a network communication card such as Ethernet.

Various data required for image analysis by the second image analysis device 130 may be transmitted between various components therein through the data bus 142.

Hereinafter, an algorithm performed by the edge intelligent image analysis device 120 and the context-based image analysis device 130 that can be implemented as a computer through a processor and a memory will be described in detail.

5 is a detailed block diagram of a CCTV control system based on situation analysis according to an embodiment of the present invention.

Referring to FIG. 5, the edge intelligent image analysis device 120, the context-based image analysis device 130, and each processor 121, 131, 151 of the integrated control server 150 and various program modules stored in the memory are executed. The CCTV control method is shown.

The edge intelligent image analysis device 120 may receive an image and camera information from the camera 110 and transmit camera control information to the camera 110. The processor 121 may set an environment in relation to the first image analysis, once the intelligent image analysis device 120. Next, the processor 121 may extract the object through the control of the corresponding program module (122). Next, the processor 121 may match the image and the learning pattern through the control of the corresponding program module (123). Next, the processor 121 may extract metadata through the control of the corresponding program module (124). The metadata extracted by the processor 121 of the edge intelligent image analysis device 120 may be referred to as first metadata. The first metadata may be transmitted to the context-based image analysis device 130.

The first metadata may include information on movement of an object occurring in an image. For example, the first metadata may include information on the appearance position of the object, the movement time of the object, the size of the object, and the color of the object.

The context-based image analysis device 130 may directly receive an image and camera information from the camera 110. In addition, the context-based image analysis device 130 may receive an image together with the first metadata from the edge intelligent image analysis device 120.

The integrated control server 150 may display an alarm of occurrence of an event and a notification of an image requiring confirmation by using the image analysis result by the context-based image analysis device 130. The integrated control server 150 may receive the second metadata and transmit camera control information included in the second metadata for controlling the operation of the camera 110 to the camera 110.

6 is a flowchart of a CCTV control method based on situation analysis according to an embodiment of the present invention. Referring to FIG. 6, a CCTV control system based on situation analysis according to an embodiment of the present invention 100 The first metadata as a result of the first image analysis focusing on determining the motion of the object, the step of constructing a model for determining the motion and a model for determining the situation change (S110), receiving a surveillance image (S120), and Generating (S130), generating second metadata as a result of the second image analysis focusing on determining the situation change (S140), and displaying the surveillance image, and displaying the first metadata and the second metadata. It may be configured to include a step (S150) of notifying the occurrence of an event based on data.

According to an embodiment of the present invention, each model may be separately configured by dividing a model for determining the movement of an object and a model for determining a situation change (S110). Here, the model for determining the motion of the object may be driven by the first image analysis device 120 connected to the edge due to the characteristic of the function. In particular, the first image analysis device 120 may be implemented in the form of an embedded system inside each camera 110. In addition, the model for determining the change of the situation may be driven by understanding the second image analysis device 130 connected to the cloud due to the characteristic of the function.

The surveillance image may be collected by the camera 110 installed at the selected location. The surveillance image is an object of image analysis performed by the first image analysis device 120 and the second image analysis device 130, and is stored in the first image analysis device 120 and the second image analysis device 130, respectively. It can also be used as necessary training data for training a motion determination model of a driven object and a situation change determination model. A learning method using a surveillance image as a training data set will be described later.

According to an embodiment of the present invention, image analysis may be performed by different devices through two steps. And, each image analysis step can be performed simultaneously regardless of the order. The first image analysis is an image analysis focusing on determining the motion of an object, and the second image analysis is an image analysis focusing on determining a change in a situation. First metadata may be generated through the first image analysis (S130), and second metadata may be generated through the second image analysis (S140).

The first metadata may include information on motion of an object detected in the surveillance image. For example, the first metadata may include information on the appearance of an object, an appearance time, an appearance position, a camera, and a color of the object. In addition, the second metadata may include information on a change in redemption detected in the surveillance image. For example, the second metadata is used to determine the identity of the recognized object, the type of the extracted background, the scene composition and recognition by the behavior analysis of the recognized object against the extracted background, and the repayment change according to the reimbursement reasoning and analysis. May include information about. For a specific example, the second image analysis device 130 recognizes that the object is a person based on the motion information of the object, recognizes a sign indicating a child protection zone in the background through character recognition, and analyzes the behavior of the object. Changes in the situation due to collision between a vehicle and a person corresponding to an object in the protected area can be recognized.

Whether or not an event has occurred is determined through a combination of the first metadata and the second metadata generated as a result of image analysis, that is, through a comprehensive determination of the movement of an object and a change in situation, and an alarm may be displayed in response thereto. (S150).

The above-described situation analysis-based CCTV control method (S100) has been described in terms of the situation analysis-based CCTV control system 100 as a subject. In addition, the situation analysis-based CCTV control method (S100) includes receiving first metadata as a result of image analysis analyzed using a model for determining the movement of an object, a model for determining a change in situation, and a second metadata. It may be configured to include receiving and displaying the surveillance image, and notifying the occurrence of an event based on the first and second metadata.

7 is a flowchart of a CCTV control method based on situation analysis according to an embodiment of the present invention.

Referring to FIG. 7, detailed steps relating to S130 to S150 of FIG. 6 are shown. Steps S211 to S214 may be performed by the first image analysis device at the edge. In addition, steps S310 to S370 may be performed by the second image analysis device in the cloud.

The integrated control server 150 may output an image in which movement of an object is determined, an image in which a situation change is determined, and an image in which an event occurs, through the integrated control viewer 160.

In step S364, according to the image analysis of the first image analysis device and the second image analysis device, the images determined to be the movement of the object, the change of the situation, and the occurrence of the event are used to determine the movement of the object and the change of the situation. It can be used as training data for the model.

In step S365, a model for determining the movement of an object and a model for determining a change in a situation may learn a corresponding image and pattern it.

The model for determining the motion of the object stored in the memory of the first image analysis device 120 connected to the edge may be difficult to update due to the nature of the edge. However, the first image analysis apparatus according to an embodiment of the present invention may perform transfer learning by using data used to determine object motion as training data. Accordingly, the performance of the model for determining the motion of an object according to an embodiment of the present invention may be gradually improved through transfer learning. In addition, the transfer learning using the image collected through the camera 110 may be equally applied to the second image analysis device.

Among the various learning methods constituting artificial intelligence, the artificial intelligence model that determines the motion of an object stored in the first image analysis device 120 connected to the edge automatically extracts features related to the motion of the object in relation to feature extraction from an image. And it can be configured to perform deep learning using this.

In addition, among the various learning methods constituting artificial intelligence, the artificial intelligence model that determines the change of the situation stored in the second image analysis device 130 connected to the cloud performs learning to detect the change of the situation for various preset situations. It can be configured to perform machine learning to make it happen.

Artificial intelligence (AI) is a branch of computer science and information technology that studies how computers can do the thinking, learning, and self-development that human intelligence can do. It means being able to imitate behavior.

In addition, artificial intelligence does not exist by itself, but is directly or indirectly related to other fields of computer science. In particular, in modern times, attempts are being made very actively to introduce artificial intelligence elements in various fields of information technology and to use them in solving problems in that field.

Machine learning is a branch of artificial intelligence, a field of research that gives computers the ability to learn without explicit programming.

Specifically, machine learning can be said to be a technology that studies and builds a system that learns based on empirical data, performs prediction, and improves its own performance, and algorithms for it. Machine learning algorithms do not execute strictly defined static program instructions, but rather build specific models to derive predictions or decisions based on input data.

The term'machine learning' can be used interchangeably with the term'machine learning'.

In terms of how to classify data in machine learning, many machine learning algorithms have been developed. Decision trees, Bayesian networks, support vector machines (SVMs), and artificial neural networks (ANNs) are typical.

The decision tree is an analysis method that charts decision rules into a tree structure and performs classification and prediction.

The Bayesian network is a model that expresses the probabilistic relationship (conditional independence) between multiple variables in a graph structure. Bayesian networks are suitable for data mining through unsupervised learning.

The support vector machine is a model of supervised learning for pattern recognition and data analysis, and is mainly used for classification and regression analysis.

An artificial neural network is an information processing system in which a number of neurons, called nodes or processing elements, are connected in the form of a layer structure by modeling the operation principle of biological neurons and the connection relationship between neurons.

Artificial neural networks are models used in machine learning, and are statistical learning algorithms inspired by biological neural networks (especially the brain among animals' central nervous systems) in machine learning and cognitive science.

Specifically, the artificial neural network may refer to an overall model having problem-solving ability by changing the strength of synaptic bonding through learning by artificial neurons (nodes) forming a network by combining synapses.

The term artificial neural network may be used interchangeably with the term neural network.

The artificial neural network may include a plurality of layers, and each of the layers may include a plurality of neurons. In addition, artificial neural networks may include neurons and synapses that connect neurons.

Artificial neural networks generally have three factors: (1) the connection pattern between neurons in different layers (2) the learning process to update the weight of the connection (3) the output value from the weighted sum of the input received from the previous layer. It can be defined by the activation function it creates.

The artificial neural network may include network models such as DNN (Deep Neural Network), RNN (Recurrent Neural Network), BRDNN (Bidirectional Recurrent Deep Neural Network), MLP (Multilayer Perceptron), CNN (Convolutional Neural Network). , Is not limited thereto.

In the present specification, the term'layer' may be used interchangeably with the term'layer'.

Artificial neural networks are divided into Single-Layer Neural Networks and Multi-Layer Neural Networks according to the number of layers.

A general single-layer neural network is composed of an input layer and an output layer.

In addition, a general multilayer neural network is composed of an input layer, one or more hidden layers, and an output layer.

The input layer is a layer that receives external data, the number of neurons in the input layer is the same as the number of input variables, and the hidden layer is located between the input layer and the output layer, receives signals from the input layer, extracts characteristics, and transfers them to the output layer. do. The output layer receives a signal from the hidden layer and outputs an output value based on the received signal. The input signal between neurons is multiplied by each connection strength (weight) and then summed. If the sum is greater than the neuron's threshold, the neuron is activated and the output value obtained through the activation function is output.

Meanwhile, a deep neural network including a plurality of hidden layers between an input layer and an output layer may be a representative artificial neural network implementing deep learning, a type of machine learning technology.

Meanwhile, the term'deep learning' can be used interchangeably with the term'deep learning'.

The artificial neural network can be trained using training data. Here, learning refers to the process of determining the parameters of the artificial neural network using training data in order to achieve the purpose of classifying, regressing, or clustering input data. I can. Representative examples of parameters of an artificial neural network include weights applied to synapses or biases applied to neurons.

The artificial neural network learned by the training data may classify or cluster input data according to patterns of the input data.

Meanwhile, an artificial neural network trained using training data may be referred to as a trained model in this specification.

The following describes the learning method of artificial neural networks.

Learning methods of artificial neural networks can be classified into supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

Supervised learning is a method of machine learning to infer a function from training data.

And among the functions to be inferred, outputting a continuous value is called regression, and predicting and outputting the class of an input vector can be called classification.

In supervised learning, an artificial neural network is trained with a label for training data.

Here, the label may mean a correct answer (or result value) that the artificial neural network must infer when training data is input to the artificial neural network.

In this specification, when training data is input, the correct answer (or result value) to be inferred by the artificial neural network is referred to as a label or labeling data.

In addition, in this specification, setting a label on training data for learning an artificial neural network is referred to as labeling the training data with labeling data.

In this case, the training data and the label corresponding to the training data) constitute one training set, and may be input to the artificial neural network in the form of a training set.

Meanwhile, the training data represents a plurality of features, and labeling of the training data may mean that a label is attached to the feature represented by the training data. In this case, the training data may represent the characteristics of the input object in the form of a vector.

The artificial neural network can infer a function for the correlation between the training data and the labeling data using the training data and the labeling data. In addition, parameters of the artificial neural network may be determined (optimized) through evaluation of a function inferred from the artificial neural network.

Unsupervised learning is a type of machine learning, where no labels are given for training data.

Specifically, the unsupervised learning may be a learning method of training an artificial neural network to find and classify patterns in the training data itself, rather than a correlation between training data and labels corresponding to the training data.

Examples of unsupervised learning include clustering or independent component analysis.

In this specification, the term'clustering' may be used interchangeably with the term'clustering'.

Examples of artificial neural networks using unsupervised learning include Generative Adversarial Network (GAN) and Autoencoder (AE).

A generative adversarial neural network is a machine learning method in which two different artificial intelligences compete and improve performance, a generator and a discriminator.

In this case, the generator is a model that creates new data and can create new data based on the original data.

Also, the discriminator is a model that recognizes a pattern of data, and may play a role of discriminating whether the input data is original data or new data generated by the generator.

In addition, the generator learns by receiving data that cannot be deceived by the discriminator, and the discriminator can learn by receiving deceived data from the generator. Accordingly, the generator can evolve to deceive the discriminator as well as possible, and the discriminator can evolve to distinguish between the original data and the data generated by the generator.

Auto encoders are neural networks that aim to reproduce the input itself as an output.

The auto encoder includes an input layer, at least one hidden layer and an output layer.

In this case, since the number of nodes in the hidden layer is smaller than the number of nodes in the input layer, the dimension of the data is reduced, and compression or encoding is performed accordingly.

Also, data output from the hidden layer goes to the output layer. In this case, since the number of nodes in the output layer is larger than the number of nodes in the hidden layer, the dimension of the data increases, and accordingly, decompression or decoding is performed.

Meanwhile, the auto-encoder controls the connection strength of neurons through learning, so that the input data is expressed as hidden layer data. In the hidden layer, information is expressed with fewer neurons than in the input layer, but being able to reproduce the input data as an output may mean that the hidden layer found and expressed a hidden pattern from the input data.

Semi-supervised learning is a kind of machine learning, and may mean a learning method using both labeled training data and unlabeled training data.

As one of the techniques of semi-supervised learning, there is a technique of inferring a label of training data that is not given a label and then performing learning using the inferred label. This technique is useful when the cost for labeling is high. I can.

Reinforcement learning is the theory that, given an environment in which an agent can judge what action to do at every moment, it can find the best way to experience without data.

Reinforcement learning can be mainly performed by the Markov Decision Process (MDP).

Explaining the Markov decision process, first, an environment is given that contains the information necessary for the agent to perform the next action, second, it defines how the agent will behave in that environment, and thirdly, what the agent does well is rewarded ( Reward) is given and the penalty is given for failing to do something, and fourthly, the optimum policy is derived by repeatedly experiencing until the future reward reaches the peak.

The structure of the artificial neural network is specified by the configuration of the model, activation function, loss function or cost function, learning algorithm, optimization algorithm, etc., and hyperparameters are pre-trained. It is set, and then, a model parameter is set through learning, so that the content can be specified.

For example, factors determining the structure of an artificial neural network may include the number of hidden layers, the number of hidden nodes included in each hidden layer, an input feature vector, a target feature vector, and the like.

Hyperparameters include several parameters that must be initially set for learning, such as initial values of model parameters. And, the model parameter includes several parameters to be determined through learning.

For example, the hyperparameter may include an initial weight value between nodes, an initial bias value between nodes, a mini-batch size, a number of learning iterations, and a learning rate. In addition, the model parameters may include weights between nodes, biases between nodes, and the like.

The loss function can be used as an index (reference) for determining an optimal model parameter in the learning process of the artificial neural network. In artificial neural networks, learning refers to the process of manipulating model parameters to reduce the loss function, and the purpose of learning can be seen as determining model parameters that minimize the loss function.

The loss function may mainly use a mean squared error (MSE) or a cross entropy error (CEE), but the present invention is not limited thereto.

The cross entropy error may be used when the correct answer label is one-hot encoded. One-hot encoding is an encoding method in which the correct answer label value is set to 1 only for neurons corresponding to the correct answer, and the correct answer label value is set to 0 for non-correct answer neurons.

In machine learning or deep learning, learning optimization algorithms can be used to minimize loss functions, and learning optimization algorithms include Gradient Descent (GD), Stochastic Gradient Descent (SGD), and Momentum. ), NAG (Nesterov Accelerate Gradient), Adagrad, AdaDelta, RMSProp, Adam, Nadam, etc.

Gradient descent is a technique that adjusts model parameters in the direction of reducing the loss function value by considering the slope of the loss function in the current state.

The direction to adjust the model parameter is called the step direction, and the size to be adjusted is called the step size.

In this case, the step size may mean a learning rate.

In the gradient descent method, a gradient is obtained by partially differentiating a loss function into each model parameter, and the model parameters may be updated by changing the acquired gradient direction by a learning rate.

The stochastic gradient descent method is a technique that increases the frequency of gradient descent by dividing training data into mini-batch and performing gradient descent for each mini-batch.

Adagrad, AdaDelta, and RMSProp are techniques that increase optimization accuracy by adjusting the step size in SGD. In SGD, momentum and NAG are techniques to increase optimization accuracy by adjusting the step direction. Adam is a technique that improves optimization accuracy by adjusting the step size and step direction by combining momentum and RMSProp. Nadam is a technique that increases optimization accuracy by adjusting step size and step direction by combining NAG and RMSProp.

The learning speed and accuracy of an artificial neural network are highly dependent on hyperparameters as well as the structure of the artificial neural network and the type of learning optimization algorithm. Therefore, in order to obtain a good learning model, it is important not only to determine an appropriate artificial neural network structure and learning algorithm, but also to set appropriate hyperparameters.

In general, hyperparameters are experimentally set to various values to train an artificial neural network, and as a result of learning, the hyperparameter is set to an optimal value that provides stable learning speed and accuracy.

8 is a flowchart of a CCTV control method based on situation analysis according to an embodiment of the present invention.

Referring to FIG. 8, an algorithm for determining whether an event occurs through step S360 of FIG. 7 is illustrated. The second image analysis apparatus 130 may determine whether an event occurs by combining the first metadata and the second metadata. For example, if a situation change occurs in relation to the movement of the object and the movement of the object through the combination of the first metadata and the second metadata, it is determined that an event has occurred. However, when only the movement of the object is determined or only the change of the situation is determined, the event does not reach the occurrence of the event, and the camera, the time and the situation where the video was captured are bookmarked as necessary to determine whether there is an abnormality.

Determination of whether an event occurs that considers both the movement of an object and a change in situation will be described as an example.

According to the situation-based CCTV control method according to an exemplary embodiment of the present invention, in addition to determining the movement of an object, whether or not an event occurs may be determined based on a determination of a change in a situation. Here, the judgment of the situation change is based on constructing a scene using the extracted object and background and recognizing it. According to the prior art, if the occurrence of an event is determined based only on the movement of the object, misinformation may occur due to the movement of an animal or a movement of a tree, but according to an embodiment of the present invention, what is an object is recognized When a situation change is determined by the object, the occurrence of an event may be determined using motion information and situation change information of the object.

According to an embodiment of the present invention, since it is based on a movement of an object and a change in a situation, it is possible to probably reduce a false report in event detection.

According to the present invention, one event detection algorithm can be applied to both CCTV for crime prevention and CCTV for traffic control.

According to the present invention, the control intelligence of CCTV can be gradually improved through transfer learning in the edge and cloud of the artificial intelligence model.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium is a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a ROM. A hardware device specially configured to store and execute program instructions, such as, RAM, flash memory, and the like.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and usable to those skilled in the computer software field. Examples of the computer program may include not only machine language codes produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

In the specification of the present invention (especially in the claims), the use of the term'above' and a reference term similar thereto may correspond to both the singular and the plural. In addition, when a range is described in the present invention, the invention to which an individual value falling within the range is applied (unless otherwise stated), and each individual value constituting the range is described in the detailed description of the invention. Same as

If there is no explicit order or contradictory description of the steps constituting the method according to the present invention, the steps may be performed in an appropriate order. The present invention is not necessarily limited according to the order of description of the steps. The use of all examples or illustrative terms (for example, etc.) in the present invention is merely for describing the present invention in detail, and the scope of the present invention is limited by the above examples or illustrative terms unless limited by the claims. It does not become. In addition, those skilled in the art can recognize that various modifications, combinations, and changes may be configured according to design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention is limited to the above-described embodiments and should not be determined, and all ranges equivalent to or equivalently changed from the claims to be described later as well as the claims to be described later are the scope of the spirit of the present invention. It will be said to belong to.

100: situation-based CCTV control system
110: camera
120: edge intelligent image analysis device
130: context-based image analysis device
150: integrated control server
160: integrated control viewer
170: video storage device
180: metadata storage device

Claims (7)

At least one or more cameras that collect and transmit surveillance images;
As first metadata related to the motion of the object through the motion-based first image analysis of the object detected in the surveillance image, the motion time of the object, the location and time of appearance of the object, the size and color of the object A first image analysis device in an edge region that generates first metadata including information about the camera and information about the camera;
As second metadata related to situation determination through a context-based secondary image analysis detected in the surveillance image, in which the first metadata is used, the identity of the object, the type of the recognized background, and the recognized background A second image analysis device in a cloud region for generating second metadata including information on a scene composition and recognition according to the movement of the object and information on a situation change according to situation inference and analysis; And
Comprising an integrated control server that displays the surveillance image through an integrated control viewer and notifies the occurrence of an event under a preset condition,
The first image analysis device,
The object is extracted from the surveillance image, and the learned pattern is matched to the object through object classification based on an artificial intelligence model to generate the first metadata for the object, and connected to the camera in the edge region. Controlling the operation of the camera according to the movement of the object based on the first metadata,
The second image analysis device,
Second metadata regarding a situation change inferred through the motion of the object in the background by applying an artificial intelligence model to the background feature extracted from the surveillance image to recognize the background, and using the first metadata And determining whether an event occurs based on a combination of the first metadata and the second metadata,
The integrated control server,
The operation of the camera is controlled according to a change in the situation based on the second metadata in the cloud area, and the object is displayed based on a combination of the first metadata and the second metadata through the second image analysis device. When a change in the situation related to the movement and the movement of the object occurs, it is determined that the event has occurred and the occurrence of the event is notified, and when only the movement of the object is determined or only the situation change is determined, the camera, the time and the situation that took the video are bookmarked. As possible,
CCTV control system based on situation analysis. The method of claim 1,
The first image analysis device,
It is configured to be included in the camera in the form of an embedded system,
CCTV control system based on situation analysis.
delete The method of claim 1,
The second image analysis device,
Using at least one of the color of the lighted beacon and signal information received from the traffic signal controller to detect the movement of a signal violation of a vehicle and a person,
CCTV control system based on situation analysis. The method of claim 1,
The second image analysis device,
By applying an artificial intelligence model trained by learning to the background extracted from the surveillance image, at least one of a child protection zone, a pedestrian crossing, and an intersection is recognized, and between the vehicle and the person in the background recognized using the first metadata. To determine whether the situation inferred through the action changes,
CCTV control system based on situation analysis. As a method performed by a CCTV control system based on situation analysis,
Constructing a model for determining a movement of an object and a model for determining a situation change based on training according to learning using a training data set composed of images collected through a CCTV camera;
Collecting a surveillance image using the CCTV camera to notify the occurrence of an event;
In the edge region, the object is extracted from the surveillance image using an artificial intelligence model that determines the movement of the object, and the learned pattern is matched to the object through object classification based on the artificial intelligence model. Generating first metadata as a result of the first image analysis focusing on motion;
In the cloud area, the first metadata and the artificial intelligence model for determining the situation change are used to recognize the background by applying an artificial intelligence model to the characteristics of the background extracted from the surveillance image, and the first metadata Generating second metadata for determining a situation as a result of a second image analysis focusing on a change in the situation detected in the surveillance image inferred through the motion of the object in the background by using: And
Displaying the surveillance image through an integrated control viewer, comprising the step of notifying the occurrence of an event of a preset condition,
Whether an event occurs is determined based on the combination of the first metadata and the second metadata,
Based on the combination of the first metadata and the second metadata, when the movement of the object and the change in the situation related to the movement of the object occur, it is determined that an event has occurred and the event is notified, and only the movement of the object is determined, or If only the situation change is determined, the camera, the time and the situation where the video was taken are bookmarked,
Further comprising the step of controlling the operation of the camera, wherein the operation of the camera is controlled at the edge according to the movement of the object using the first metadata, and the cloud according to the change of the situation using the second metadata The operation of the camera is controlled in
The first metadata includes a movement time of the object, an appearance position and time of the object, information on the size and color of the object, and camera information,
The second metadata includes information on the identity of the object, the type of the recognized background, the scene composition and recognition according to the movement of the object with respect to the recognized background, and the situation change according to the situation inference and analysis. ,
CCTV control method based on situation analysis. delete

Images