KR20240016572A - Cloud-Multiple Edge Server Collaboration System and Method Based on Service Classification in Intelligent Video Security Environments - Google Patents

Abstract

A cloud-multiple edge server collaboration system and method based on service classification in an intelligent video security environment is disclosed. The system includes one or more terminals; In an intelligent video security environment, the context of the service generated from the terminal and the context information of the computing node are received through a wired or wireless communication network, and the threshold value for service classification, service classification, service transmission, and service are performed through each module. Multiple edge servers processing; and a cloud linked to a reinforcement learning agent that determines a collaboration target based on the context of the service received from each edge server and the context information of the computing node and ensures that the determined collaboration target is serviced.

Description

Cloud-Multiple Edge Server Collaboration System and Method Based on Service Classification in Intelligent Video Security Environments}

The present invention relates to a system and method for performing collaboration between a cloud and multiple edge servers connected to a reinforcement learning agent according to periodically shared services and edge server-related context-based service classification for intelligent video security.

The structure of the collaboration system consists of a cloud connected to a reinforcement learning agent and a number of edge servers and terminals. The edge server classifies the service by extracting the service received from the connected terminal, the context of the edge server's service, and the context of the computing node. The context of the service includes the size and deadline of the service. Computing nodes include cloud or edge servers. The context of a computing node, including a cloud or edge server, includes the computing resources of the computing node (CPU occupancy rate, RAM occupancy rate, storage usage amount), available bandwidth between computing nodes, and the number of services stored in the buffer of the computing node. The cloud selects a collaboration target (edge server) based on reinforcement learning using the context of classified services and the context of computing nodes. Computing nodes selected for collaboration reduce service failure and shorten service processing time compared to other computing nodes.

Cloud computing is centralized on servers and processes transaction data in a client/server manner. In contrast, edge computing is a computing method that is in contrast to cloud computing and is not cloud computing or centralized. Computing takes place at the edge of the network. Edge computing does not process data in cloud computing or in a centralized server where the load of large amounts of data is concentrated, but instead processes data on mobile devices and IoT devices themselves or on edge servers located physically nearby, as shown in Figure 1. Data analysis and device operation are performed using a server).

Figure 1 is a conceptual diagram of intelligent edge computing.

In contrast to cloud computing, which centrally processes data, Mobile Edge Computing (MEC) technology collects, analyzes and processes big data individually from around mobile devices through distributed edge servers. Recently, as LTE 4G/5G mobile communication technology, drones, and IoT technology have developed, it is developing into an intelligent edge computing technology that uses big data analysis using artificial intelligence and machine learning (ML).

For example, CCTV intelligent video processing requires intelligent edge computing technology using multiple edge servers distributed by region to handle the overload of large-capacity video data stored in the central server and large-capacity video data processing.

As prior art 1 related to this, “Distributed processing system between cloud and edge in IoT computing environment” is registered in Patent Registration No. 10-2110592.

The distributed processing system between the cloud and the edge in the IoT computing environment is

The cloud and edge in the IoT computing environment, including a plurality of edges that are connected to the object side to collect data and send control signals to the object side, and a cloud that receives data from each edge, processes it, and delivers the processing results to the edge. As a distributed processing system between

The cloud includes a model creation unit that creates a model for processing the collected data, a model distribution rule for distributed processing between the cloud and the edge, and a model distribution unit that distributes the model to the edge according to the model distribution rule. And

The model distribution rule is determined according to at least one of the processing performance and memory capacity of each edge, and the type and number of models distributed from the cloud to each edge are different according to the determined distribution rule.

As prior art 2 related to this, “Edge computing-based network monitoring method, device, and system” is registered in Patent Registration No. 10-2163280.

Edge computing-based network monitoring methods, devices, and systems are provided with a second entity included in a network including a first entity, a second entity, and a switching device provided between the first and second entities. The second entity includes a transmitter and receiver for transmitting and receiving packets to and from the first entity, and the second entity has a processor and performs edge computing, or performs the switching via an edge computing device that has a processor. connected to the device,

The processor is configured to determine whether a security issue exists with respect to the network associated with the first entity and the second entity based on at least some of the information contained in at least one packet received to or transmitted from the second entity. It is composed.

Determining whether a security problem occurs based on the packet information is

Determining that the number of connections in the section from the predetermined first source IP to the first destination IP is greater than or equal to a predetermined threshold, determining that the request for the predetermined first URL is greater than or equal to the predetermined threshold, and BPS of the predetermined first server. A determination that (Bit Per Second) is more than a predetermined threshold, a determination that PPS (Packet Per Second) from a predetermined second source IP is more than a predetermined threshold, a synchronization signal (SYN) from a predetermined third source IP. Determining that a security problem has occurred in the network based on at least one of a determination that the number of packets is greater than a predetermined threshold, and a determination that a predetermined fourth source IP simultaneously attempts to connect to a number of server IPs greater than the predetermined threshold. do.

As prior art 3 related to this, “Adaptive context sharing and autonomous collaboration system between edge servers for intelligent video security” is registered in Patent Registration No. 10-2304477.

The adaptive context sharing and autonomous collaboration system between edge servers for intelligent video security includes a terminal equipped with a wired and wireless communication unit; Worker edge server that extracts and formalizes the context of connected terminals and edge servers in real time and adaptively shares the context with the master edge server; And it includes a Master edge server that collects and manages context shared from the Worker edge servers and enables autonomous collaboration between edge servers through terminal connection control.

The adaptive context sharing is performed by classifying the characteristics of the context and determining the sharing time based on the classified context characteristics. “Adaptive sharing of the context” means that each Worker edge server uses the information it has or has extracted. The process of transmitting the terminal and edge server context to the Master edge server means sharing the context, and the characteristics of the context determine which edge server requires autonomous collaboration and which edge server will perform autonomous collaboration in autonomous collaboration between worker edge servers. This is a characteristic that indicates whether it is a context that needs to be considered core or a context that plays a supporting role. The context characteristics include factors directly involved in autonomous collaborative linkage [computing resources (CPU clock, CPU occupancy rate, RAM occupancy rate)] and It is classified according to indirectly involved factors [quality (bit rate) of the video transmitted to the edge server, video encoding method, GPS location information of the device, storage space size of the edge server, number of connected devices, and IP address of the edge server]. .

Recently, with the development of network processing technology and smart terminals, the number of simultaneously connected terminals increases and various types of services emerge, and intelligent video security technology using cloud-multiple edge servers is attracting attention. In the case of video security technology based on the existing VMS (Video Management System), many terminals transmit and process all services through a cloud such as a VMS, so there was a problem of reduced service processing efficiency due to the cloud's resource load and delay.

Intelligent video security technology using cloud-edge servers processes services that can be processed efficiently on edge servers to reduce the load on the cloud at edge servers that are geographically close to the terminal to solve problems with cloud-based video processing technology. Services that cannot be processed efficiently are processed in the cloud and collaboration is performed.

In the intelligent video security environment proposed by the present invention, services are classified to minimize service failure in an environment where multiple edge servers and terminals exist through a service classification-based cloud-multiple edge server collaboration system, and the classification of the classified services is provided. By determining collaboration targets based on reinforcement learning to ensure minimum processing time, it is expected to increase service real-time and efficiency in intelligent video security services and various cloud-multiple edge server collaboration system environments.

Patent registration number 10-2110592 (registration date May 7, 2020), "Distributed processing method and system between cloud and edge in IoT computing environment", Electronic Component Research Institute Patent registration number 10-2163280 (registration date: September 29, 2020), “Edge computing-based network monitoring method, device, and system”, MacData Co., Ltd. Patent registration number 10-2304477 (registration date September 14, 2021), “Adaptive context sharing and autonomous collaboration system between edge servers for intelligent video security”, Kwangwoon University Industry-Academic Cooperation Foundation

The purpose of the present invention to solve the above problems is to perform collaboration between the cloud and multiple edge servers connected to a reinforcement learning agent according to periodically shared services for intelligent video security and context-based service classification related to the edge server. It provides a collaboration system between cloud and multiple edge servers based on service classification in an intelligent video security environment.

Another object of the present invention is to provide a service classification-based cloud-multiple edge server collaboration method in an intelligent video security environment.

In order to achieve the purpose of the present invention, a service classification-based cloud-multiple edge server collaboration system in an intelligent video security environment includes one or more terminals; In an intelligent video security environment, the context of the service generated from the terminal and the context information of the computing node are received through a wired or wireless communication network, and the threshold value for service classification, service classification, service transmission, and service are performed through each module. Multiple edge servers processing; and a cloud linked to a reinforcement learning agent that determines a collaboration target based on the context of the service received from each edge server and the context information of the computing node and ensures that the determined collaboration target is serviced.

In order to achieve another object of the present invention, a service classification-based cloud-multiple edge server collaboration method in an intelligent video security environment includes: (a) In an intelligent video security environment, the context of a service generated from one or more terminals and the computing node Receiving context information to an edge server through a wired or wireless communication network, calculating a threshold for service classification, classifying the service, transmitting the service, and processing the service through each module of the edge server; and (b) determining a collaboration target based on the context of the service received from each edge server and the context information of the computing node by a reinforcement learning agent connected through the cloud linked to the plurality of edge servers, so that the determined collaboration target is serviced. It includes steps to:

The cloud-multiple edge server collaboration system and method based on service classification in an intelligent video security environment of the present invention is a service classification-based cloud-multiple edge server collaboration system in an intelligent video security environment to lower the service failure rate of the edge server. The failure rate and processing time of the service can be minimized by classifying the service based on a threshold and selecting an edge server for collaboration by a reinforcement learning agent to minimize the processing time of the service based on the classified service. By minimizing the failure rate and processing time of services between collaboration target dpt servers and terminals, real-time performance of intelligent video security services can be increased and service efficiency can be improved.

Intelligent video security technology utilizing a cloud-edge server is designed to reduce the load on the cloud on an edge server that is geographically close to the terminal in order to solve the problems of cloud-based video processing technology. Services that can be processed efficiently are provided on the edge server. Services that cannot be processed efficiently are processed and collaborated in the cloud linked to a reinforcement learning agent that determines collaboration.

The cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment proposed by the present invention is an autonomous collaborative edge computing environment in which there are a cloud connected to a reinforcement learning agent and multiple edge servers and terminals, and the edge server provides services. In order to minimize failures, services are classified, and collaboration targets are determined based on reinforcement learning agents to ensure the minimum processing time of classified services, and terminal connections are performed to create an intelligent video security service and a collaboration system environment between various clouds and multiple edge servers. It is expected that service real-time and efficiency can be improved.

This system provides an intelligent video security service based on autonomous collaboration between cloud and multiple edge servers, an autonomous collaboration platform between cloud and edge servers for smart cities, a collaboration platform between cloud and edge servers that guarantees real-time service, and an automated environment and It can be used in command and control systems between control centers.

Figure 1 is a conceptual diagram of intelligent edge computing.
Figure 2 is a configuration diagram showing an autonomous collaboration environment between cloud and multiple edge servers based on system cluster structure and service classification in the intelligent video security environment of the present invention.
Figure 3 shows the framework of an autonomous collaboration system between cloud and multiple edge servers based on service classification in an intelligent video security environment according to the present invention.
Figure 4 is a graph of the threshold change of the sigmoid function, where the edge server calculates the threshold.
Figure 5 is an example diagram of task transmission according to service classification.
Figure 6 shows the learning structure of a reinforcement learning agent linked to the cloud.

Hereinafter, the configuration and operation of the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention is not limited to the disclosed embodiments, but may be implemented in various different forms by those skilled in the art. In the description of the present invention, if it is determined that a detailed description of a related known technology or a known configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the attached drawing numbers are assigned the same drawing numbers in other drawings when indicating the same configuration.

It is not limited to specific embodiments through this research and development, and should be understood to include all conversions, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Figure 2 is a configuration diagram showing an autonomous collaboration environment between cloud and multiple edge servers based on system cluster structure and service classification in the intelligent video security environment of the present invention.

Figure 3 shows the framework of an autonomous collaboration system between cloud and multiple edge servers based on service classification in an intelligent video security environment according to the present invention.

In an intelligent video security environment (smartphone video, CCTV video, car black box video, drone camera video), a collaboration system between edge servers based on computing resource prediction includes a cloud with an agent, multiple edge servers, and It consists of multiple terminals. For intelligent video security, terminals, edge servers, and context are periodically shared, and autonomous collaboration between edge servers is performed based on computing resource prediction based on terminal connection control using the context of shared services and the context of computing nodes.

The context is classified by elements involved in autonomous collaborative linkage, and the overall edge server computing resource distribution is minimized to achieve load balancing based on the classified context characteristics, so that the usage of computing resources is less than the threshold. Once it is decided to control the connection between the edge server and the terminal to perform collaboration, the edge server and the terminal for which collaboration has been decided are connected through a wired or wireless communication network by a reinforcement learning agent connected to the cloud. Minimize the load on each edge server's computing resources (CPU occupancy, RAM occupancy, storage usage) and provide load balancing to perform autonomous collaboration to improve the efficiency of intelligent video security services based on real-time video analysis. can increase.

The edge server's computing resources include CPU occupancy rate, RAM occupancy rate, and storage usage amount. Service context (service information) includes the service size and service deadline.

Classification of services means classifying them into services that meet the deadline and services that do not satisfy the deadline.

The reinforcement learning agent connected to the cloud 100 receives the context of the service requiring collaboration and the context of the computing node from the edge server 200 to the cloud 100, and performs reinforcement linked to the cloud 100 that received it. The collaboration target (edge server for which collaboration has been decided) is determined by the learning agent, and services are provided to the determined collaboration target.

In an intelligent video security environment, a collaboration system between edge servers based on computing resource prediction consists of a cloud (100) linked to a reinforcement learning agent, a number of edge servers (200), a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication). It consists of terminals 300 equipped with a cable) or wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit).

For example, the terminals 300 include a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - A smartphone equipped with a CCTV camera, a CCTV camera equipped with a camera, a GPS receiver, and a wired communication unit or a wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a camera, a GPS receiver, and a wireless communication unit. Use any one of the equipped drone terminals.

The present invention provides a system and method for performing collaboration between cloud and multiple edge servers according to periodically shared services and edge server-related context-based service classification for intelligent video security.

The structure of a system that performs collaboration between clouds and multiple edge servers consists of a cloud 100 linked to a reinforcement learning agent, a plurality of edge servers 200, and terminals 300. The computing node includes a cloud 100 or a plurality of edge servers 200. The edge server extracts the services received from one or more connected terminals, the service context of the edge server, and the context of the computing node and classifies the service (classifies services into services that meet the deadline and services that do not satisfy the deadline). . The context of the service includes the size of the service and deadline (service start time to service completion time to). The computing node includes a cloud 100 and a plurality of edge servers 200. The context of a computing node includes the computing resources of the computing node (CPU occupancy rate, RAM occupancy rate, storage usage amount), available bandwidth between computing nodes, and the number of services stored in the computing node's buffer. The cloud 100 determines/selects a collaboration target (edge server) by a reinforcement learning agent using the context of the classified service and the context of the computing nodes. Computing nodes selected for collaboration reduce service failure and shorten service processing time compared to other computing nodes.

The cloud-multiple edge server autonomous collaboration system based on service classification for intelligent video security operates based on a cluster structure where a cloud (100) linked to a reinforcement learning agent and a plurality of edge servers (200) are connected.

Figure 2 shows an autonomous collaboration environment between cloud and multiple edge servers based on the system cluster structure and service classification defined in the present invention.

The terminal 300 is connected to an edge server, and the terminal 300 creates services by the user. The terminal 300 transmits the generated task to the edge server 200 due to insufficient computing resources to process a service requiring low delay. The service transmitted by the terminal 300 includes service deadline and service size information. The deadline of a service refers to the time required for the service to be created and processing completed. If computing nodes such as the cloud 100 and the edge server 200 do not meet the service deadline, user satisfaction decreases. The edge server 200, which has received the service, classifies the service based on the computing resources of the edge server, the number of services in the buffer, information on the buffer, the context of the service of the edge server, and the context of the computing node (services that satisfy the deadline) Wow, classified as a service whose deadline is not met). Services are classified into services that satisfy the deadline and services that do not satisfy the deadline when processed by the edge server 200. The edge server 200 transmits context information of services for which service failure (service deadline is not met) is predicted through service classification to the cloud 100, and the service is processed by a reinforcement learning agent linked to the cloud 100. A collaboration target is selected for . The cloud 100 transmits the classified services received from each edge server, the edge server's service context information, and the cloud's computing resource information to a reinforcement learning agent, so that computing resources are relatively spare and service processing time is shortened. Select the available edge server (200) or cloud (100). Information about the selected collaboration target is transmitted to the edge server on which the collaboration that received the task was determined, and the edge server transmits the service to the collaboration target.

Figure 3 shows the framework of an autonomous collaboration system between cloud and multiple edge servers based on service classification in an intelligent video security environment according to the present invention.

In the intelligent video security environment of the present invention, the service classification-based cloud-multiple edge server collaboration system

One or more terminals 300 in an intelligent video security environment;

A plurality of people receive the context of the service generated from the terminal 300 and the context information of the computing node through a wired or wireless communication network, and calculate the threshold for service classification through each module, classify the service, transmit the service, and process the service. edge server (200); and

A cloud 100 linked to the plurality of edge servers and a reinforcement learning agent that determines a collaboration target based on the context of the service received from each edge server and the context information of the computing node and ensures that the determined collaboration target is serviced. Includes.

The edge server 200 includes a threshold calculation module, a service classification module, a service transmission module, and a service processing module.

In addition, the cloud-multiple edge server collaboration method based on service classification in the intelligent video security environment of the present invention is (a) in the intelligent video security environment, the context of the service generated from one or more terminals and the context information of the computing node are transmitted to the wired and wireless communication network. receiving data from an edge server, and calculating a threshold for service classification, classifying the service, transmitting the service, and processing the service through each module of the edge server; and (b) determining a collaboration target based on the context of the service received from each edge server and the context information of the computing node by a reinforcement learning agent connected through the cloud linked to the plurality of edge servers, so that the determined collaboration target is serviced. It includes steps to:

The reinforcement learning agent connected to the cloud 100 receives the context of the service requiring collaboration and the context of the computing node from the edge server 200 to the cloud 100, and performs reinforcement linked to the cloud 100 that received it. The collaboration target (edge server for which collaboration has been decided) is determined by the learning agent, and services are provided to the determined collaboration target.

In an intelligent video security environment, a collaboration system between edge servers based on computing resource prediction consists of a cloud (100) linked to a reinforcement learning agent, a number of edge servers (200), a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication). It consists of terminals 300 equipped with a cable) or wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit).

For example, in an intelligent video security environment, the terminals 300 include a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit). , a smartphone equipped with an NB-IoT communication unit (CCTV camera), a CCTV camera equipped with a camera, a GPS receiver, and a wired communication unit or a wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a camera. Use one of the following terminals: and a drone equipped with a GPS receiver and a wireless communication unit.

The edge server 200 receives the context of the service generated from the terminal 300 and the context information of the computing node, and calculates the threshold for service classification, service classification, service transmission, and service through each module of the dpt g server. Perform processing.

The threshold calculation module of the edge server 200 includes the context of the service (size of the service, deadline of the service) transmitted from the terminal connected to the edge server 200, the context of the service stored in the buffer of the edge server, and the edge server 200 ) Calculate the threshold for service classification based on computing resources. The context of the service includes the size and deadline of the service. Computing nodes include cloud or edge servers. The context of a computing node includes the computing resources of the computing node (CPU occupancy rate, RAM occupancy rate, storage usage amount), available bandwidth between computing nodes, and the number of services stored in the computing node's buffer.

The service classification module is based on the computing resources of the edge server 200, the number of services in the buffer, the information in the buffer, and the context of the service of the edge server and the context of the computing node. The service classification module determines the terminal based on the calculated threshold. The services received from (300) are classified (classified into services that meet the deadline and services that do not satisfy the deadline).

The service transmission module transmits the context of the service requiring collaboration to the cloud 100, and transmits the service to the collaboration target (edge server for which collaboration has been determined) determined by the reinforcement learning agent of the cloud 100 that received it. .

The service processing module processes services according to the decision of the service classification module or cloud reinforcement learning agent.

The cloud 100 includes a context reception module, a context transmission module, and a service processing module.

The context receiving module receives the context of the service transmitted from the edge server 200, edge server computing resources, edge server buffer information (number of services stored in the buffer, memory information in the buffer), and computing nodes (cloud, edge server) Receives information on inter-network bandwidth. The context transmission module transmits the information (service context and computing node context) received from the context reception module to the reinforcement learning agent through the context transmission module.

The service processing module of the cloud 100 processes services by transmitting them to the cloud 100 and the edge server 200 according to the collaboration target by a reinforcement learning agent among services requiring collaboration.

The reinforcement learning agent determines the collaboration target (edge server) to process the service based on the reinforcement learning agent's decision based on the context of the service received through the context transfer module and the context of the computing node. The information transmitted from the cloud 100 to the reinforcement learning agent is used as an element of the 'state' of the reinforcement learning agent to shorten service time and reduce service failure through an edge server ( 200) to decide (act) on a collaboration target. Services are transmitted to the cloud 100 or edge server 200 according to the collaboration target determined by the reinforcement learning agent, and the corresponding compensation (service failure and service processing time) is measured. The reinforcement learning agent learns the optimal collaboration decision policy using 'next state' according to 'state', 'action', and 'reward' to determine the collaboration target.

Figure 4 shows a graph of the threshold change of the sigmoid function according to the threshold value calculated by the edge server and the threshold value for service classification.

는 번째 엣지 서버의 임계값,

는 번째 엣지 서버의 가용 컴퓨팅 리소스, 그리고 sigmoid 함수의 인자로 쓰이는

는 식을 통해 계산된다. The autonomous collaboration system between cloud and multiple edge servers based on service classification for intelligent video security performs service classification by setting a threshold based on the sigmoid function for efficient service processing. The threshold is Calculate using the formula.

Is Threshold of the edge server,

Is Available computing resources of the first edge server, and used as an argument to the sigmoid function

Is It is calculated through the formula.

는 엣지 서버가 처리 중인 서비스의 수, j는 서비스의 index, Lj는 j번째 서비스의 크기,

는 번째 서비스의 deadline, k는 j번째 서비스가 시작될 때까지 버퍼에 저장된 서비스의 index,

는 엣지 서버가 처리 중인 작업의 남은 작업 크기, 그리고 N은 버퍼에 저장된 서비스의 집합을 나타낸다. here,

is the number of services being processed by the edge server, j is the index of the service, Lj is the size of the jth service,

Is deadline of the th service, k is the index of the service stored in the buffer until the jth service starts,

represents the remaining work size of the work being processed by the edge server, and N represents the set of services stored in the buffer.

Calculate the difference between the processing time of tasks stored in the edge server's buffer and the service's deadline. When the edge server is processing a task, computing resources for the task stored in the buffer are allocated to the service only when the task being processed is completed. Therefore, the edge server calculates the threshold considering the remaining processing time of the task being processed. When a large number of tasks are sent to the edge server for a certain period of time and the load on the edge server increases and the service requirements cannot be satisfied, the threshold approaches 0 and the computing node on which the service will be processed is controlled by a reinforcement learning agent. Another new collaboration target (edge server) is determined/selected. In the opposite case, the threshold is similar to the computing resource capacity of the edge server, so services are processed on the edge server to which the terminal is connected.

Figure 5 shows an example of job transmission according to service classification in an edge server.

The edge server calculates the threshold based on the characteristics of the service, the number of services waiting in the buffer of the local edge server, and the computing resources of the local edge server.

The threshold is compared with the required resources of the services stored in the buffer, and if the required resources of the service are smaller than the threshold, the service is classified as a service that does not require collaboration because it can be processed by the local edge server to meet the service's deadline, This service is stored in the buffer of the local edge server, and the edge server sequentially processes the services stored in the buffer.

If the service's required resources are greater than the above threshold, it is classified as a service requiring collaboration. The service does not remain in the local edge server buffer, but the collaboration target to process the service is determined/selected according to the decision of the reinforcement learning agent located in the cloud 100.

Figure 6 shows a structure in which a reinforcement learning agent linked to the cloud performs learning.

는 서비스 실패 여부, 그리고

는 서비스 완료 시간을 나타낸다. '다음 상태'는 행동 이후에 나타나는 엣지 서버 및 클라우드의 컴퓨팅 리소스, 엣지 서버의 버퍼에 저장된 작업, 서비스의 크기 및 deadline, 그리고 컴퓨팅 노드간 네트워크 대역폭이다. A reinforcement learning agent needs 'state', 'action', 'reward', and 'next state' for learning. 'Status' refers to the computing resources of the edge server and cloud obtained from the autonomous collaboration environment between cloud and multiple edge servers, the number and tasks of services stored in the buffer of the edge server, the size of the service, and the deadline of the service (service start time from - service completion) Time to), and network bandwidth between computing nodes. ‘Action’ was defined as a computing node subject to collaboration. Computing nodes include cloud or multiple edge servers connected to reinforcement learning agents. 'Compensation' was defined as the failure of the task according to the 'action' and the processing time of the task. The reward is It is calculated as in Eq. r is the reward (calculated according to service failure and service processing time, which indicates how well the service collaboration was performed), α is the weight of service failure, β is the weight of service processing time,

determines whether the service has failed, and

represents the service completion time. The 'next state' is the computing resources of the edge server and cloud that appear after the action, the tasks stored in the edge server's buffer, the size and deadline of the service, and the network bandwidth between computing nodes.

Learning of reinforcement learning agents is carried out using the Actor-Critic algorithm. Actor-Critic consists of actor network and critic network.

In an autonomous collaborative edge computing environment between cloud and multiple edge servers, the actor network determines the importance (learning weight) of 'state' information. As for learning weights, the higher the importance, the higher the weight, and the lower the importance, the lower the weight. Depending on the learning weights, the actor network shows the collaboration probability for each computing node (cloud - multiple edge servers). The number of collaboration probabilities of each computing node shown is equal to the sum of the number of edge servers and the number of clouds. The collaboration target is the computing node with the highest probability among the various probabilities of each dpt server, and the collaboration probability for each computing node changes as the learning weight increases and decreases. The probability of a computing node's collaboration can process a service within the deadline, and increases as the service is processed faster.

Critic network evaluates the value of 'state' (computing resource information of computing nodes of cloud and edge servers, network bandwidth between computing nodes, number of services stored in the buffer of computing nodes). The value of 'state' is measured based on the computing node's resources (CPU occupancy, RAM occupancy, storage usage, etc.), the bandwidth of the network between computing nodes, and the number of services stored in the computing node's buffer. Additionally, the value of 'state' is higher as there are many services that computing nodes can process within the deadline and the shorter the processing time of the services.

The actor network collaborates to maximize the values evaluated by the critic network [resources of computing nodes (CPU occupancy, RAM occupancy, storage usage, etc.), bandwidth of network between computing nodes, number of services stored in buffers of computing nodes] While learning the decision policy, the critic network compares the predicted value of the 'next state' according to the 'current state' with the value of the 'next state' and proceeds with learning in a direction where the difference between the two values is small. As a result, the actor network adjusts the learning weight so that the reward according to the state-action (indicating how well the collaboration was done - service failure and service processing time) is maximized, and the critic network adjusts the learning weight to maximize the 'current state' The learning weight is adjusted to minimize the difference between the two values by comparing the predicted value of 'next state' and the value of the next state.

The cloud-multiple edge server collaboration system and method based on service classification in an intelligent video security environment of the present invention is a service classification-based cloud-multiple edge server collaboration system in an intelligent video security environment. The failure rate and processing time of the service can be minimized by classifying the service based on the service and selecting a collaboration target server through reinforcement learning to minimize the processing time of the service based on the classified service. By minimizing the service failure rate and processing time, the real-time nature of the intelligent video security service can be increased and the service efficiency can be improved.

Intelligent video security technology utilizing a cloud-edge server is designed to reduce the load on the cloud on an edge server that is geographically close to the terminal in order to solve the problems of cloud-based video processing technology. Services that can be processed efficiently are provided on the edge server. Services that cannot be processed efficiently are processed and collaborated in the cloud linked to a reinforcement learning agent that determines collaboration.

The cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment proposed by the present invention is an autonomous collaborative edge computing environment in which there are a cloud connected to a reinforcement learning agent and multiple edge servers and terminals, and the edge server provides services. In order to minimize failures, services are classified, and collaboration targets are determined based on reinforcement learning agents to ensure the minimum processing time of classified services, and terminal connections are performed to create an intelligent video security service and a collaboration system environment between various clouds and multiple edge servers. It is expected that service real-time and efficiency can be improved.

This system provides an intelligent video security service based on autonomous collaboration between cloud and multiple edge servers, an autonomous collaboration platform between cloud and edge servers for smart cities, a collaboration platform between cloud and edge servers that guarantees real-time service, and an automated environment and It can be used in command and control systems between control centers.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Optical media (magneto-optical media), and storage media such as ROM, RAM, flash memory, storage, etc. may include hardware devices configured to store and execute program instructions. Examples of program instructions may include those produced by a compiler, machine language code, as well as high-level language code that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention is implemented as a program and can be stored on a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) in a form that can be read using computer software. ) can be stored in .

Although the present invention has been described with reference to specific embodiments of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiments to illustrate the technical idea as described above, and is not limited to the technical idea and scope of the present invention. It can be implemented with various modifications, and the scope of the present invention should be determined by the claims described later.

100: Cloud server
200: Edge server
300: Terminal (smartphone, CCTV camera, drone...)

Claims (17)

One or more terminals;
In an intelligent video security environment, the context of the service generated from the terminal and the context information of the computing node are received through a wired or wireless communication network, and each module performs threshold calculation for service classification, service classification, service transmission, and service. Multiple edge servers processing; and
A cloud linked to a reinforcement learning agent that determines a collaboration target based on the context of the service received from each edge server and the context information of the computing node and ensures that the determined collaboration target is serviced;
A cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment that includes. According to paragraph 1,
In an intelligent video security environment, the terminals include a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - CCTV camera) ), a smartphone equipped with a camera, a GPS receiver, and a wired or wireless communication unit, a CCTV camera equipped with a camera, a GPS receiver, and a wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a camera, a GPS receiver, and a wireless communication unit. Use any one of the drones available,
The service transmitted by the terminal includes service deadline and service size information, and the service deadline refers to the time required from the time the service is created to completion of processing, and is a service classification-based service in an intelligent video security environment. Cloud-multiple edge server collaboration system. According to paragraph 2,
The edge server is
The edge calculates a threshold for service classification based on the context of the service (size of the service, deadline of the service) transmitted from the terminal connected to the edge server, the context of the service stored in the buffer of the edge server, and the computing resources of the edge server. Threshold calculation module of the server;
a service classification module that classifies services received from the terminal into services that satisfy a deadline and services that do not satisfy a deadline, based on the calculated threshold;
a service transmission module that transmits the context of a service requiring collaboration to the cloud and transmits the service to a collaboration target (an edge server for which collaboration has been determined) determined by the reinforcement learning agent in the cloud; and
A service classification-based cloud-multiple edge server collaboration system in an intelligent video security environment that includes a service processing module that processes services according to the collaboration decision of the service classification module or the reinforcement learning agent of the cloud. According to paragraph 3,
The context of the service includes the size of the service and the service deadline, the computing node includes a cloud or an edge server, and the context of the computing node includes the computing resources of the computing node, the available bandwidth between computing nodes, and the buffer of the computing node. A cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment, where the computing resources include CPU occupancy, RAM occupancy, and storage usage. According to paragraph 1,
The cloud is
A context receiving module that receives the context of the service transmitted from the edge server, edge server computing resource information, information on the number of services stored in the edge server's buffer, and information on bandwidth between computing nodes (cloud, edge server);
a context transmission module that transmits information (service context and computing node context) received from the context reception module to the reinforcement learning agent;
It includes a cloud service processing module that processes services for which the collaboration target is determined to be the cloud by a reinforcement learning agent among the services requiring collaboration,
A cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment that includes. According to paragraph 1,
The reinforcement learning agent determines the edge server to collaborate with based on the context of the service and the context of the computing node received through the context transmission module, and the information transmitted from the cloud to the reinforcement learning agent is the 'state' of the reinforcement learning agent. It is used as an element of 'state' to shorten service time, determines (action) the collaboration target with an edge server that can reduce service failure, and acts as a cloud or edge server determined according to the collaboration target determined by the reinforcement learning agent. Services are transmitted, the resulting rewards (service failure and service processing time) are measured, and the reinforcement learning agent uses the ‘next state’ according to ‘state’, ‘action’, and ‘reward’ to determine the optimal collaboration decision policy. A cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment that learns and determines collaboration targets. According to paragraph 1,
In an autonomous collaboration system between cloud and multiple edge servers based on service classification for intelligent video security, for efficient service processing, the edge server sets a threshold based on the sigmoid function to classify the service, and the threshold is Calculate through the formula,

Is Threshold of the edge server,

Is Available computing resources of the first edge server, and used as an argument to the sigmoid function

Is Calculated through the formula,
here,

is the number of services being processed by the edge server, j is the index of the service, Lj is the size of the jth service,

Is deadline of the th service, k is the index of the service stored in the buffer until the jth service starts,

represents the remaining task size of the task being processed by the edge server, and N represents the set of services stored in the buffer.
The difference between the processing time of the tasks stored in the buffer of the edge server and the service deadline is calculated. When the edge server is processing a task, computing resources for the task stored in the buffer are allocated to the service only when the task being processed is completed. The threshold is calculated by considering the remaining processing time of the task being processed by the edge server. If a large number of tasks are sent to the edge server during a certain period of time, the load on the edge server increases to meet the service requirements. When there is none, the threshold approaches 0, and the computing node where the service will be processed determines/selects another new collaboration target by the reinforcement learning agent. In the opposite case, the threshold is similar to the computing resource capacity of the edge server, so the services are connected to the terminal. A cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment that is processed on this connected edge server. According to paragraph 1,
The edge server calculates the threshold based on the characteristics of the service, the number of services waiting in the buffer of the local edge server, and the computing resources of the local edge server,
The threshold is compared with the required resources of the services stored in the buffer, and if the required resources of the service are smaller than the threshold, the service is classified as a service that does not require collaboration because it can be processed by the local edge server to meet the service's deadline, This service is stored in the buffer of the local edge server, and the edge server sequentially processes the services stored in the buffer.
If the required resources of a service are greater than the threshold, the service is classified as a service requiring collaboration, and the service does not remain in the local edge server buffer, but becomes a collaboration target to process the service according to the decision of the reinforcement learning agent located in the cloud. A cloud-multiple edge server collaboration system based on service classification in a determined/selected intelligent video security environment. According to paragraph 1,
The reinforcement learning agent requires 'state', 'action', 'reward', and 'next state' for learning,
The 'state' refers to the computing resources of the edge server and cloud obtained from the autonomous collaboration environment between the cloud and multiple edge servers, the number and tasks of services stored in the buffer of the edge server, the size of the service and the deadline of the service, and the network bandwidth between computing nodes. It was defined as
The 'action' is defined as a computing node subject to collaboration, and the computing node includes a cloud or multiple edge servers connected to a reinforcement learning agent,
The ‘compensation’ was defined as the failure of the task according to the ‘action’ and the processing time of the task. It is calculated as in the formula,
Here, r is the reward (calculated according to service failure and service processing time, which indicates how well the service collaboration was performed), α is the weight of service failure, β is the weight of service processing time,

determines whether the service has failed, and

represents the service completion time,
The 'next state' is the classification of services in an intelligent video security environment, including the computing resources of the edge server and cloud that appear after the action, tasks stored in the edge server's buffer, service size and service deadline, and network bandwidth between computing nodes. A cloud-based collaboration system between multiple edge servers. According to clause 9,
The training of the reinforcement learning agent is carried out using the Actor-Critic algorithm, and the Actor-Critic consists of an actor network and a critic network,
In an autonomous collaborative edge computing environment between cloud and multiple edge servers, the actor network determines the importance (learning weight) of 'state' information, and the learning weight increases as the importance state increases, and as the importance state increases, the actor network determines the importance (learning weight) of 'state' information. The higher the state, the lower the weight, and according to the learning weight, the actor network shows the collaboration probability for each computing node (cloud - multiple edge servers), and the number of collaboration probabilities for each computing node shown is the number of edge servers and It is equal to the sum of the number of clouds, and the collaboration target is the computing node with the highest probability among the multiple probabilities of each edge server sequentially. As the learning weight increases and decreases, the collaboration probability for each computing node changes. , the collaboration probability of the computing node can process the service within the deadline, and increases as the service is processed faster.
The critical network evaluates the value of the 'state' (computing resource information of computing nodes of cloud and edge servers, network bandwidth between computing nodes, number of services stored in the buffer of computing nodes), and the value of the 'state' It is measured based on the computing node's resources (CPU occupancy, RAM occupancy, storage usage, etc.), the bandwidth of the network between computing nodes, and the number of services stored in the computing node's buffer. Additionally, the value of 'state' is calculated based on the computing node's There are many services that can be processed within the deadline, and the shorter the processing time of the services, the higher it is.
The actor network maximizes the values evaluated by the critic network [resources of computing nodes (CPU occupancy, RAM occupancy, storage usage, etc.), bandwidth of network between computing nodes, number of services stored in buffers of computing nodes]. Learning a collaborative decision policy, the critic network compares the predicted value of the 'next state' according to the 'current state' with the value of the 'next state' and proceeds with learning in a direction where the difference between the two values is small. As a result, the actor network Adjusts the learning weight so that the reward according to the state-action (indicating how well the collaboration was done - service failure and service processing time) is maximized, and the critic network determines the 'next status' according to the 'current state'. A cloud-multiple edge server collaboration system based on service classification in an intelligent video security environment that compares the predicted value of 'state' with the value of the next state and adjusts the learning weight to minimize the difference between the two values. (a) In an intelligent video security environment, the context of a service generated from one or more terminals and the context information of a computing node are received by an edge server through a wired or wireless communication network, and a threshold value for service classification is calculated through each module of the edge server, Classification of services, transmission of services, and processing of services; and
(b) Determining the collaboration target based on the context of the service received from each edge server and the context information of the computing node by a reinforcement learning agent connected through the cloud linked to the plurality of edge servers and ensuring that the determined collaboration target is serviced. step;
A cloud-multiple edge server collaboration method based on service classification in an intelligent video security environment including. According to clause 11,
The terminals are smartphones equipped with a camera, a GPS receiver, and a wired communication unit (Ethernet, UTP cable, optical communication cable) or a wireless communication unit (Wi-Fi, LTE 4G/5G communication unit, LoRa RF communication unit, NB-IoT communication unit - CCTV camera). , a CCTV camera equipped with a camera, a GPS receiver, and a wired or wireless communication unit, a car equipped with a black box equipped with a camera, a GPS receiver, and a wireless communication unit, and a drone equipped with a camera, a GPS receiver, and a wireless communication unit. Using a terminal,
The service transmitted by the terminal includes service deadline and service size information, and the service deadline refers to the time required from the time the service is created to completion of processing, and is a service classification-based service in an intelligent video security environment. Collaboration method between cloud and multiple edge servers. According to clause 11,
The edge server is
The edge calculates a threshold for service classification based on the context of the service (size of the service, deadline of the service) transmitted from the terminal connected to the edge server, the context of the service stored in the buffer of the edge server, and the computing resources of the edge server. Threshold calculation module of the server;
a service classification module that classifies services received from the terminal into services that satisfy a deadline and services that do not satisfy a deadline, based on the calculated threshold;
A service transmission module that transmits the context of a service requiring collaboration to the cloud and transmits the service to a collaboration target (edge server for which collaboration has been determined) determined by a reinforcement learning agent of the cloud; and
A service classification-based cloud-multiple edge server collaboration method in an intelligent video security environment, including a service processing module that processes services according to a collaboration decision of the service classification module or the reinforcement learning agent of the cloud. According to clause 13,
The context of the service includes the size of the service and the service deadline, the computing node includes a cloud or an edge server, and the context of the computing node includes the computing resources of the computing node, the available bandwidth between computing nodes, and the buffer of the computing node. A cloud-multiple edge server collaboration method based on service classification in an intelligent video security environment, where the computing resources include CPU occupancy, RAM occupancy, and storage usage. According to clause 11,
The cloud is
A context receiving module that receives the context of the service transmitted from the edge server, edge server computing resource information, information on the number of services stored in the edge server's buffer, and information on bandwidth between computing nodes (cloud, edge server);
a context transmission module that transmits information (service context and computing node context) received from the context reception module to the reinforcement learning agent;
It includes a cloud service processing module that processes services for which the collaboration target is determined to be the cloud by a reinforcement learning agent among the services requiring collaboration,
A cloud-multiple edge server collaboration method based on service classification in an intelligent video security environment including. According to clause 11,
The reinforcement learning agent determines the edge server to collaborate with based on the context of the service and the context of the computing node received through the context transmission module, and the information transmitted from the cloud to the reinforcement learning agent is the 'state' of the reinforcement learning agent. It is used as an element of 'state' to shorten service time, determines (action) the collaboration target with an edge server that can reduce service failure, and acts as a cloud or edge server determined according to the collaboration target determined by the reinforcement learning agent. Services are transmitted, the resulting rewards (service failure and service processing time) are measured, and the reinforcement learning agent uses the ‘next state’ according to ‘state’, ‘action’, and ‘reward’ to determine the optimal collaboration decision policy. A cloud-multiple edge server collaboration method based on service classification in an intelligent video security environment that learns and determines collaboration targets. According to clause 11,
In an autonomous collaboration system between cloud and multiple edge servers based on service classification for intelligent video security, for efficient service processing, the edge server sets a threshold based on the sigmoid function to classify the service, and the threshold is Calculate through the formula,

Is Threshold of the edge server,

Is Available computing resources of the first edge server, and used as an argument to the sigmoid function

Is Calculated through the formula,
here,

is the number of services being processed by the edge server, j is the index of the service, Lj is the size of the jth service,

Is deadline of the th service, k is the index of the service stored in the buffer until the jth service starts,

represents the remaining task size of the task being processed by the edge server, and N represents the set of services stored in the buffer.
The difference between the processing time of the tasks stored in the buffer of the edge server and the service deadline is calculated. When the edge server is processing a task, computing resources for the task stored in the buffer are allocated to the service only when the task being processed is completed. The threshold is calculated by considering the remaining processing time of the task being processed by the edge server. If a large number of tasks are sent to the edge server during a certain period of time, the load on the edge server increases to meet the service requirements. When there is none, the threshold approaches 0, and the computing node on which the service will be processed determines/selects another new collaboration target by the reinforcement learning agent. In the opposite case, the threshold is similar to the computing resource capacity of the edge server, so the services are connected to the terminal. A cloud-multiple edge server collaboration method based on service classification in an intelligent video security environment processed by this connected edge server.