KR102094933B1 - Edge-based Load-Shedding System for Fast-Data Analysis and the operating method thereof - Google Patents

Abstract

The edge-based load shedding system according to an aspect of the present invention determines to perform server / edge partitioning, transmits some computational modules of the analysis application to the following edge machine, requests to offload, and receives preprocessing data from the following edge machine An analysis server to analyze; And an edge machine that receives and offloads the calculation module from the analysis server, preprocesses sensor data collected from the sensor using the offloaded calculation module, and transmits the preprocessing data to the analysis server.

Description

Edge-based Load-Shedding System for Fast-Data Analysis and the operating method thereof

The present invention relates to a load shedding technique based on server / edge segmentation execution for fast data analysis, and the Internet of Things (IoT), which means a connection between an object and an object, not a connection between a person and a person, is the maximum. It is becoming a hot topic, and data generated by a large number of networked systems is rapidly changing to a Fast Data environment beyond Big Data.

Unlike the big data, such a fast data environment loads collected data at high speed, and has a feature of responding to analysis results at high speed in real time or semi real time, and various techniques for stream data processing essential to such high speed data processing are being studied. have.

The present invention relates to an edge-based data filtering technique in order to analyze an application more quickly in a high-speed stream data processing process.

As many distributed applications have been used in recent years, continuous information flow has been generated from distributed systems. For example, network traffic data, sensor network data, and stock quote data used in intrusion detection systems are examples of typical information flows. In order to obtain useful information from the information flow, it must be continuously processed in a timely manner. In a traditional database management system (DBMS), this requirement is faithfully fulfilled because data must be stored and indexed before being acquired. It is difficult to perform. As an alternative, there is a data stream management system (DSMS), through which a system capable of stream processing in a continuously flowing data stream is widely used.

Due to the nature of the sensor data, it is difficult to apply general raw data directly to an analysis application, and it undergoes an initial preprocessing process in which raw data must be processed, purified, and loaded in advance.

In the conventional load shedding technology, load shedding is performed in a way that ignores some loads in consideration of server resources and network traffic, which helps in stable operation of the system, but ignores some input data to be analyzed. There is a problem that the accuracy of the drop.

An object of the present invention is to provide an edge-based load shedding method that is processed by an edge machine that collects sensor data in a pre-processing process, which is a part of the entire processing process of the analysis application, to solve the above-mentioned problems.

In the present invention, unlike the general load shedding method, when the input load increases, it is not a passive method that automatically lowers the role, but preprocesses among the calculation modules included in the analysis application according to the input data flow according to the input load. Providing an edge-based adaptive load shedding method that offloads the analysis server by a method of offloading the computation module corresponding to the process to the edge machine, and processing the input data from the edge machine by the offloaded computation module. The purpose.

In accordance with the trend of changing from a big data environment to a fast data environment, the present invention applies data analysis software by applying distributed processing technology in a method that can be applied pre-emptively before loading the input data for analysis of the existing data into the analysis application. It is an object to provide a method that can increase the processing efficiency of.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

The edge-based load shedding system according to an aspect of the present invention determines to perform server / edge partitioning, transmits some computational modules of the analysis application to the following edge machine, requests to offload, and receives preprocessing data from the following edge machine An analysis server to analyze; And an edge machine that receives and offloads the calculation module from the analysis server, preprocesses sensor data collected from the sensor using the offloaded calculation module, and transmits the preprocessing data to the analysis server.

Edge-based load shedding system including an analysis server that analyzes sensor data pre-processed by an edge machine according to another aspect of the present invention, and one or more edge machines that collect sensor data from a sensor and process the sensor data and transmit it to an analysis server. The operating method of the operation server comprises: (1) creating profile information of calculation modules included in the analysis application; (2) collecting own resource information and network information; (3) collecting resource information of the edge machine; (4) determining to perform server / edge partitioning based on the profile information, its own resource information and network information, and resource information of the edge machine; (5) determining an edge machine to be offloaded and a calculation module to be offloaded; (6) transmitting the operation module to be offloaded to the edge machine to be offloaded; And (7) requesting to offload the edge machine to be offloaded.

According to the present invention, a load shedding effect of the analysis server occurs by distributing the load of the analysis server using an edge machine. Since load balancing is performed in consideration of the current situation of the analysis server and the edge machine, there is an effect of constructing a more efficient system.

Since multiple edge machines have the characteristics of a parallel system, it is easy to add / remove the edge machines in consideration of the overall system load, and the recent advancement in technology has improved the price / performance ratio of the edge machines, making the analysis server system large. It has the effect of building an economical system without it.

In the present invention, each edge machine is not manually managed using an offloading technique, but has an effect that can be efficiently managed using an analysis server.

The load shedding method capable of executing server / edge segmentation based on operator offloading proposed by the present invention enables server / edge collaborative data analysis in a conventional server-centric data analysis, and is generated in the Internet of Things (IoT). It will have the effect of analyzing data more intelligently.

1 is an exemplary view for explaining a load reduction method in a system to which a load shedding technique is applied.
2 is a block diagram of an edge-based road shedding system according to the present invention.
3 is a block diagram for explaining a method of operating a load shedding system by offloading at an edge machine.
4 is a structural diagram for explaining the analysis process of the load shedding system in the absence of offloading.
5 is a block diagram for explaining a method of operating a road shedding system by offloading according to the present invention.
6 is an exemplary view for explaining a frequency-based load shedding method.
7 is an exemplary view for explaining a load shedding method by offloading according to the present invention.
8 is a block diagram illustrating an offloading procedure in a child daemon according to a partial embodiment of the present invention.
9 is a block diagram illustrating the overall configuration of a road shedding system according to the present invention.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and the ordinary knowledge in the technical field to which the present invention pertains. It is provided to fully inform the holder of the scope of the invention, and the invention is only defined by the scope of the claims. Meanwhile, the terms used in the present specification are for explaining the embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" and / or "comprising" refers to the components, steps, operations and / or elements mentioned above, the presence of one or more other components, steps, operations and / or elements. Or do not exclude additions.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram for describing a load reduction method in a system to which a load shedding technique is applied.

Load shedding (Load Shedding) generally increases the data input from the data stream management system (DSMS, Data Stream Managed System), and when the data input rate is higher than the limit load that the processing engine can handle, the input data is appropriately It is a technique that performs the role of automatically lowering the load by discarding.

Consider a server in which the input request load increases with time as shown in FIG. 1. After a certain period of time, the load reaches 100%. In fact, when the load reaches 100%, the server does not operate normally, so when the load reaches a certain level, a load shedding technique is applied. This is called the input load adaptive load shedding method. According to this, some data is ignored according to the load shedding method, so that the input load level is maintained at a constant level.

Here, the load is a value determined according to network traffic, CPU performance, memory capacity, etc. If only one load reaches 100%, the server may not operate normally regardless of other factors. The load shedding (load reduction) method has the disadvantage that some input data is ignored, but it has the advantage that the server operates normally even if the load input to the server is high.

In the load shedding method according to the partial embodiment of the present invention, the analysis server measures the load according to the resources of the server and network resources in real time, and when the load is below a certain level, the load shedding method is not used, and the analysis server 210 The sensor data generated by the sensor 230 is received and processed through the edge machine 220. When the load exceeds a certain level, the analysis server 210 makes a data load shedding request (offloading request of a specific operation module) to the edge machine 220 to receive data preprocessed by the edge machine.

2 is a structural diagram of an edge-based road shedding system according to the present invention.

The load shedding system according to the present invention includes one or more analysis servers 210 that analyze data, one or more edge machines that receive sensor data from the sensor 220 and transmit the processed data to the analysis server as it is or process the data. 220), one or more sensors 230 for transmitting sensor data to the edge machine 220.

The sensor data refers to raw data generated by the sensor 230, and pre-processing data refers to data processed by one or more specific calculation modules and transmitted to a distributed server.

The analysis server 210, the edge machine 220, and the sensor 230 are connected by a network. Typically, between the analysis server 210 and the edge machine 220 is an Ethernet protocol conforming to IEEE 802.3 or IEEE 802.11 or lower compatibility. It may be connected to a wireless network conforming to, and between the edge machine 220 and the sensor 230 may be connected to a sensor network conforming to IEEE 802.15.4. However, the scope of the invention is not limited according to the type of the network protocol connected.

Here, the edge machine 220 includes various devices capable of processing information, such as a mobile phone, a PC, a server, and a portable terminal. In addition, the edge machine 220 may be a light and inexpensive device such as a Raspberry Pi that can collect sensor data and perform a pre-processing process to transmit it to an analysis server.

The edge daemon running on the edge machine 220 can receive sensor data from the sensor 230 and deliver it to the analysis server 210 as it is, or process the sensor data and transmit it to the analysis server 210, in this sense the edge The machine 220 also serves as a gateway.

3 shows a block diagram for explaining a method of operating a load shedding system by offloading at an edge machine.

The edge machine 220 executes an edge daemon, and the edge daemon includes a parent process and a child process.

Some operations of the analysis process to be performed by the analysis server are separated and performed by the edge machine 220, which is referred to as off-loading.

As a method of offloading (a) A method of exchanging and processing a file (input / output parameter mapping table) that describes input parameter information, output parameter information, and specifications for converting input parameters to output parameters, (b) from an analysis server As a method of receiving an operation module that is part of the analysis application and executing it on the edge machine, a method of offloading an operator to offload to the child daemon 320 while the child daemon 320 is running, (c) execution of the child daemon 320 You can use the method of re-executing the child daemon, including the operation module to temporarily stop and offload.

The method (a) has a narrow width that can be applied, and the method (c) can be applied to a variety of operation modules. However, since the service is interrupted while stopping the execution of the child daemon, there is a big overhead, so there is a disadvantage. As a method of loading the loading operation module into the child daemon, the method (b) may be the main means. However, in some cases, it may not be possible to offload the operation module by the method (b), and in this case, the method (c) is used.

The fast data system may not be able to predefine the format of raw data because it has the characteristics of processing real-time requests with extremely high update rates and targeting inaccurate and various unstructured data. In addition, when a new sensor is added to generate atypical sensor data to be analyzed, it may be necessary to modify the analysis application in accordance with the data format of the corresponding sensor data.

In principle, the sensor data of a specific sensor is transmitted to the analysis server when the analysis server is equipped with the analysis tool, but the sensor data may reach the analysis server without the analysis server equipped with the analysis server. Sensor data is ignored.

If you have a post-analysis tool, you can offload some of the operations included in the analytics application of the analytics server for that sensor data to the edge machine. The offloading function is very useful as an adaptive analytic tool when analyzing a sensor that generates new types of sensor data. As described above, a method of functionally dividing the calculations of the analysis server 210 and the edge machine 220 by offloading some operation modules to the edge machine is called server / edge division execution.

The edge daemon of the edge machine 220 includes the parent daemon 310 and the child daemon 320, and the parent daemon 310 can execute the child daemon 320 and stop the execution. That is, the parent daemon 310 serves to manage the child daemon 320. The parent daemon receives and stores the offloading calculation module (the calculation module that can be offloaded from the edge machine as part of the analysis application) from the analysis server, and requests to be executed by offloading the calculation module where the child daemon is stored. The parent daemon stores the offloading operation module file in a predetermined location so that the child daemon can access the operation module to be offloaded and offload the operation module. It is also possible to kill the child daemon and re-run the child daemon so that the child daemon can perform the offloading operation module offloading.

If the input data and output data of the edge machine are both text type or binary format, it is possible to map a predefined data structure, and the input data data format, output data format, and input data output data matching definition file ( It is possible to offload specific operations by exchanging input / output data mapping table). In essence, this corresponds to data offloading, but in the case of simple data processing, it is possible to implement the same function as directly copying an operation module.

At this time, the format of input / output data can use JSON format, markup language, SQL data format, and user-defined format, and input / output data mapping table can also be written in the above format, so even if the OS of the edge machine or the program used differs, JSON or The xml parser is a library provided by most programming languages, so it can be offloaded in a field matching format.

The child daemon 320 may be multi-processor, or multi-threaded despite the name. Depending on the resource conditions of the edge machine, one or more child daemons are executed, but all child daemons basically perform the same function. Since the external sensor can input data to any child daemon running on the connected edge machine, when the calculation module in which the child daemon is offloaded is different, the sensor 230 inputs sensor data to the child daemon that is not offloaded. In this case, execution of the operation module may fail. However, even if the necessary operation module is not offloaded, the child daemon can transmit the input data to the analysis server as it is. When the calculation module is offloaded to the child daemon, the preprocessed data is transmitted to the analysis server after being preprocessed using the offloaded calculation module.

4 shows a structural diagram for explaining an analysis process of a load shedding system when there is no offloading.

The edge machine in Fig. 4 functions as a gateway and has no other functions. Recently, many lightweight, inexpensive, and cost-effective devices such as the Raspberry Pi have been developed, making it possible to perform some of the functions provided by conventional servers in addition to the data collection and transmission functions.

The analysis server outputs an analysis result using various calculation modules (operation 1, operation 2, operation 3, operation 4). The analysis server in FIG. 4 is a case where the server / edge division is not executed, and the sensor data can be analyzed, and the data preprocessed by the edge machine can be analyzed. To this end, the analysis server can distinguish whether the data received from the edge machine is sensor data or data that has been preprocessed, and when the sensor module receives the sensor data as it is not offloaded, the analysis server performs all of the analysis applications. Operation is performed.

Hereinafter, an operation performed by each operation module will be described with specific examples. For analysis of log data, the input data is in the form of "Warning: server is overloaded", "Error: server network connections are refused", "Error: too many connections", "Warning: out of memory", and the network When analyzing to know the number of related errors in (network), for example, operation 1 is an operation module that divides log data into space units, and operation 2 extracts that the first word in each sentence is "Error" Operation module, operation 3 is the operation module for counting the number of words included in the sentence containing "Error", operation 4 is sorted by the number of words counted in operation 3, and the number of words counted is the same number of logs Speaking of the calculation module, it is possible to analyze the list data in order of the number of words among the error logs generated by sequential calculation. This analysis is not a problem even if it is all done in the analysis server, but some operations can be divided and executed on the edge machine, which will be further described in FIG. 5.

5 is a block diagram illustrating a method of operating a load shedding system by offloading according to the present invention.

Although data collection is performed as shown in FIG. 4, in FIG. 5, operation 1 and operation 2 of the operation modules are executed in the edge machine 220, and operation 3 and operation 4 are the biggest differences in that they are executed in the analysis server 210. This is called server / edge split execution. Here, operation 1 and operation 2 may be off-loading operation modules, and operation 3 and operation 4 may be off-loading operation operation modules.

When the input load of the analysis server is low, even if it is executed as shown in FIG. 4, the stability of the analysis server is not significantly affected. However, as shown in FIG. 1, when the input load exceeds a certain level, the analysis server 210 does not operate normally, and thus, some of the analysis modules of the analysis application of the analysis server 210 are turned off to the edge machine 220. It is loaded and executed.

The analysis server includes one or more analysis applications, and each analysis application includes one or more calculation modules. The analysis server has profile information about the calculation module, such as the execution order of the calculation module, the amount of resources required when the calculation module is executed, the information of the input parameters, the information of the output parameters, and the mapping table of the input / output parameters, and has a specific edge. When offloading a specific operation module to a machine, the profile information of the operation module is also transmitted to the edge machine, so the edge machine executes the operation order of the operation module and input parameters and outputs of each operation module based on the received profile information. Know the format of the parameters.

The required resource amount is a value calculated in consideration of CPU usage, memory usage, etc. required when the operation module is executed, and the mapping table of the input / output parameters means a file that describes specifications for converting input parameters to output parameters. In some calculation modules, the same effect as offloading the calculation module can be obtained by transmitting the mapping table to the edge machine.

The analysis server 210 collects information about the current system. The specifications of the edge machine, that is, information such as CPU performance, memory capacity, storage space of the edge machine, the current load of the edge machine, whether the current computation module is offloaded, the traffic volume of the network between the analysis server and the edge machine, and Information such as network bandwidth is collected.

Based on the collected information, the analysis server decides whether to offload the computation module to the edge machine.

If the analysis server decides to offload a specific computational module to a specific edge machine, the analysis server sends the computational module to the edge machine, stores the computational module received by the edge machine in storage, and offloads from the analysis server. When a request is received, the calculation module is offloaded and input data is preprocessed using the offloaded calculation module.

Even if the operation module is offloaded at the edge machine, input data may not be preprocessed using the offloaded operation module. This is because server / edge partition execution is not always efficient from the perspective of the entire system depending on the characteristics of the operation module and the system environment. However, since the offloading process has a large overhead, it is only possible to offload the calculation module and determine whether to process data with the offloaded calculation module according to the situation.

If the load of the edge machine is already high before the edge machine is offloaded, the sensor machine has insufficient resources by simply sending the sensor data to the analysis server as a large amount of sensor data flows from the sensor connected to the edge machine. Since it can be determined that the specific calculation module is already offloaded, the offloading calculation module may not be used. However, the CPU load and the memory load of the edge machine are sufficient, but when the network usage is high, if the output data is smaller than the input data by the offloaded operation module, an offloading request may be made. Whether the output data is smaller than the input data can be determined from input parameter information and output parameter information.

The CPU and memory of each edge machine are generally smaller in performance and capacity than the analysis server, but since the edge machine 220 is usually composed of a plurality, the edge machine 220 performs some functions of the analysis server 210. Just by doing this, it functions as load shedding (load reduction).

In terms of distributing the functions of the edge machine 220 and the analysis server 210, the load shedding system can be viewed as a kind of distributed processing system, and there is a difference in that each edge machine offloads different operators, but generally Multiple edge machines perform the same function.

One or more edge machines 220 may be viewed as a parallel processing system, and when a high-load edge machine is present, an inexpensive edge machine can be easily added to reduce the overall load level of the edge machine.

Although it is difficult to manage a plurality of edge machines, the analysis server 210 can manage the edge machine using the offloading technology presented in the present invention, and such management is automatically performed according to the situation of the system. It can operate a stable system while reducing the burden of manual maintenance.

The analysis server has necessary resource information such as CPU load and memory load required when the operation module to be offloaded is executed. The analysis server may generate necessary resource information in consideration of the value obtained by testing it by itself and the performance of the edge machine, and may generate required resource information by receiving the required resource information calculated by the edge machine. When the edge machine periodically calculates and delivers the necessary resource information to the analysis server when executing the offloading calculation module, the analysis server can update the required resource information owned by the weighted server.

The required resource information may be used to determine whether to offload by comparing the free resource information of the edge machine with the required resource information for executing the offloading of the corresponding operation module when offloading a specific operation module to a specific edge machine. .

Due to the system configuration, tasks such as extracting specific information from all collected data are processed by the analysis server only as an offloading-notable computation module.

The edge machine 220 can pre-process the pre-processing operation modules such as data purification, data extraction, and data verification, and has an effect of reducing the load of the analysis server as the pre-processing process progresses. The data pre-processed in the machine may have a smaller capacity than the input data, and in the pre-processing, meaningless sensor data is not transmitted to the analysis server, so traffic is also reduced on the network side.

As an actual example, when describing the situation of FIG. 5, as shown in FIG. 4, input data is "Warning: server overloaded", "Error: server network connections are refused", "Error: too manay connections", "Warning: out of memory If it is analyzed in order to know the number of errors related to the network in the same format as ", operation 1, which is an operation module that divides log data into space units, and the first word in each sentence is" Error " Arithmetic module 2, which is an operation module that extracts the thing, is executed on the edge machine, and counts by sorting based on the number of words counted in operations 3 and 3, which are operations modules that count the number of words included in the sentence containing "Error" Operation 4, which is an operation module that calculates the number of logs having the same number of words, is executed in the analysis server to perform server / edge division execution. The results of analyzing the list data in order of the number of words among the error logs generated by sequential operations while reducing the load on the analysis server are as shown in FIG. 4.

6 is an exemplary diagram for explaining a frequency-based load shedding method.

Among the techniques related to load shedding, the most common method is a sampling-based method. Reducing the load by ignoring the appropriate number of input data based on probability considering the resource and network bandwidth of the analysis server. This is a method of sampling and processing only part of the data generated in t0, t01, t02, and t03 by time as shown in FIG. 6. Data is collected from the edge machine 220, and operation 1, operation 2, operation 3, and operation 4 are sequentially performed in the analysis server.

For example, in FIG. 6, if only two input data are to be analyzed for each time section in the current load situation, two input data are selected for each section by a random generator. If there are two or fewer, select all, and for a time period in which two or more input data exist, determine the probability of the random generator so that two or more input data can be selected.However, if two input data are already selected, the remaining data is Ignoring, if the number of remaining data is the same as the number of data to be selected, all the remaining data can be selected so that two input data can be selected. The number of input data to be selected is not the same in all sections and is set differently according to the load situation of the analysis server.

In FIG. 6, black dots indicate two input data selected in the corresponding section.

7 is an exemplary view for explaining a load shedding method by offloading according to the present invention.

The same input data as in Fig. 6 will be described. For example, in the case of an operation for filtering within a range of a specific temperature, by offloading this operator to the edge daemon, the load shedding operator is filtered by an offloaded operator, rather than sampling by a random variable in each time interval. Only data is sent to the analysis server. As a result, a result of load shedding may be obtained by a distributed processing technique that performs some operations on the edge machine 220.

Considering the case of FIG. 5, since the processed data preprocessed only the sensor data starting “Error” among the entire log data is transmitted to the analysis server, the load of the analysis server can be reduced.

In FIG. 7, the input data indicated by a black dot means data determined by the edge daemon to be filtered by the offloaded operator and transmitted to the analysis server. The transmitted data is different, but the load shedding effect will be similar.

However, while FIG. 6 removes randomly selected input data by a random variable, FIG. 7 has an advantage of reducing the load of the analysis server while processing all necessary data. In addition, since the load of the analysis server is reduced as the pre-processing is performed on the edge machine, the number of input data to be processed for each time section can be set higher in FIG. 6.

The offloaded calculation module is limited to comparatively simple calculations excluding large data processing and data statistics calculation, which are preferably processed by the analysis server. It can be easily applied to pre-processing operations such as data processing before comprehensive analysis of the analysis server.

8 is a block diagram illustrating an offloading procedure in a child daemon according to a partial embodiment of the present invention.

The child daemon 320 transmits the collected data to the analysis server 210 without processing, or receives an offloading execution command from the parent daemon 310 and processes the data by an operator that offloads the collected data. , Serves to transmit the processed data to the analysis server 210.

The memory 840 may be shared by the child daemon 320 in the form of shared memory or semaphore. This technique is also applicable between the child daemon 320 and the parent daemon 310.

The child daemon 320 includes a data collection unit 810, a data processing unit 820, and a data output unit 830.

The data collection unit 810 collects sensor data from the sensor, and the data processing unit 820 processes the sensor data according to whether the operator is offloaded. The data transmission unit 830 transmits the processed data to the analysis server 210.

The child daemon 320 may further include an offloading control unit.

The analysis server sends an operation module to be offloaded to the parent daemon 310, and makes an offloading request to the parent daemon 310. The parent daemon 310 stores the received operation module and sends it to the child daemon 320. When the offloading operation module stores the location, and requests the child daemon 320 to offload the operation modules (operation 1 and operation 2 in FIG. 5), and the child daemon offloads the operation module In, the parent daemon 310 requests the child daemon to execute the offloaded operation module so that the child daemon 320 can preprocess the sensor data in the data processing unit. The offloading control unit 870 of the child daemon 310 receiving this, offloads the corresponding calculation module, and the data processing unit 820 executes the corresponding calculation module.

When the operation module cannot be offloaded through the offloading control unit 870, the parent daemon 310 may stop execution of the child daemon 320 and re-execute it so that it can be offloaded.

The processing data (output data) preprocessed by the offloaded calculation modules is transmitted by the data transmission unit 830 to the analysis server 210.

9 is a block diagram illustrating the overall configuration of a road shedding system according to the present invention.

After the sensor data generated from the sensor 230 is collected by the data collection unit 810 of the child daemon 320, and processed by the data processing unit 820, the data transmission unit 830 transmits the data to the analysis server. The data receiving unit 910 of the analysis server 210 receives this, and the data analyzing unit 915 analyzes it, and the output unit 920 outputs it.

The profile generation unit 930 of the analysis server 210 generates a profile containing information such as whether an operation module can be offloaded, required resource amount, execution time, input parameter information, output parameter information, and input / output parameter mapping table, The load shedding control unit determines whether to load shed according to the profile information.

The first monitoring unit 935 collects server resource information such as CPU usage, memory usage, network traffic, and network bandwidth of the server, and the load shedding control unit determines whether to load shed based on the server resource information.

The third monitoring unit 950, from the second monitoring unit 955 of the edge machine 220, the CPU performance, CPU usage, memory capacity, memory usage, number of offloaded operators, the number of connected sensors, etc. Receives the edge resource information and the status information of the load shedding control unit 940 based on the resource information of the edge machine existing on the entire network to determine whether or not load shedding.

The load shedding control unit 940 determines that the load shedding means in the present invention that the load shedding control unit determines to execute server / edge partitioning by offloading a specific operation module to a specific edge machine.

First, when the operator module 960 transmits an operation module to be offloaded to the operator receiver 965 of the parent daemon 310 running on the edge machine 220, the operator receiver 965 receives it and stores it in the storage. To save.

The offloading request unit 970 requests to offload the operation module transmitted to the offloading unit 973, and if the operation module is not transmitted, no action is taken or the operator transmission unit 960 is operated Request retransmission of the module. In order to reduce the system overhead, the offloading request unit does not perform any action until a new offloading request is received from the load shedding control unit 940. When the calculation module is already offloaded, the control request unit 976 requests the offloading control unit 870 to process data by the offloading calculation module.

After the calculation module is transmitted, the offloading unit 973 requests the offloading control unit 870 to actually offload the calculation module to the data processing unit. Subsequently, the control request unit 976 requests the offloading control unit 870 to process data by the offloading operation module.

The configuration of the present invention has been described in detail with reference to the accompanying drawings, but this is only an example, and those skilled in the art to which the present invention pertains have various modifications and changes within the scope of the technical spirit of the present invention. Of course this is possible. Therefore, the protection scope of the present invention should not be limited to the above-described embodiments and should be defined by the following claims.

210: analysis server
220: edge machine
230: sensor
310: parent daemon
320: child daemon
810: data collection unit
820: data processing unit
830: data transmission unit
840: memory
850: processor
860: receiver
870: offloading control
910: data receiving unit
915: data analysis department
920: output
930: Profile generation unit
935: first monitoring unit
940: road shedding control
950: third monitoring unit
955: second monitoring unit
960: operator transmission
965: operator receiver
970: offloading request unit
973: offloading unit
976: control request unit

Claims (13)

An analysis server for deciding to perform server / edge partitioning, requesting to send and offload some computational modules of the analysis application to the following edge machine, and receiving and analyzing preprocessing data from the edge machine; And
An edge machine for receiving and offloading a calculation module from the analysis server, preprocessing sensor data collected from a sensor using an offloaded calculation module, and transmitting preprocessing data to the analysis server,
The analysis server,
An edge-based load shedding system that determines whether to perform server / edge partitioning based on profile information, and wherein the profile information includes execution order and required resource amount of the calculation module, information of input parameters, and information of output parameters.
According to claim 1,
The analysis server is to determine whether to execute the server / edge partition, considering the CPU usage, memory usage and network traffic
Edge-based road shedding system.
According to claim 1,
The analysis server is to determine whether to perform server / edge partitioning based on CPU usage and memory usage information of the edge machines received from one or more edge machines.
Edge-based road shedding system.
delete delete According to claim 1,
The analysis application is characterized in that it comprises an offloadable module and a non-offloadable module.
Edge-based road shedding system.
According to claim 1,
The edge machine
A child daemon that controls offloading of the computation module, collects sensor data from the sensor, preprocesses it, and transmits it to the analysis server;
Parent daemon that monitors its own CPU usage and memory usage, transmits monitoring information to the analysis server, receives and stores a computation module from the analysis server, and requests the child daemon to pre-process the data by offloading the computation module. Characterized in that it comprises
Edge-based road shedding system.
In the operating method of the edge-based load shedding system comprising an analysis server for analyzing the sensor data pre-processed by the edge machine, one or more edge machines to collect the sensor data from the sensor and process the sensor data to transmit to the analysis server,
The analysis server
(1) creating profile information of calculation modules included in the analysis application;
(2) collecting own resource information and network information;
(3) collecting resource information of the edge machine;
(4) determining to perform server / edge partitioning based on the profile information, its own resource information and network information, and resource information of the edge machine;
(5) determining an edge machine to be offloaded and a calculation module to be offloaded;
(6) transmitting the operation module to be offloaded to the edge machine to be offloaded;
(7) requesting to offload the edge machine to be offloaded;
The profile information includes the execution order of the operation module and the required resource amount, information of an input parameter, and information of an output parameter.
delete The method of claim 8,
The resource information of itself includes CPU usage, memory usage, and network traffic.
How to operate the edge-based road shedding system
The method of claim 8,
The required resource amount is when the calculation module is executed on the edge machine,
This is a value calculated considering the required CPU load, memory load, and execution time of the calculation module.
How to operate an edge-based road shedding system.
The method of claim 8,
The edge machine
(a) collecting sensor data from the sensor;
(b) processing the collected data using an offloaded operator;
(c) transmitting the processed data to an analysis server;
The analysis server
(d) analyzing the received data; And
(e) outputting the analyzed data
Method of operating an edge-based road shedding system further comprising a.
The method of claim 8,
The edge machine
① Monitoring information of own CPU usage and memory usage;
② receiving a calculation module from the analysis server;
③ storing the operation module in a storage;
④ offloading the calculation module stored in the storage; And
⑤ requesting data processing by the offloaded operation module;
Method of operating an edge-based road shedding system further comprising a.

Images