KR102648743B1 - Apparatus for determining of tree, network switch apparatus, system of in-network data aggregation including the same and method thereof - Google Patents

Abstract

It relates to an in-network management device, a network switch, and an in-network data collection system and method, wherein the in-network management device is based on a communication unit that receives information about at least one network switch device and information about the network switch device. Generate a plurality of aggregate trees corresponding to a plurality of queries, merge the plurality of aggregate trees, and create a reconstructed aggregate tree for at least one aggregate tree among the plurality of aggregate trees based on the merged aggregate tree. May include a processor that generates

Description

In-network management device, network switch, in-network data aggregation system and method {APPARATUS FOR DETERMINING OF TREE, NETWORK SWITCH APPARATUS, SYSTEM OF IN-NETWORK DATA AGGREGATION INCLUDING THE SAME AND METHOD THEREOF}

It relates to an in-network management device, a network switch, an in-network data aggregation system, and an in-network data aggregation method.

Previously, the data plane simply performed a packet forwarding function between endpoints or between endpoints and servers, but recently, programmable data plane technology has enabled it to perform more diverse functions. Programmable data plane technology is a technology for programming network devices such as programmable network switches or network interface cards using a programming language related to data plane operation, such as P4. Using these programs, it is possible to define the operations of the network switch's parser, match-action stage, and de-parser, allowing users to flexibly provide the functions they want and process the functions more quickly.

As data plane technology develops, in-network computing technology is also growing. In-network computing technology is a technology that allows all or part of a series of computing processes performed in conventional server devices or terminal devices to be performed by a network switch or router within a network. This in-network computing not only reduces the delay time caused by packets requiring computing being diverted to the computing server, but also allows computing to be performed based on high-performance hardware instead of software-based computing, allowing for faster computation. Processing is possible.

Among in-network computing technologies, in-network aggregation technology exists. In-network aggregation technology is a technology in which an intermediate node (programmable network switch, etc.) located on the data transmission path receives data from other nodes or terminals, aggregates the received data, and then delivers it to other nodes or terminals. It's technology. In-network integration technology has recently received a lot of attention due to its advantages in terms of processing delay and traffic load reduction. However, programmable network switches usually have extremely small memory capacities of tens of megabytes, while the data to be transmitted and processed is much larger, sometimes hundreds of megabytes in size. Due to the relatively insufficient memory capacity of the network switch compared to the data capacity, the aggregation of data transmitted between nodes can only be performed partially, which is a major obstacle to improving in-network aggregation performance.

Meanwhile, data center networks are increasingly using software multitenancy technology to simultaneously provide necessary functions to multiple tenants. Even in the case of multi-tenancy technology, in-network integration is necessary for reasons such as resource constraints or traffic reduction, and recently, an equal division method has been used for such in-network integration. The equal division method is a technology that divides the available resources of a network switch equally and allocates them exclusively to each function in order to integrate functions that operate simultaneously. However, the equal division method of resources has a problem in that a specific memory partition is not used depending on the number of functions being executed, or a memory partition allocated to a specific function cannot be reused for other functions. This causes waste of network switch resources and deteriorates in-network integration performance.

The goal is to provide an in-network management device, an in-network data aggregation system, and an in-network data aggregation method that can optimally select a network switch and determine the size of the memory partition considering the amount of generated data and available resources. Make it a task.

In addition, another problem to be solved is to provide a network switch that can perform more aggregation even with a relatively small memory capacity.

In order to solve the above-mentioned problems, an in-network management device, a network switch, and an in-network data collection system and method are provided.

The in-network management device generates a plurality of aggregate trees corresponding to a plurality of queries based on a communication unit that receives information about at least one network switch device and information about the network switch device, and the plurality of aggregate trees It may include a processor that merges and generates a reconstructed aggregate tree for at least one aggregate tree among the plurality of aggregate trees based on the merged aggregate tree.

The network switch device determines whether the slot of the hash table and the communication unit that receives the key-value pair from the operation server device or another network switch device are empty, and if the slot of the hash table is empty, the received key-value pair is A processor that stores the pair in the slot and, if the slot in the hash table is not empty, compares the key of the key-value pair previously stored in the slot with the received key, and updates the slot according to the comparison result. can do.

The in-network data aggregation method includes receiving information about at least one network switch device, generating a plurality of aggregation trees corresponding to a plurality of queries based on the information about the network switch device, and the plurality of aggregation trees. It may include merging aggregate trees and generating a reconstructed aggregate tree for at least one aggregate tree among the plurality of aggregate trees based on the merged aggregate trees.

The in-network data aggregation method includes the steps of: receiving, by at least one network switch device, a key-value pair from an operation server device or another network switch device; determining whether a slot in a hash table is empty; If the slot is empty, storing the received key-value pair in the slot; if the slot in the hash table is not empty, comparing the key of the key-value pair previously stored in the slot with the received key. , If the key pre-stored in the slot is different from the received key according to the comparison, the key pre-stored in the slot is extracted, and the received key-value pair is replaced with the key pre-stored in the slot to replace the key pre-stored in the slot. Comparing the stored coefficient result corresponding to the key pre-stored in the slot with the coefficient result corresponding to the key pre-stored in another slot, and the stored coefficient result corresponding to the key pre-stored in the slot in the other slot If it is smaller than the coefficient result corresponding to the key previously stored in the other slot, extracting the key previously stored in the other slot, replacing the key-value previously stored in the slot with the key previously stored in the other slot and storing it in the other slot. It may also include .

The in-network data aggregation system generates a plurality of aggregation trees corresponding to a plurality of queries based on at least one network switch device and information about the network switch device, merges the plurality of aggregation trees, and merges the merged aggregation trees. It may include an in-network management device that generates a reconstructed aggregate tree for at least one aggregate tree among the plurality of aggregate trees based on the aggregate tree.

According to the above-described in-network management device, in-network data aggregation system, and in-network data aggregation method, network switches are optimally selected in consideration of the amount of data and available resources in an environment requiring multi-tenancy, and aggregation is performed based on this. You can achieve the effect of being able to determine the route.

According to the above-mentioned network switch, it is possible to achieve the effect of performing more aggregation even with a relatively small memory capacity.

According to the above-described in-network management device, network switch, in-network data aggregation system, and in-network data aggregation method, the effect of being able to adaptively determine the size of the memory partition of each optimally selected network switch can be obtained. .

According to the above-described in-network management device, network switch, in-network data aggregation system, and in-network data aggregation method, aggregation performance on the network path can be improved, thereby shortening the processing time for data and You can also achieve the effect of reducing network traffic.

1 is a schematic diagram of one embodiment of an in-network data aggregation system.
Figure 2 is a diagram of one embodiment of an aggregation tree.
Figure 3 is a block diagram of one embodiment of an in-network management device.
Figures 4 to 11 are first to eighth figures for explaining an embodiment of the operation of the in-network management device.
Figure 12 is a block diagram of an embodiment of a network switch device.
Figures 13 and 14 are first and second figures for explaining an example of integrated operation of a network switch device.
Figure 15 is a graph measuring the change in integration performance according to the number of generated data types.
Figure 16 is a graph showing the change in aggregation performance according to the aggregation path setting method in a multi-tenancy environment in which three applications operate simultaneously.
Figure 17 is a flowchart of an embodiment of an in-network data aggregation method.
Figure 18 is a flowchart of an embodiment of an integrated operation process of a network switch device.

Throughout the specification below, the same reference signs refer to the same components unless there are special circumstances. Terms with the addition of 'unit' used below may be implemented as software and/or hardware, and depending on the embodiment, one 'unit' may be implemented as one physical or logical part, or a plurality of 'units' may be implemented. It is also possible to be implemented with one physical or logical part, or one 'part' to be implemented with a plurality of physical or logical parts. When a part is said to be connected to another part throughout the specification, this may mean that the part and the other part are physically connected to each other and/or electrically connected. In addition, when a part includes another part, this does not mean excluding another part other than the other part unless specifically stated to the contrary, and means that another part may be included depending on the designer's choice. do. Expressions such as the first to Nth (N is a natural number greater than or equal to 1) are intended to distinguish at least one part(s) from other part(s), and do not necessarily mean that they are sequential unless otherwise specified. Additionally, singular expressions may include plural expressions, unless the context clearly makes an exception.

Hereinafter, an embodiment of an in-network data aggregation system including an in-network management device and a plurality of network switch devices will be described with reference to FIGS. 1 to 16.

FIG. 1 is a schematic diagram of an embodiment of an in-network data aggregation system, and FIG. 2 is a diagram of an embodiment of an aggregation tree.

Referring to FIG. 1, the in-network data aggregation system 1 may include at least one network switch device 10 (10-1 to 10-6) and at least one in-network management device 100. , Depending on the embodiment, it may include at least one operation server device 90 (90-1 to 90-k). At least two of each network switch device (10: 10-1 to 10-6), operation server device (90: 90-1 to 90-k), and in-network management device (100) transmit data one-way or two-way. It may be connected through at least one of a wired communication network and a wireless communication network to transmit commands, commands, etc. The number of each of the network switch devices (10: 10-1 to 10-6), the operation server device (90), and the in-network management device (10) may be arbitrarily determined by the user, operator, or designer (hereinafter referred to as user, etc.). there is. The wireless communication network may include at least one of a short-range wireless communication network (Wi-Fi, Wi-Fi Direct, Bluetooth, or Bluetooth Low Energy, etc.) and a long-range wireless communication network (HSPA, HSPA+, LTE, LTE-Advanced, 5G or 6G, etc.).

As shown in FIG. 2, the in-network management device 100 communicates with an external terminal device (2, smartphone, tablet PC, desktop computer or laptop computer, etc.) through a wired or wireless communication network, and if Upon receiving data, commands, or queries from the terminal device 2, the data, commands, or queries may be transmitted to each operation server device 90 (90-1 to 90-k). In this case, the in-network management device 100 divides data, commands, or queries, etc. according to the operation server devices (90: 90-1 to 90-k), and sends the divided data, commands, or queries to each By assigning and transferring to the corresponding operation server device (90: 90-1 to 90-k) among the operation server devices (90: 90-1 to 90-k), the operation server device (90: 90-1 to 90) -k) You can also have each handle its assigned portion of data, commands, or queries. In addition, the in-network management device 100 includes at least one of a plurality of network switch devices 10: 10-1 to 10-12 (for example, 10-1, 10-2, 10-3, 10-5, 10). -7, 10-8, 10-10, and 10-12, etc.) are used to determine and generate at least one aggregate tree (T1, an aggregate path), and at least one aggregate tree (T1) is operated by the operation server device 90. : 90-1 to 90-k) or a network switch device (10: 10-1 to 10-12). Here, at least one aggregate tree (T1) may include a multi-tenancy-aware aggregate tree.

The operation server devices 90-1 to 90-3 independently or dependently process data, commands, or queries received from the in-network management device 100, and send the processing results to a plurality of network switch devices. It can be transmitted through an aggregate tree (T1) built based on (10: 10-1 to 10-12). In this case, the operation server devices 90-1 to 90-3 are network switch devices that will first receive the processing results within the aggregation tree (T1) so that the processing results can be aggregated and delivered according to the given aggregation tree (T1). The processing results can be delivered to (10-1, 10-2).

The network switch device (10: 10-1 to 10-12) may aggregate the processing results and transmit the integrated result to the next connected network switch device (10: 10-1 to 10-12). In this case, the network switch device (10: 10-1 to 10-12) corresponding to the generated aggregation tree (T1) may receive the processing result or the aggregation result and perform aggregation of the processing result or aggregation result. . Results aggregated via one or more network switch devices (e.g. 10-1, 10-2, 10-3, 10-5, 10-7, 10-8, 10-10, and 10-12) Can be finally transmitted to the in-network management device 100, and the in-network management device 100 can store the aggregated processing result or transmit it to the terminal device 2, etc.

Hereinafter, the specific operation of the in-network management device 100 will be described.

FIG. 3 is a block diagram of an embodiment of an in-network management device, and FIGS. 4 to 11 are first to eighth views for explaining an embodiment of the operation of the in-network management device.

Referring to FIG. 3, the in-network management device 100 performs communication with at least one of the operation server devices 90-1 to 90-3 and the network switch devices 10: 10-1 to 10-12. It may include a communication unit 101 capable of performing operations, and a processor 110 for determining the tree T1 and distributing data. Depending on the embodiment, the in-network management device 100 transmits data, commands, or queries transmitted from the terminal device 2, etc., acquired during processing of the processor 110, or acquired according to the processing result of the processor 110. A storage unit (102, which may include at least one of a main memory and an auxiliary memory) that temporarily or non-temporarily stores information, etc., and a storage unit (102) that receives commands from a user, etc. and visually or audibly provides processing results, etc. to the user, etc. It further includes a user interface (105, which may include an input unit such as a keyboard or mouse device and/or an output unit such as a display or speaker device, and may also include a communication module or a data input/output module, etc. depending on the embodiment). You may.

The processor 110 may include an information collection unit 111, a tree construction unit 112, a tree reconstruction unit 113, and a distribution processing unit 114. The information collection unit 111, tree construction unit 112, tree reconstruction unit 113, and distribution processing unit 114 may be logically and/or physically separated.

As shown in FIG. 4, the information collection unit 111 uses the communication unit 101 to collect information necessary for configuring the aggregation tree (T2) from each network switch device (10: 10-1 to 10-12). It can be obtained. Information required for configuring the aggregation tree (T2) may include information about the remaining memory capacity of each network switch device (10: 10-1 to 10-12). Information about the remaining memory capacity is, for example, the first network switch device 10-1, the second network switch device 10-2, the eighth network switch device 10-8, and the ninth network switch device. (10-9), the tenth network switch device (10-10) and the twelfth network switch device (10-12) have a remaining capacity of three quarters, and the third network switch device (10-3) has a remaining capacity of three quarters. There is no used capacity, the fourth network switch device (10-4) and the eleventh network switch device (10-11) have half of the remaining capacity, and the fifth network switch device (10-5) and the sixth network switch The device 10-6 and the seventh network switch device 10-7 may include information that they have a remaining capacity of one quarter. Additionally, information required to construct the aggregate tree (T1) may include the amount of data to be processed. For multi-tenancy processing, the amount of data may include the amount of data resulting from commands or queries of each application.

The tree constructor 112 may determine and create a tree (T2) for each data, command, or query using the collected information. In this case, the generated tree (T2) may include a Steiner Tree. That is, the tree configuration unit 112 uses at least one operation server device 90 and the in-network management device 100 as terminals, and the network switch device 10: 10-1, based on the Steiner tree approximation algorithm. to 10-12), each of which can be used as a Steiner vertex to construct an aggregate tree (T2). Meanwhile, when at least one query is input, the tree constructor 112 reflects the query, as shown in FIGS. 5 and 6, to create aggregate trees (T2-1, T2) corresponding to each of the at least one query. -2) can be created. For example, when two queries (a first query and a second query) are input, but both queries generate data at a ratio of 3 to 2, the aggregate tree (T2-1) for the first query is generated as shown in Figure 5. It can be. For example, the processing result of the first query is input to the first network switch device 10-1 and the second network switch device 10-2, respectively, and the third network switch device 10-3 and the third network switch device 10-3. 5 network switch device (10-5), 6th network switch device (10-6), 8th network switch device (10-8), 10th network switch device (10-10) and 12th network switch device (10) It is transmitted to the in-network management device 100 through -12). Meanwhile, the aggregate tree (T2-2) for the second query can be created as shown in FIG. 6. For example, the processing result of the second query is the first network switch device 10-1 and the third network switch. Device (10-3), fifth network switch device (10-5), sixth network switch device (10-6), eighth network switch device (10-8), tenth network switch device (10-10) and transmitted to the in-network management device 100 via the 12th network switch device 10-12, or the 11th network switch device 10-11, the 10th network switch device 10-10, and the 12th network. It is transmitted to the in-network management device 100 via the switch device 10-12.

The aggregate tree (T2-1) for the first query and the aggregate tree (T2-2) for the second query are generated without considering the resources consumed by the different aggregate trees (T2-2, T2-1). Therefore, when performing intensive processing on the first query and the second query using the corresponding aggregation trees (T2-1, T2-2) as is, as shown in FIG. 7, a specific network switch device (10-1) , 10-3, 10-5, 10-6, 10-8, 10-10, 10-12), data is excessively concentrated and the concentration load is high, while other network switch devices (10-4, 10-7, 10-9), data is not concentrated at all, and resource utilization is extremely low. In other words, network switch devices (10-1, 10-3, 10-5, 10-6, 10-8, 10-10, 10-12) with greater data usage in the merged aggregate tree (T2-3). , must be shared by a larger number of queries or aggregate more data from those devices (10-1, 10-3, 10-5, 10-6, 10-8, 10-10, 10-12) means.

The tree reconfiguration unit 113 relieves overload of specific devices (10-1, 10-3, 10-5, 10-6, 10-8, 10-10, and 10-12) due to data concentration and reorganizes idle devices. In order to utilize (10-4, 10-7, 10-9), first, based on the resources consumed in response to each query, the aggregate tree (T2-1, T2-2) for each query is reorganized to create a new An aggregate tree (T2-6 in Figure 10, T-7 in Figure 11) is created. Specifically, for example, referring to FIGS. 7 and 8, the tree reconstruction unit 113 constructs an aggregate tree (T2-1) for the first query and an aggregate tree (T2-2) for the second query. ) to obtain the merged aggregate tree (T2-3). Specifically, the tree reconfiguration unit 113 vertices shared by the two trees (T2-1 and T2-2), that is, the network switch devices (10-1, 10-3, 10-5, 10-6, 10- 8, 10-10, 10-12) can also be merged to obtain a merged aggregate tree (T2-3). In this case, it is also possible to perform merging on all vertices, that is, network switch devices 10-1 to 10-12. The merging of the network switch devices (10-1 to 10-12) is based on the data usage of each network switch device (10-1 to 10-12) in the aggregation tree (T2-1) for the first query and the second query. It includes adding up the data usage of each network switch device (10-1 to 10-12) of the aggregation tree (T2-2). If necessary, the tree reconfiguration unit 113 adds a weight to each amount of data used by the network switch devices (10-1 to 10-12) in each aggregate tree (T2-1, T2-2) to reorganize the two trees. Merge vertices shared by (T2-1, T2-2), i.e. network switch devices (10-1, 10-3, 10-5, 10-6, 10-8, 10-10, 10-12) It is also possible to do so. Next, as shown in FIG. 8, the tree reconfiguration unit 113 refers to the merge processing result and selects data among the network switch devices 10-4, 10-7, and 10-9 with relatively low data processing load. Network switch devices (10-4, 10-) corresponding to network switch devices (10-1, 10-3, 10-5, 10-6, 10-8, 10-10, 10-12) with high processing load. Search for 7, 10-9). For example, for the third network switch device 10-3, the fourth network switch device 10-4 is detected, and for the sixth network switch device 10-6, the seventh network switch device 10- 7) is detected, and for the tenth network switch device (10-10), the ninth network switch device (10-9) is detected. In this case, the network switch devices (10-4, 10-7, 10-9) corresponding to the overloaded network switch devices (10-3, 10-6, 10-10) are connected to the overloaded vertex (i.e., the network switch device (10-3, 10-6, 10-10)) and the same level peak (i.e., network switch device (10-4, 10-7, 10-9)), and the corresponding network switch device (10-4) , 10-7, 10-9), the connectivity of the overall tree (T2-3) must be maintained and/or each overloaded vertex (10-3, 10-6, 10- 10) The integrated load should be smaller than before migration. According to an embodiment, for all overloaded network switch devices (10-1, 10-3, 10-5, 10-6, 10-8, 10-10, 10-12), a corresponding network switch device may be detected. Alternatively, corresponding network switch devices 10-4, 10-7, and 10-9 may be detected for some of the overloaded network switch devices 10-3, 10-6, and 10-10. When the corresponding network switch device (10-4, 10-7, 10-9) is detected, migration of the load of the overloaded network switch device (10-3, 10-6, 10-10) is performed accordingly. . Migration is performed for at least one network switch device (10-3, 10-6, 10-10) among the aggregate tree (T2-1) for the first query and the aggregate tree (T2-2) for the second query. It can be done. For example, as shown in FIG. 8, the load of the network switch devices 10-3, 10-6, and 10-10 with high data processing load in the aggregate tree T2-2 for the second query is, In response to this, the data processing load can be transferred to other network switch devices (10-4, 10-7, 10-9) with a low load. The above-described detection and migration may be performed sequentially, and depending on the embodiment, may be performed sequentially for each of the network switch devices 10-1 to 10-12. If there are no more network switch devices (10-3, 10-6, 10-1) capable of performing migration, the migration is terminated.

When migration is completed, the tree reconstruction unit 113 creates a new aggregate tree (T2-5) for the first query and a new aggregate tree (T2-6) for the second query, as shown in FIGS. 9 and 10. It can be obtained. At least one of the new aggregate trees (T2-5, T2-6) may be the same as or different from the existing aggregate trees (T2-1, T2-2). For example, for the first query, the new aggregate tree (T2-5) is the same as the existing aggregate tree (T2-1), and the new aggregate tree (T2-6) for the second query is the same as the existing aggregate tree (T2-1). It may be different from T2-2). Meanwhile, when merging the newly acquired aggregate trees (T2-5, T2-6) (depending on the situation, both trees (T2-5, T2-6) may have been migrated or only some of them may have been migrated. 11, the aggregate tree (T2-5) merged after migration has more enrichment on the path than the aggregate tree (T2-3) merged before migration. , 10-5, 10-8, 10-2) are less, and it can be seen that resource consumption is appropriately distributed.

The distribution processor 114 processes data corresponding to the aggregate trees (T2-5, T2-6) (e.g., a processing path for data acquired through reconstruction of the aggregate trees (T2-5, T2-6), etc. ) can be distributed to operation server devices (90: 90-1 to 90-k) or network switch devices (10: 10-1 to 10-12) to process data. Accordingly, the operation server device (90: 90-1 to 90-k) or the network switch device (10: 10-1 to 10-12) stores data based on the optimized aggregate tree (T2-5, T2-6). Delivery and aggregation can be performed.

The processor 110 includes a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller unit (MCU), an application processor (AP), and an electronic control unit. It may include an Electronic Controlling Unit (ECU), a Baseboard Management Controller (BMC), a Micro Processor (Micom), and/or at least one electronic device capable of performing various calculations and control processing. You can.

In addition, the above-described in-network management device 100 may be implemented using a device specifically designed to perform the above-described tree (T1 to T2-7), or may be implemented using an information processing device alone or in combination. It can also be implemented by using Information processing devices may include, but are not limited to, server hardware devices, desktop computers, laptop computers, or smart phones.

FIG. 12 is a block diagram of an embodiment of a network switch device, and FIGS. 13 and 14 are first and second diagrams for explaining an example of an integrated operation of a network switch device.

The network switch device 10 receives data from an operation server device (90: 90-1 to 90-k) or another network switch device (10: at least one of 10-1 to 10-12), and stores data in the received data. The integrated processing may be performed, and the processing result may be transmitted to the in-network management device 100 or another network switch device (10: at least one of 10-1 to 10-12).

According to one embodiment, the network switch device 10 may include a storage unit 12 and a processor 20 as shown in FIG. 12, and, if necessary, a communication unit 11 and a user interface 15. It may contain at least one more.

The storage unit 12 includes at least one of a main memory and an auxiliary memory, and data received from another network switch device (10: at least one of 10-1 to 10-12) is processed by the processor 20. Generated data or data acquired by the processor 20 may be temporarily or non-temporarily stored. According to one embodiment, the storage unit 12 may store the hash table 50. The hash table 50 (51, 52, 53) may be constructed by storing the key of the received data 30 and the value corresponding to the key, as shown in FIGS. 13 and 14. . Specifically, the hash table 50 (51, 52, 53) can be obtained by storing key and value pairs based on a hash index generated by hashing the key of the received data. The key may include a specific word, and the value may include, but is not limited to, the number of times a specific word occurs in the text. Additionally, the hash table 50 (51, 52, 53) may include a count result (counter) indicating the number of times data containing a specific key has been received.

The communication unit 11 communicates with other network switch devices (10: at least one of 10-1 to 10-12), operation server devices (90: 90-1 to 90-k), and in-network devices through a wired or wireless communication network. Performs communication with at least one of the management devices 100 and receives data from another network switch device (10: at least one of 10-1 to 10-12) or an operation server device (90: 90-1 to 90-k). Alternatively, information about another network switch device (10: at least one of 10-1 to 10-12) that will deliver the aggregate processing result may be received from the in-network management device 100.

The user interface 15 receives data, commands, or instructions related to the operation of the network switch device 10 directly from the user, etc., or displays data obtained according to the results of the operation of the network switch device 10 to the user, etc. Alternatively, it can be provided auditorily.

The processor 20 may be implemented based on one or two or more central processing units, graphics processing units, microcontroller units, or application processors, and as shown in FIG. 12, logically or physically distinguishable slots. It may include a determination unit 21, a key-value storage processing unit 22, a key-value comparison unit 23, a coefficient processing unit 24, a counting result comparison unit 25, and a transmission processing unit 26.

13 and 14, the slot determination unit 21 determines the hash indexes (h1(A), h1(B), h1(I), h1(K), and h2(J) for the received data 30. ), etc.) can be obtained, and it can be determined whether the slot connected to the corresponding hash index is empty or filled as shown in FIGS. 13 and 14. That is, the slot determination unit 21 can check whether the slot for storing the key-value pair 30 of the received data in the hash table 50 is empty. If the slot to be stored is empty, the slot determination unit 21 transmits information about this to the key-value storage processing unit 23. Conversely, if the slot to be stored is not empty, the slot is not empty to the key-value comparison unit 23. It conveys information that it does not exist.

The key-value storage processing unit 22 may store the key-value pair 30 in the hash table 50 (51, 52) of the storage unit 12. For example, if the slot is empty according to the determination result of the slot determination unit 21, the key-value storage processing unit 22 may record the key-value pair 30 in the corresponding slot. In addition, if the slot is not empty, the key-value storage processing unit 22 adds the value of the new data and the previously stored value according to the comparison result of the key-value comparison unit 23 and stores it in the corresponding slot, or Depending on the processing results of the key-value comparison unit 23 and the counting result comparison unit 25, the existing stored key and value of the corresponding slot may be removed and replaced with the key and value of new data.

If the slot for storing the key-value pair 30 of the received data in the hash table 50 is not empty, the key-value comparison unit 23, as shown in FIG. 13, uses the existing hash table 51 ) can be compared with the key of the key-value pair 51-1 stored in ) and the key of the key-value pair 30 of the newly received data. If both keys are the same (for example, both are B), the comparison result is transmitted to the first key-value storage processing unit 22 and the coefficient processing unit 24, and the key-value storage processing unit 22 stores the corresponding key. The values are added and the summed result is recorded in the slot corresponding to the key (51-1, that is, the slot in which the same key as the key of the key-value pair 30 of the newly received data was stored) and updated. The slot (51-1a) can be obtained. For example, the key-value storage processing unit 22 obtains 4 by adding the value 1 of the received data 30 and the value 3 of the previously stored data 51-1, and uses the obtained 4 to The update result (51-1a) of the corresponding slot (51-1) can be obtained.

The coefficient processing unit 24 receives the comparison result of the key-value comparison unit 23, and if the key of the key-value (51-1) previously stored in the hash table 51 and the key of the newly received data -If the keys of the value pair 30 are the same, the value of the count result (count) of the corresponding slot 51-2 is increased by 1. For example, if the coefficient result of the existing slot (51-1) was 1, the coefficient result of the corresponding slot (51-2) can be updated to 2 by adding 1 to the coefficient result (i.e., 2). there is.

On the other hand, there is no empty slot in the hash table 50 to store the key-value pair 30 of the received data, but if both keys are different from each other as shown in FIG. 14 (for example, the received data In the key-value pair 30, the key is B, but only the different keys of A or K are stored in the stored hash table 52), the key-value storage processing unit 22 stores the key of the received data- Only one of the value pair 30 and the key-value previously stored in the corresponding slot 52-1 of the hash table 52 can be stored in the corresponding slot 52-1 of the hash table 52. According to one embodiment, the data stored in the corresponding slot 52-1 includes the key-value pair 30 of the received data and the key previously stored in the corresponding slot 52-1 of the hash table 52- Among the values, the coefficient result may include smaller data. In this case, since the coefficient result of the key-value pair 30 of the newly input data is always 1, the corresponding slot 52-1 is replaced by the key-value pair 30 of the newly input data and recorded. It becomes (52-1a). Meanwhile, the key-value and coefficient result 52-1b previously recorded in the slot 52-1 may be extracted and then transmitted to the coefficient result comparison unit 25.

The counting result comparison unit 25 compares the counting result of the extracted key-value 52-1b with the counting result of the key-value of the other slot 52-2 in the hash table 52, and the comparison result is transmitted to the key-value storage processing unit 22, and the key-value storage processing unit 22 can cause data with a large coefficient result to be recorded in the corresponding slot 52-2 according to the comparison result. For example, as shown in FIG. 14, if the coefficient result (for example, 6) corresponding to the extracted key-value (52-1b) is greater than the coefficient result (5) of the predetermined slot 53-1, the key -The value storage processing unit 22 extracts data 52-2b (i.e., key-value and coefficient result) of a predetermined slot 52-2 and inserts the previously extracted key-value into the corresponding slot 52-2 ( An updated slot 52-2a can be obtained by recording 52-1b). The coefficient result comparison unit 25 sequentially compares the coefficient result (for example, 5) of the raw data 52-2b to the hash table 52 for the raw data 52-2b of the predetermined slot 53-1. The key-value coefficient result of another slot 52-3 within the key-value is again compared, and the key-value storage processing unit 22 may repeatedly update or maintain the corresponding slot 52-3 according to the comparison result. .

The final key-value pair (60) finally extracted and obtained through the above-described process has low redundancy. These key-value pairs are transferred from the coefficient result comparison unit 25 or the key-value storage processing unit 22 to the transfer processing unit 26.

The delivery processing unit 26 performs encapsulation processing on the final key-value pair 60 to generate a corresponding packet, and sends it to the next node (next network switching device 10: 10-1) through the communication unit 11. to at least one of 10-12) or the in-network management device 100, etc.). Accordingly, the final key-value pair 60 can be transmitted to the next node.

Hereinafter, the effects of the above-described in-network data collection system 1 will be described in detail with reference to FIGS. 15 and 16.

Figure 15 is a graph measuring the change in integration performance according to the number of generated data types. In Figure 15, the x-axis represents the number of key-value pairs, and the y-axis represents the reduction rate. The blue line shows the change in aggregation performance of the conventionally known data aggregation method within the network (DAIET: Data Aggregation Inside the Network), which divides memory equally to support multi-tenancy. However, this method processes and aggregates memory in the order in which it enters the switch without considering the redundancy of the data. In addition, the yellow line (MARINA-DH) is performed based on the network switch device 10 and the in-network management device 100 described above, but indicates the change in integration performance when each stage uses a different hash function, The green line (MARINA-SH) represents the change in integration performance when each stage is processed based on the above-described network switch device 10 and the in-network management device 100 using the same hash function.

Referring to FIG. 15, in the case of the conventional data aggregation method (DAIET) within a network and the method (MARINA-SH) in which each stage uses a different hash function, the amount of data (i.e. key-value pairs) is While there is data that is not aggregated even when the size is smaller than the available memory, the in-network aggregation system (1) using the same hash function shows that most data can be aggregated. This is because the above-described in-network aggregation system 1 first selects highly redundant data and stores it in memory, so the probability that data that has not been aggregated due to a hash collision has a low degree of redundancy is relatively high.

Figure 16 is a graph showing the change in aggregation performance according to the aggregation path setting method in a multi-tenancy environment in which three applications operate simultaneously. In FIG. 16, the x-axis sequentially represents a spanning tree used in the conventional data aggregation method within a network (DAIET), the aggregation operation of the network switch device described above, and the conventional data aggregation method within a network (DAIET). ) refers to a method of setting an aggregated path using (MARINA-S) and a method of setting an aggregated path as described above (MARINA-H). In addition, App1 is an application that generates data of a size smaller than the capacity of the equally divided memory partition, App2 is an application that generates data of the same size as the capacity of the equally divided memory partition, and App3 is an application that generates data of the same size as the capacity of the equally divided memory partition. An application that generates data exceeding its capacity. The y-axis represents the reduction rate for each method and each application.

Referring to FIG. 16, the conventional data aggregation method (DAIET) within a network has excellent aggregation performance when the size of generated data is small, but as the size of generated data increases, the aggregation performance decreases. This is the same even when the aggregate path is set using spanning tree (MARINA-S). On the other hand, when setting up an aggregation path (MARINA-H) like the in-network aggregation system (1) described above, excellent aggregation performance is generally achieved for all applications (app1, app2, app3) regardless of the amount of data generated. It is provided. Specifically, conventional methods using spanning trees have inferior aggregation performance because they aggregate a smaller amount of data with the same resources due to aggregation path settings and equal memory partitions, but the in-network aggregation system (1) described above has inferior aggregation performance. Aggregation paths (T1 to T2-7) are set considering the amount of data of each application and the resources of the network switch 10, and the size of the memory partition of each network switch 10 is adjusted adaptively, thereby improving the concentration performance. This is relatively excellent.

Hereinafter, an embodiment of the in-network data aggregation method will be described with reference to FIGS. 17 and 18.

Figure 17 is a flowchart of an embodiment of an in-network data aggregation method.

Referring to FIG. 17, first, at least one piece of information required to construct an aggregation tree is collected from each network switch device (300). Here, the information required for configuring the aggregation tree may include information about the remaining memory capacity of each network switch device or the amount of data to be processed. The amount of data to be processed may include the total amount of data generated from each command or query in the case of multi-tenancy processing.

Based on the collected information, one or more aggregate trees are created for each command or query (302). In this case, generation of the aggregate tree may be performed based on the Steiner tree generation algorithm.

The one or more aggregate trees created are merged (304). Merging the aggregation trees may include summing the data usage of each network switch device in the aggregation tree for the first query and the data usage of each network switch device in the aggregation tree for the second query.

According to the merge result, if an overloaded network switch device exists among the aggregated loads of each network switch device for all commands or queries, load migration is performed accordingly (306). In detail, a network switch device corresponding to a network switch device with relatively high resource consumption is selected from among network switch devices with no or low resource consumption, and the load on the network switch device with high resource consumption is selected for one or two or more commands or queries. is transferred to a network switch device with low resource consumption. For example, as shown in FIG. 8, instead of the network switch device in the aggregation tree for the second query, another corresponding network switch device performs the processing of the corresponding network switch device. Here, the network switch device corresponding to the network switch device with high resource consumption is at the same level as the network switch device with high aggregation, overall connectivity is maintained even after load migration to the network switch device, and/or the network switch device has high resource consumption. It may include a network switch device that allows the aggregate load after migration of the network switch device to be smaller than before migration.

Accordingly, a reconstructed aggregate tree corresponding to at least one command or query among all commands or queries is obtained (308). In this case, the aggregate tree for every command or query is not reconstructed. For example, the aggregate tree for the first query may maintain its existing form, and the aggregate tree for the second query may be rebuilt and modified in a new form.

The reconstructed aggregate tree can be distributed to an operation server device or network switch device to process data (310). Accordingly, the operation server device or network switch device can transmit and aggregate data based on the optimized aggregation tree. The collection of information and distribution of data (300 to 310) described above may be performed by an in-network management device, depending on the embodiment.

The network switch device can receive data according to the delivered aggregation tree, perform aggregation processing, and transmit the aggregation processing result to another network switch device or in-network management device (312).

Figure 18 is a flowchart of an embodiment of an integrated operation process of a network switch device.

Referring to FIG. 18, first, the network switch device can receive a key-value pair from another device (for example, an operation server device or another network switch device) (400).

It may be determined whether at least one slot of the hash table associated with the hash index for sequentially received data (key-value pairs) is empty (402).

If the slot is empty because no data is stored in the slot (example in 402), the received key-value pair can be stored in the slot (404).

Conversely, if there is data previously stored in the slot and the slot is not empty (No in 402), a comparison between the key of the received data and the key previously stored in the slot may be performed (406, 408).

If the key of the received data and the key previously stored in the slot are the same (example in 408), the values for the corresponding keys may be added (410). For example, if the keys of both the received data and the stored data are B as shown in FIG. 14, the value 1 of the received data and the value 3 of the previously stored data are added, and the sum result of 4 is stored in the corresponding slot. In addition, the coefficient result for the corresponding key is also updated by increasing it by 1.

Conversely, if the key of the received data and the key previously stored in the slot are not the same (No in 408), the key previously stored in the slot is extracted separately, and a new key-value is stored in the slot (412). In this case, the coefficient result corresponding to the new key-value may also be stored in the corresponding slot, and the coefficient result corresponding to the new key-value may be 1.

Next, another slot of the hash table is selected (414), and the coefficient result corresponding to the key-value of the other selected slot is compared with the coefficient result corresponding to the key-value extracted from the existing slot (416).

If the coefficient result corresponding to the key-value extracted from the existing slot is greater than the coefficient result corresponding to the key-value of another slot (example in 418), another slot is selected (420), and another slot The coefficient result corresponding to the key-value is compared with the coefficient result corresponding to the key-value extracted from the existing slot (416).

Conversely, if the coefficient result corresponding to the key-value extracted from the existing slot is smaller than the coefficient result corresponding to the key-value of another slot (example 418), the key-value extracted from the existing slot is stored in the other slot. Other slots are updated by recording, and data (key-value pairs and coefficient results) of other slots are extracted separately (422).

The extracted coefficient result corresponding to the key-value of another slot is compared with the coefficient result corresponding to the key-value of another slot (414, 416), and is recorded in another slot according to the comparison result of the coefficient result to another slot. Can be used to update slots (examples 418, 422, 424). This process can be repeated until the key-value pair is finally obtained (424).

When the above-described process is completed (No in 424), the finally obtained key-value pair is packetized and can be transmitted to the next network switching device or in-network management device (426).

The above-described in-network data aggregation method may be implemented in the form of a program that can be driven by a computer device. A program may include instructions, libraries, data files, and/or data structures, etc., singly or in combination, and may be designed and produced using machine code or high-level language code. The program may be specially designed to implement the above-described in-network data collection method, or may be implemented using various functions or definitions known and available to those skilled in the art in the computer software field. In addition, here, the computer device may be implemented by including a processor or memory that enables the function of the program, and may further include a communication device if necessary. A program for implementing the above-described in-network data aggregation method may be recorded on a recording medium readable by a device such as a computer. Recording media that can be read by a computer include, for example, semiconductor storage media such as ROM, RAM, SD cards, or flash memory (eg, solid state drives (SSD), etc.), or magnetic disk storage such as hard disks or floppy disks. At least one medium capable of temporarily or non-temporarily storing one or more programs that are executed upon call from a device such as a computer, such as an optical recording medium such as a compact disk or DVD, or a magneto-optical recording medium such as a floptical disk. It may include any type of physical storage medium.

Although the in-network management device, network switch, in-network data aggregation system, and in-network data aggregation method have been described above, the in-network management device, network switch, in-network data aggregation system, and/or in-network data aggregation method have been described above. The aggregation method is not limited to only the above-described embodiments. Various other devices, systems, or methods that can be implemented by those skilled in the art by modifying and modifying the above-described embodiments are also the in-network management device, network switch, in-network data collection system, or It may be an embodiment of an in-network data aggregation method. For example, the described method(s) may be performed in an order other than as described, and/or component(s) of the described system, structure, device, circuit, etc. may be combined, connected, or otherwise in a form other than as described. Even if combined or replaced or replaced by other components or equivalents, it can be an embodiment of the above-described in-network management device, network switch, in-network data aggregation system, and/or in-network data aggregation method.

1: In-network data aggregation system 10: Network switch device
20: Processor of network switch device 21: Slot determination unit
22: key-value storage unit 23: key-value comparison unit
24: counting processing unit 25: counter comparison unit
26: Delivery processing unit 30: Key-value data
90: operation server device 100: in-network management device
111: Information collection unit 112: Tree configuration unit
113: tree reconstruction unit 114: distribution processing unit

Claims (21)

a communication unit that receives information about at least one network switch device; and
Generate a plurality of aggregate trees corresponding to a plurality of queries based on information about the network switch device, merge the plurality of aggregate trees, and aggregate at least one of the plurality of aggregate trees based on the merged aggregate tree. a processor that generates a reconstructed aggregate tree for the tree;
The at least one network switch device receives a key-value pair from an operation server device, stores the key-value pair in the slot if the slot of the hash table is empty, and stores the key-value pair in the slot if the slot of the hash table is empty. Alternatively, the key of the key-value pair previously stored in the slot is compared with the received key, and if the key previously stored in the slot and the received key are the same, the value previously stored in the slot and the received value are added together. , updating the slot using the summation result,
In-network management device. According to paragraph 1,
The processor detects a network switch device with a low data processing load that corresponds to a network switch device with a high data processing load within the merged aggregation tree, and selects a network switch device with a high data processing load from the at least one aggregation tree. An in-network management device that generates a reconstructed aggregation tree for the at least one aggregation tree by migrating to a network switch device with a low data processing load. According to paragraph 2,
A network switch device that corresponds to the network switch device with a high data processing load and has a low data processing load,
An in-network management device comprising a network switch device whose data processing load is at the same level as a high network switch device, connectivity of the aggregation tree is maintained after migration, and the aggregation load is made smaller than before migration. According to paragraph 1,
The communication unit transmits data corresponding to the aggregation tree to at least one of the at least one network switch device and an operation server device that transmits data to the at least one network switch device. delete According to paragraph 1,
The at least one network switch device, if the key pre-stored in the slot and the received key are different, records the received key-value pair in the slot, and stores a stored coefficient corresponding to the key pre-stored in the slot. The result is compared with the coefficient result corresponding to the key pre-stored in another slot, and if the stored coefficient result corresponding to the key pre-stored in the slot is smaller than the coefficient result corresponding to the key pre-stored in the other slot, the coefficient result previously stored in the slot is An in-network management device that stores key-values in the different slots. a communication unit that receives a key-value pair from an operation server device or another network switch device;
Determine whether the slot of the hash table is empty, if the slot of the hash table is empty, store the received key-value pair in the slot, and if the slot of the hash table is not empty, the key previously stored in the slot - Compares the key of the value pair with the received key and updates the slot according to the comparison result,
If the key pre-stored in the slot and the received key are the same according to the comparison, the processor adds the value pre-stored in the slot and the received value, and updates the slot using the sum result.
Network switch device. delete In clause 7,
If the key pre-stored in the slot is different from the received key according to the comparison, the processor extracts the key pre-stored in the slot and replaces the received key-value pair with the key pre-stored in the slot. A network switch device that writes to the slot. According to clause 9,
The processor compares a stored coefficient result corresponding to a key pre-stored in the slot with a coefficient result corresponding to a key pre-stored in another slot, and the stored coefficient result corresponding to the key pre-stored in the slot is stored in the other slot. If it is smaller than the coefficient result corresponding to the stored key, the network switch device extracts the key previously stored in the other slot, replaces the key-value previously stored in the slot with the key previously stored in the other slot, and stores the key in the other slot. . Receiving information about at least one network switch device;
generating a plurality of aggregate trees corresponding to a plurality of queries based on information about the network switch device;
merging the plurality of aggregate trees; and
Generating a reconstructed aggregate tree for at least one aggregate tree among the plurality of aggregate trees based on the merged aggregate tree,
The at least one network switch device receiving a key-value pair from an operation server device or another network switch device;
determining whether a slot in the hash table is empty;
If the slot of the hash table is empty, storing the received key-value pair in the slot;
If the slot of the hash table is not empty, comparing the key of the key-value pair previously stored in the slot with the received key; and
Including, updating the slot according to the comparison result.
In-network data aggregation method. According to clause 11,
The step of generating a reconstructed aggregate tree for at least one aggregate tree among the plurality of aggregate trees based on the merged aggregate tree,
detecting a network switch device with a low data processing load and corresponding to a network switch device with a high data processing load within the merged aggregation tree;
generating a reconstructed aggregation tree for the at least one aggregation tree by migrating the at least one aggregation tree from a network switch device with a high data processing load to a network switch device with a low data processing load; In-network data aggregation method including. According to clause 12,
A network switch device that corresponds to the network switch device with a high data processing load and has a low data processing load,
An in-network data aggregation method comprising a network switch device whose data processing load is at the same level as a high network switch device, connectivity of the aggregation tree is maintained after migration, and the aggregation load is made smaller than before migration. According to clause 11,
In-network data aggregation method further comprising: transmitting data corresponding to the aggregation tree to at least one of the at least one network switch device and an operation server device that transmits data to the at least one network switch device. delete According to clause 11,
The step of updating the slot according to the comparison result is,
If the key pre-stored in the slot and the received key are the same according to the comparison, adding up the value pre-stored in the slot and the received value, and updating the slot using the sum result; -Network data aggregation method. According to clause 16,
The step of updating the slot according to the comparison result is,
If the key pre-stored in the slot is different from the received key according to the comparison, the key pre-stored in the slot is extracted, and the received key-value pair is replaced with the key pre-stored in the slot to be stored in the slot. An in-network data aggregation method further comprising: recording. According to clause 17,
Comparing a stored coefficient result corresponding to a key pre-stored in the slot with a coefficient result corresponding to a key pre-stored in another slot; and
If the stored coefficient result corresponding to the key pre-stored in the slot is smaller than the coefficient result corresponding to the key pre-stored in the other slot, the key pre-stored in the other slot is extracted, and the key-value pre-stored in the slot is used in the other slot. An in-network data aggregation method further comprising: replacing a key previously stored in a slot and storing it in the other slot. According to clause 11,
An in-network data aggregation method further comprising extracting and obtaining a final key-value pair, and transmitting the extracted and obtained key-value pair to another network switch device or an in-network management device. At least one network switch device receiving a key-value pair from an operation server device or another network switch device;
determining whether a slot in the hash table is empty;
If the slot of the hash table is empty, storing the received key-value pair in the slot;
If the slot of the hash table is not empty, comparing the key of the key-value pair previously stored in the slot with the received key;
If the key pre-stored in the slot is different from the received key according to the comparison, the key pre-stored in the slot is extracted, and the received key-value pair is replaced with the key pre-stored in the slot to be stored in the slot. recording step;
Comparing a stored coefficient result corresponding to a key pre-stored in the slot with a coefficient result corresponding to a key pre-stored in another slot; and
If the stored coefficient result corresponding to the key pre-stored in the slot is smaller than the coefficient result corresponding to the key pre-stored in the other slot, the key pre-stored in the other slot is extracted, and the key-value pre-stored in the slot is used in the other slot. In-network data aggregation method including; replacing the key previously stored in the slot and storing it in the other slot. delete

Images