WO2020078381A1 - Data aggregation method, device, equipment, storage medium and system - Google Patents

Abstract

The present application belongs to the technical field of big data, and disclosed thereby are a data aggregation method, device, equipment, storage medium and system. The method is applied to control equipment of a first data aggregation system, the method comprising: obtaining configuration information of a plurality of virtual nodes; according to a received data aggregation instruction, determining a starting virtual node and a destination virtual node, and according to the configuration information of the plurality of virtual nodes and node equipment corresponding to each virtual node, determining a transmission channel from the starting node equipment to the destination node equipment, distributing the transmission channel to each node equipment of the transmission channel, carrying out data transmission by means of each node equipment on the transmission channel, and aggregating data of the starting node equipment into the destination node equipment. Thus, configuration information can be customized by the means of inputted configuration instructions. The configuration information can be updated according to service requirements, so as to change the node equipment in the data aggregation system, expand the service capabilities of the data aggregation system, and improve flexibility.

Description

Data aggregation method, device, equipment, storage medium and system

This application requires the priority of the Chinese patent application with the application number 201811205313.5 and the invention titled "Data Convergence Method, Device, Equipment, Storage Media, and System" filed on October 16, 2018, the entire content of which is incorporated herein by reference Applying.

Technical field

This application relates to the field of big data technology, in particular to a data aggregation method, device, equipment, storage medium, and system.

Background technique

With the continuous development of big data technology and the continuous expansion of network scale, the collection and aggregation of data has become a matter of great concern. In the actual network, the distributed deployment of each server results in the data on each server being isolated from each other, and unified data management cannot be performed. Therefore, the data on different servers needs to be aggregated.

Related technologies provide a method for data aggregation based on the Apache Flume (log collection) framework. In this method, the data aggregation system includes at least one collection server and at least one aggregation server. Each collection server is associated with one or more Aggregation server connection. And install the client of the Apache Flume framework on each collection server, and install the aggregation end of the Apache Flume framework on each aggregation server, so that the collection server and the aggregation server can interact based on the Apache Flume framework. In addition, multiple terminal devices are respectively connected to one or more collection servers, and each terminal device can upload the acquired data to the connected collection server, and the collection server classifies the collected data and collects the collected data. The data is sent to the aggregation server, and the aggregation server can store the collected data, realizing the function of aggregating the data on different terminal devices together.

However, because the collection server and the aggregation server in the data aggregation system are configured, they cannot be changed and the expansion performance is poor. Even if there are other servers with data collection functions, they cannot be added to the data aggregation system.

Summary of the invention

The embodiments of the present application provide a data aggregation method, device, equipment, storage medium, and system, which can solve the problem that other servers with data collection functions cannot be added to the data aggregation system. The technical solution is as follows:

In a first aspect, a data aggregation method is provided, which is applied to a control device of a first data aggregation system. The first data aggregation system includes multiple node devices and the control device. Multiple node devices are connected. The method includes:

Acquiring configuration information of a plurality of virtual nodes, the configuration information is determined according to a configuration instruction input on a configuration device connected to the control device, the configuration information includes at least a device identification of an input device and a device identification of an output device The multiple virtual nodes correspond to the multiple node devices in one-to-one correspondence;

When receiving a data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, based on the data aggregation instruction, the starting virtual node corresponding to the starting node device and the destination node device are determined The corresponding destination virtual node;

According to the configuration information of the plurality of virtual nodes and the node device corresponding to each virtual node, determine a transmission channel from the starting node device to the destination node device, the transmission channel is in order by at least two node devices Connected

Deliver the transmission channel to each node device on the transmission channel, and each node device on the transmission channel performs data transmission to aggregate the data of the starting node device to the destination node device in.

In a second aspect, a data aggregation method is provided, which is applied to any node device in a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device. To connect with the plurality of node devices, the method includes:

Receiving a transmission channel delivered by the control device, the transmission channel is determined by the control device according to configuration information of multiple virtual nodes and a node device corresponding to each virtual node, and the configuration information is based on The configuration instruction input on the connected configuration device determines that it includes at least the device identification of the input device and the device identification of the output device, the transmission channel is formed by connecting at least two node devices in sequence, and includes the local node device;

When receiving the data sent by the last node device of the local node device on the transmission channel, send the data to the next node device of the local node device on the transmission channel, to Aggregate the data to the destination node device on the transmission channel.

In a third aspect, a data aggregation device is provided, which is applied to a control device of a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device. Multiple node devices are connected, and the apparatus includes:

An obtaining module, configured to obtain configuration information of a plurality of virtual nodes, the configuration information is determined according to a configuration instruction input on a configuration device connected to the control device, and the configuration information includes at least a device identifier and an output device of the input device Device identification, the multiple virtual nodes are in one-to-one correspondence with the multiple node devices;

The first determining module is configured to, when receiving a data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, determine the starting virtual corresponding to the starting node device based on the data aggregation instruction A node and a destination virtual node corresponding to the destination node device;

The second determining module is configured to determine a transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, the transmission channel is At least two node devices are connected in order;

The first sending module is used to deliver the transmission channel to each node device on the transmission channel, and each node device on the transmission channel performs data transmission to transfer the data of the starting node device Converged into the destination node device.

According to a fourth aspect, a data aggregation device is provided, which is applied to any node device in a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device. The control devices are respectively Connected to the multiple node devices, the apparatus includes:

The receiving module is configured to receive a transmission channel delivered by the control device, and the transmission channel is determined by the control device according to configuration information of multiple virtual nodes and a node device corresponding to each virtual node, and the configuration information is based on The configuration instruction input on the configuration device connected to the control device is determined to include at least the device identification of the input device and the device identification of the output device, the transmission channel is formed by connecting at least two node devices in sequence, and includes the local end Node equipment

A first sending module, configured to send the data to the local node device in the transmission channel when receiving the data sent by the previous node device of the local node device in the transmission channel The next node device to aggregate the data into the destination node device on the transmission channel.

According to a fifth aspect, a control device is provided, which is applied in a first data aggregation system, where the first data aggregation system includes multiple node devices and the control device, and the control device and the multiple node devices are respectively Connected, the control device includes a processor and a memory, and the memory stores at least one instruction, the at least one instruction is loaded and executed by the processor to implement any one of the data aggregation described in the first aspect above method.

According to a sixth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the data aggregation described in the first aspect method.

According to a seventh aspect, a node device is provided, which is applied to a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device, and the control device and the plurality of node devices are respectively connection;

The local node device includes a processor and a memory, where at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement any one of the data aggregation methods described in the second aspect above.

According to an eighth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the data aggregation described in the second aspect method.

In a ninth aspect, a data aggregation system is provided. The data aggregation system includes a plurality of node devices and a control device, and the control devices are respectively connected to the plurality of node devices;

The control device is the control device as described in the fifth aspect above;

Each node device is the node device described in the seventh aspect above.

The data aggregation method, apparatus, device, storage medium, and system provided in the embodiments of the present application obtain configuration information of multiple virtual nodes by acquiring configuration information according to the configuration instructions entered on the configuration device, and thus can receive data aggregation When instructing, determine the starting virtual node and the destination virtual node according to the data aggregation instruction, and determine the transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node , A transmission channel is delivered to each node device of the transmission channel, and each node device on the transmission channel performs data transmission, and aggregates the data of the starting node device to the destination node device. The configuration information can be customized through the input configuration instruction, so that the control device determines the transmission channel according to the customized configuration information, and the data aggregation is performed through the transmission channel, so the configuration information can be updated according to the business needs, thereby changing the data aggregation system The node equipment in the system expands the business capability of the data aggregation system and improves flexibility.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For a person of ordinary skill in the art, without paying any creative work, other drawings can also be obtained based on these drawings.

1 is a schematic structural diagram of a data aggregation system provided by an embodiment of the present application;

2 is a flowchart of a data aggregation method provided by an embodiment of the present application;

3 is a flowchart of another data aggregation method provided by an embodiment of the present application;

4 is a flowchart of another data aggregation method provided by an embodiment of the present application;

5 is a schematic diagram of a node device performing monitoring tasks provided by an embodiment of the present application;

6 is a schematic diagram of yet another node device buffer area provided by an embodiment of the present application;

7 is a schematic diagram of a data aggregation scenario provided by an embodiment of the present application;

8 is a schematic diagram of another data aggregation scenario provided by an embodiment of the present application;

9 is a schematic diagram of another data aggregation scenario provided by an embodiment of the present application;

10 is a schematic structural diagram of a data aggregation device provided by an embodiment of the present application;

11 is a schematic structural diagram of another data aggregation device provided by an embodiment of the present application;

12 is a schematic structural diagram of a server provided by an embodiment of the present application.

detailed description

To make the objectives, technical solutions, and advantages of the present application clearer, the following describes the embodiments of the present application in further detail with reference to the accompanying drawings.

For ease of understanding, before explaining the embodiments of the present application in detail, the implementation environment of the present application is first introduced.

1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Referring to FIG. 1, the implementation environment includes a configuration device 110 and a first data aggregation system 120. The first data aggregation system 120 includes multiple node devices 121 and The control device 122 and the configuration device 110 are connected to the control device 122, and the control device 122 is connected to a plurality of node devices 121, respectively.

Among them, any two node devices in the plurality of node devices 121 can be connected, so that a network topology structure is formed in the plurality of node devices 121, and based on the network topology structure, data can be transmitted between any two node devices. It should be noted that when the above network topology is formed, every two node devices in the plurality of node devices 121 may be connected. Of course, only some node devices may be connected.

At least one node device of the plurality of node devices 121 may be a collection device, and the collection device has a data collection function for collecting data from the terminal device. In addition, at least one node device is an aggregation device. The aggregation device has a data aggregation function, and the data collected by the collection device can be aggregated and stored in the storage device. In addition, at least one node device is an intermediate device. The intermediate device has a data forwarding function. During the process of sending data from the collection device to the aggregation device, the intermediate device can play the role of forwarding data.

The configuration device 110 is used to generate configuration information of a plurality of virtual nodes according to the received configuration instruction, and send the generated configuration information to the control device 122, and the control device 122 is used to obtain the configuration information of the plurality of virtual nodes from the configuration device 110 . The multiple virtual nodes correspond to the multiple node devices 121 in one-to-one relationship. The virtual nodes are the nodes mapped by the node devices in the configuration information. The configuration information of the virtual nodes is actually information used to describe the node devices.

Taking the starting node device to send data to the destination node device and aggregating the data to the destination node device as an example, the starting node device is used to send a data aggregation instruction to the control device 122, and the data aggregation instruction carries the device identification of the starting node device And the device identification of the destination node device.

The control device 122 is also used to determine the starting virtual node corresponding to the starting node device and the destination virtual node corresponding to the destination node device when receiving the data aggregation instruction, and based on the configuration information of the plurality of virtual nodes and each virtual node The corresponding node device determines a transmission channel from the starting node device to the destination node device, and the transmission channel is formed by connecting at least two node devices in sequence. After that, the control device 122 delivers the transmission channel to each node device on the transmission channel.

The starting node device is used to send the data in the local node device to the next node device of the local node device in the transmission channel when receiving the transmission channel; the destination node device is used to receive the transmission channel and receive When data sent to the previous node device of the local node device is stored, the data is stored.

If there are other node devices besides the starting node device and the destination node device in the node device on the transmission channel, the other node devices serve as intermediate node devices for receiving the transmission channel and receiving the local node device When the data sent by the last node device of the server is sent, the data is sent to the next node device in the transmission channel of the local node device to converge the data to the destination node device on the transmission channel.

In another embodiment, the first data aggregation system 120 further includes multiple storage devices, the storage device is used to store data during the data aggregation process, and the input device of the node device refers to the storage device previously connected to the node device , The output device of the node device refers to the storage device connected after the node device, then the two adjacent node devices in the transmission channel are connected through the storage device, that is to say, the output device of the previous node device and the latter node The input device of the device is the same storage device.

During the data aggregation process, each node device can output data to the connected storage device, store the data in the storage device, and send the data to the next node device by the storage device. Realize data storage to prevent data loss.

FIG. 2 is a flowchart of a data aggregation method provided by an embodiment of the present application, which is applied to the control device of the embodiment shown in FIG. 1. Referring to Figure 2, the method includes the following steps:

Step 201: Obtain configuration information of multiple virtual nodes. The configuration information is determined according to configuration instructions entered on the configuration device. The configuration information includes at least the device identification of the input device and the device identification of the output device. Multiple node devices correspond to each other.

Step 202: When receiving the data aggregation instruction including the device identification of the device identifier of the starting node device and the device identification of the destination node device, based on the data aggregation instruction, determine the starting virtual node and the destination node corresponding to the starting node device The target virtual node corresponding to the device.

Step 203: Determine the transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, the transmission channel is formed by connecting at least two node devices in order .

Step 204: Deliver the transmission channel to each node device on the transmission channel, and each node device on the transmission channel performs data transmission, and converges the data of the starting node device to the destination node device.

In summary, in the data aggregation method provided by the embodiment of the present application, the control device obtains the configuration information of multiple virtual nodes, and the configuration information is determined according to the configuration instruction input on the configuration device, so the data aggregation instruction can be received At the time, determine the starting virtual node and the destination virtual node according to the data aggregation instruction, and determine the transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, The transmission channel is delivered to each node device of the transmission channel, and each node device on the transmission channel performs data transmission, and aggregates the data of the starting node device to the destination node device. The configuration information can be customized through the input configuration instruction, so that the control device determines the transmission channel according to the customized configuration information, and the data aggregation is performed through the transmission channel, so the configuration information can be updated according to the business needs, thereby changing the data aggregation system The node equipment in the system expands the business capability of the data aggregation system and improves flexibility.

FIG. 3 is a flowchart of another data aggregation method provided by an embodiment of the present application, which is applied to any node device in the embodiment shown in FIG. 1. Referring to FIG. 3, the method includes the following steps:

Step 301: Receive a transmission channel delivered by the control device. The transmission channel is determined by the control device according to the configuration information of multiple virtual nodes and the node device corresponding to each virtual node. The configuration information includes at least the device identification and output device of the input device Device identification, the transmission channel is formed by connecting at least two node devices in sequence, and includes the local node device.

Step 302: When receiving the data sent by the previous node device of the local node device in the transmission channel, send the data to the next node device of the local node device in the transmission channel to aggregate the data to the transmission In the destination node device on the channel.

In summary, in the data aggregation method provided in the embodiments of the present application, any node device receives the transmission channel delivered by the control device, and when receiving the data sent by the previous node device in the transmission channel of the local node device, Send the data to the next node device in the transmission channel of the local node device to converge the data to the destination node device on the transmission channel. The configuration information can be customized through the input configuration instruction, so that the control device determines the transmission channel according to the customized configuration information and sends it to each node device in the transmission channel. Any node device can be based on the transmission channel Data aggregation is performed, so configuration information can be updated according to business requirements, thereby changing node devices in the data aggregation system, expanding the business capabilities of the data aggregation system, and increasing flexibility.

FIG. 4 is a flowchart of another data aggregation method provided by an embodiment of the present application, which is applied to the implementation environment of the embodiment shown in FIG. 1. Referring to Figure 4, the method includes the following steps:

Step 401: The configuration device generates system configuration information according to the received configuration instruction.

The configuration device may display an input interface, and a technician inputs a configuration instruction carrying configuration information in the input interface. When the configuration device receives the configuration instruction, the system configuration information is generated according to the configuration information in the configuration instruction.

The system configuration information includes configuration information of multiple virtual nodes, which are used to describe multiple virtual nodes. The multiple virtual nodes correspond to multiple node devices in the first data aggregation system, and the virtual nodes are nodes. The node to which the device is mapped in the configuration information, and the configuration information of the virtual node is actually used to describe the node device.

The configuration information of the virtual node includes at least the device identification of the node device corresponding to the virtual node, the device identification of at least one input device, and the device identification of at least one output device. The input device is a node device that inputs data to the node device corresponding to the virtual node. That is, the previous node device of the node device corresponding to the virtual node, and the output device refers to the node device that receives the data output by the node device corresponding to the virtual node, that is, the next node of the node device corresponding to the virtual node device. Therefore, the configuration information may at least represent at least one node device before or after the node device corresponding to the virtual node.

Then, according to the statistics of the configuration information of the multiple virtual nodes in the system configuration information, it can be determined which node devices are included in the first data aggregation system and which node devices are connected before or after each node device.

In a possible implementation, in the configuration information of the virtual node, the input device is a storage device that inputs data to the node device corresponding to the virtual node, that is, the storage device connected to the node device corresponding to the virtual node, and output The device refers to a storage device that receives data output by a node device corresponding to the virtual node, that is, a storage device connected after the node device corresponding to the virtual node. Therefore, the configuration information may at least represent at least one storage device before or after the node device corresponding to the virtual node.

Then, according to the statistics of the configuration information of multiple virtual nodes in the system configuration information, it can be determined which node devices included in the first data aggregation system and which storage devices are connected before or after each node device.

Among them, the device identifier of the input device and the device identifier of the output device in the configuration information are the from end and the to end of the virtual node. For example, the device identifier of the input device of the virtual node node_1 is kafka_1, and the device identifier of the output device is file_1. And each virtual node can have multiple from and to ends. For example, the device identifiers of the input devices in the virtual node node_2 are file_1 and file_2, and the device identifier of the output device is file_1. The other virtual nodes are similar to the aforementioned virtual nodes similar to node_1 and node_2, and will not be elaborated here.

In a possible implementation manner, the device identifier of the node device is used to determine a unique corresponding node device, and may be the network address, device name, and device number of the node device, such as node device 1, node device 2, and so on. The device identifier of the above storage device is used to determine the unique corresponding storage device, and may be the network address, device name, and device number of the storage device, such as kafka_1, file_1, ftp_1, and so on.

In a possible implementation, the configuration information of the virtual node may also include attribute information of the input device and the output device. The attribute information is determined by the type of data stored by the input device and the output device, and the specific content of the attribute information is passed by the technical Enter the configuration command to configure.

For example, if the data type allowed by the input device to be stored is kafka, then the attribute information may be information required by kafka, such as topic (data topic), group.id (queue address). If the data type allowed by the input device to be stored is file, the attribute information may include information required by file such as the directory location (localDir) where the file is stored. If the data type allowed by the input device to be stored is the FTP service, then the attribute information may include the host name (hostname), the port (port), the storage path (path), and the FTP service username ( username) and password (password), etc. Of course, the above data type may also be other data types, and the above attribute information may also include information of other attributes, which is not limited in this embodiment of the present application.

Among them, the configuration instructions that generate the system configuration information can be as follows:

In a possible implementation, the configuration information of the virtual node further includes a data processing chain, and the data processing chain is used to specify at least one data processing mode when the corresponding node device receives the data to be transmitted.

Wherein, the data processing chain includes at least one processing method, and the at least one processing method is arranged in order. When data processing is performed, the processing methods in the data processing chain are sequentially used for data processing in order.

For example, any node device can filter the received data, by configuring the PROCESS_FILTER process and the corresponding filtering conditions in the virtual node corresponding to the node device, so that the node will filter according to the configured processing method when receiving the data Actions.

Among them, the configuration device provides a special data processing interface. When the data processing mode currently configured by the virtual node cannot meet the data processing needs of the technician, the processing process including multiple data processing methods customized by the technician can be received through the interface Package, put the process package into the data processing library of the configuration device. When configuring the virtual node, the technician can enter the configuration instructions in the configuration device as required to configure the required virtual node to call the process The instruction of any processing method of the package, so that during the aggregation process, the node device corresponding to the virtual node can perform data processing on the aggregated data according to the specified processing method according to the needs of the technician, and realize the custom node device of the technician Data processing method, which can expand the data processing capacity of the node equipment, more in line with the needs of technical staff.

For example, the configuration instructions for configuring the processing mode in the data processing chain of a virtual node can be as follows:

It should be noted that the data processing methods configured in the above example are to filter the data, convert the data, and call a custom processing method (customize). Among them, filtering and conversion are the capabilities of the data processing method originally provided by the node device corresponding to the virtual node, and the custom processing method is configured by the technician through the data processing interface, that is, the technician expands the node device corresponding to the virtual node Treatment.

It should be noted that the data can be directly filtered according to the configuration instructions in the above example. To achieve the conversion of the data format, the technician needs to provide a data structure description file (schema.yml) of the data. The description file is used to describe the format of the data to be converted this time, and a description file of the converted data format is also required (XXX.xls).

For example, the current data structure description file for the data can be as follows:

Then, the mapping relationship of format conversion can be obtained according to any description file. For example, as shown in Table 1 below:

Table 1

old_col	old_type	new_col	new_type	convert_func
column1	STRING	column_1	STRING	$ {column1} + $ {column4}
column2	FLOAT64	column_2	STRING	‘TEST’ + $ {column2}

It should be noted that the above Table 1 is only an exemplary format conversion mapping relationship provided by the embodiment of the present application, and may also be other mapping relationships, which are not limited in the embodiment of the present application.

In a possible implementation manner, in the first data aggregation system, a new node device may be added, and the new node device is connected to any two original node devices in the first data aggregation system, and the Enter the configuration instruction on the device, add the configuration information of the virtual node corresponding to the new node device, and set the device identifiers of the input device and output device of the virtual node corresponding to the new node device to the device identifiers of the two node devices In order to add the virtual node corresponding to the new node device to the virtual topology network, so as to add a new node device in the first data aggregation system.

In another possible implementation manner, if there are too many idle node devices in the first data aggregation system, in order to avoid waste of resources, any idle node device may be removed. Specifically, the idle node device may be removed. Connection of other node devices, then remove the idle node device, and delete the configuration information of the virtual node corresponding to the idle node device by inputting a configuration instruction on the configuration device.

It should be noted that, in the embodiments of the present application, the node devices may be connected through a storage device, that is, the input device and the output device of each node device are storage devices.

In a possible implementation manner, in the first data aggregation system, a new node device may be added, and the new node device is connected to two storage devices that have not established a connection through the node device, and through the configuration device Enter the configuration instruction, add the configuration information of the virtual node corresponding to the new node device, and set the device identifiers of the input device and output device of the virtual node corresponding to the new node device to the device identifiers of the two storage devices, respectively, to The virtual node corresponding to the new node device is added to the virtual topology network, so that a new node device is added to the first data aggregation system.

In another possible implementation manner, if there are too many idle node devices in the first data aggregation system, in order to avoid waste of resources, any idle node device may be removed. Specifically, the idle node device may be The connection between the two storage devices of the device and the output device is removed, and then the node device is removed, and the configuration information of the virtual node corresponding to the idle node device is deleted by entering a configuration instruction on the configuration device.

It should be noted that in the embodiments of the present application, a technician can input the above configuration instructions on the configuration device to configure any desired virtual node, so that the node device included in the first data aggregation system can be expanded, Add new node equipment for data aggregation to achieve personalized configuration. It is also possible to delete unnecessary virtual nodes, remove idle node devices in the first data aggregation system, and avoid waste of resources.

Moreover, the system configuration information may be in the form of a configuration file. When the configuration device generates configuration information of multiple virtual nodes according to the input configuration instruction, the configuration information of the multiple virtual nodes is encapsulated to obtain the system configuration in the form of a configuration file The information can be obtained by parsing the system configuration information to obtain the configuration information of multiple virtual nodes.

Step 402: The configuration device sends system configuration information to the control device.

The configuration device is connected to the control device. The specific connection method may be that the configuration device and the control device are connected through a network or other connection methods, which is not limited in this embodiment of the present application.

The configuration device can directly send the system configuration information to the control device, or it can store the system configuration information and send the system configuration information to the control device when receiving the acquisition request sent by the control device.

Step 403: The control device receives system configuration information, analyzes the system configuration information, and obtains configuration information of multiple virtual nodes.

Step 404: The starting node device sends a data aggregation instruction to the control device, where the data aggregation instruction carries the device identifier of the starting node device and the device identifier of the destination node device.

The starting node device may be any node device in the first data aggregation system. Preferably, the starting node device may be a collection device in the first data aggregation system, which is specifically used to collect data and aggregate the data into the aggregation device.

When the starting node device needs to aggregate data into other node devices, it can send a data aggregation instruction to the control device. Since the control device needs to determine the starting point and the ending point of data aggregation, the transmission channel can be subsequently determined for data aggregation. Therefore, when the starting node device sends the data aggregation instruction, the device identification of the starting node device and the device identifier of the destination node device need to be carried in the data aggregation instruction and sent to the control device.

Step 405: When the control device receives the data aggregation instruction, based on the data aggregation instruction, the starting virtual node corresponding to the starting node device and the destination virtual node corresponding to the destination node device are determined.

When the control device receives the data aggregation instruction, according to the device identifier of the starting node device and the device identifier of the destination node device carried in the aggregation instruction, the virtual node device identifier is the same as the device identifier of the starting node device. The node is used as the starting virtual node, and the virtual node with the same node device ID as that of the destination node device is used as the destination virtual node.

Step 406: The control device determines the transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, and delivers to each node device on the transmission channel A transmission channel, which is formed by connecting at least two node devices in sequence.

In a possible implementation manner, after acquiring configuration information of multiple virtual nodes and before receiving a data aggregation instruction, for any two virtual nodes of the multiple virtual nodes, the device of the output device of one of the virtual nodes When the identifier is the same as the device identifier of the input device of another virtual node, connect any two virtual nodes to obtain a virtual topology structure, the virtual topology structure is composed of the multiple virtual node connections, and the virtual topology structure is used for description The connection relationship of the multiple node devices.

Since each virtual node can include the device identification of multiple input devices and the device identification of output devices, after the virtual nodes are connected in the above manner, the resulting virtual topology structure will be a structure in which multiple virtual nodes are connected to each other. The structure can be mapped to the connection relationship between multiple node devices.

Therefore, when the control device receives the data aggregation instruction, in the virtual topology, it looks for the virtual transmission channel from the starting virtual node to the destination virtual node; according to the node device corresponding to each virtual node on the virtual transmission channel, determine from The transmission channel from the starting node device to the destination node device.

Wherein, the transmission channel may be a channel in which the device identifiers of multiple node devices are arranged in order. The transmission channel may be multiple. Assuming that the starting virtual node is node_1 and the destination virtual node is node_4, then if node_1 in the virtual topology Connected to node_2, node_3 and node_4 respectively, node_2 is connected to node_3 and node_4, then the virtual transmission channel obtained by the control device may include node_1-> node_2-> node_4, node_1-> node_4; node_1-> node_2-> node_3-> node_4, node_1-> node_3-> node_4 etc.

Optionally, when delivering the transmission channel to each node device on the transmission channel, the control device may determine the device identification of the input device and the output device of each virtual node from the configuration information of multiple virtual nodes, The devices corresponding to the two device identifiers are the storage device between the node device corresponding to the virtual node and the previous node device, and the storage device connected to the next node device corresponding to the virtual node, so that the starting node can be determined The device reaches the transmission channel of the destination node device, and then the transmission channel is delivered to each node device in the transmission channel, and the transmission channel includes not only multiple node devices arranged in sequence, but also any two node devices Between storage devices, so that when each node device sends data to the next node device, it can first send the data to the corresponding storage device, and then the corresponding storage device to the next node device.

Optionally, the control device may deliver the transmission channel and the monitoring task to each node device on the transmission channel, and each node device on the transmission channel performs data transmission, and monitors the data transmission process at the local end. The control device sends the data transmission result.

Step 407: When the starting node device receives the transmission channel delivered by the control device, it sends the data of the starting node device to the next node device in the transmission channel of the starting node device.

There may be multiple transmission channels. When the control device delivers the transmission channel, when the starting node device receives multiple transmission channels, it can obtain the load of each transmission channel from the control device, and then according to the amount of data to be transmitted Determine the data volume of each transmission channel, distribute the data according to the data volume of each transmission channel, and send each distributed data to the storage device between the starting node device and the next node device in the corresponding transmission channel. Sent to the next node device of the starting node device in the transmission channel, so as to separately transmit through multiple transmission channels. In this way, the transmission pressure of each transmission channel can be reduced, thereby improving the efficiency of data aggregation.

It should be noted that the starting node device can also carry the data amount of the data to be transmitted in the data aggregation instruction and send it to the control device. The control device determines multiple transmission channels and each transmission channel in a manner similar to the above When the determined transmission channel is delivered to the starting node device, the data amount of each transmission channel is sent to the starting node device at the same time. The starting node device is directly allocated according to the data amount of each transmission channel Data, send the distributed data to the next node device of the starting node device in each transmission channel.

In addition, if the determined at least one transmission channel is overloaded, or the transmission capacity is limited to meet this data aggregation, you can customize it in the configuration information to directly increase the connection relationship between the two unconnected virtual nodes Or, you can add node devices and configure the configuration information of the corresponding virtual nodes to add new node devices to the data aggregation system to expand the transmission channel, so as to expand the capacity of the data aggregation system and enhance the efficiency of data aggregation.

Step 408: The other node device receives the transmission channel delivered by the control device, and when receiving the data sent by the previous node device of the local node device in the transmission channel, sends the data to the local node device in the transmission channel The next node device.

Among them, the other node devices are node devices other than the start node device and the destination node device among the node devices on the transmission channel.

It should be noted that each node device is provided with a buffer, and the buffer stores the data to be processed transmitted by the previous node device of the local node device in the form of a buffer queue. When the buffer queue processes the stored data in order, when the processing is completed, the data will be removed from the buffer queue, thereby ensuring the available space of the buffer queue.

And, as shown in FIG. 5, in any node device, if the storage space of the buffer queue is insufficient, a suspension notification message can be sent to the previous node device of the local node device in the transmission channel to remind the previous node device to suspend The end node device sends data. When the buffer queue has sufficient storage space, it can send a recovery notification message to the next node device to enable the previous node device to restart transmitting data to the local node device.

In a possible implementation, when any node device receives the monitoring task issued by the control device, it can monitor its own data transmission in real time, and generate data transmission every preset time according to the real-time monitored data transmission The result is sent to the control device. The control device can obtain the overall transmission status of each transmission channel according to each measurement report, so as to uniformly manage and control each transmission channel during the transmission process. The measurement report may be the above data transmission result, or may be obtained by processing multiple data transmission results of the transmission channel, for example, may include the average data transmission speed and average calculated from the multiple data transmission results of the transmission channel The amount of cached data, etc.

As shown in FIG. 6, for example, the virtual transmission channel is node_1-> node_2-> node_3, then when the node devices 1, node device 2, and node device 3 corresponding to these three virtual nodes perform data transmission, each node device can be timed Detect the remaining space of the current buffer area, the speed of data processing and the total number of processed data, etc., and then generate a data transmission result and send it to the control device.

When the control device receives the data transmission result uploaded by each node device of any transmission channel, it can calculate the average data transmission speed and average buffered data amount of the transmission channel according to the multiple data transmission results of the transmission channel. When the data transmission speed of the channel is higher than the preset speed value for a long time, or when the average buffered data volume of the transmission channel is higher than the preset data volume for a long time, an abnormal alarm may be issued to prompt the technician to process.

In addition, when the control device receives the data transmission result uploaded by each node device of any transmission channel, it can determine the operating state of each node device of the transmission channel according to the multiple data transmission results of the transmission channel. The running state of the node device determines whether the node device needs to be restarted. If any node device needs to be restarted, the control device can send a start command to the node device to restart the node device for normal data transmission. Of course, the control device can also perform other operations based on the received data transmission results sent by each node device, for example, when the control device determines that any node device is currently inoperable and cannot be restarted, it can perform Abnormal alarm to prompt technicians to check for abnormal problems and maintain multiple node devices in the data aggregation system.

Step 409: The destination node device receives the transmission channel delivered by the control device, and stores the data when the destination node device receives the data sent by the previous node device in the transmission channel.

Wherein, when the destination node device receives the data, the data is stored in the output device of the destination node device, that is, the storage device connected to the destination node device.

In a possible implementation manner, the input device and the output device of any node device are storage devices, and storage devices that store the same data type are connected to each other, and data sharing can be performed.

The transmission channel can be multiple, and the control device can send the amount of data to be transmitted to the destination node device when sending the transmission channel. If the storage capacity of the output device of the destination node device is limited, the data cannot be completely stored. When receiving data, the node device can forward part of the data that the output device of the destination node device cannot load, through the output device, to other storage devices of the data type stored with the output device, thereby solving the output of the destination node device The problem of limited storage capacity of equipment.

In another possible implementation manner, when the control device receives the data aggregation instruction sent by the starting node device, the data aggregation instruction includes the device identification of the starting node device, the device identification of the destination node device, and the data of the data to be transmitted The storage capacity of the output device can be obtained from the destination node device according to the device identifier of the destination node device. If the load capacity of the output device is limited, at least one output device of the same data type as the output device can be determined, and the node device connected to these output devices can also be used as the destination node device, based on the starting node device and multiple purposes The node device determines multiple transmission channels to send the same type of data to different storage devices for storage through the multiple transmission channels, so as to forward part of the data that the output device of the destination node device cannot load to the The other storage device of the data type stored by the output device solves the problem of limited storage capacity of the output device of the destination node device, and achieves an effect similar to the above-mentioned possible implementation manner.

In summary, in the embodiment of the present application, the control device obtains the configuration information of a plurality of virtual nodes, the configuration information is determined according to the configuration instruction input on the configuration device, so that when the data aggregation instruction is received, the data aggregation The instruction determines the starting virtual node and the destination virtual node, and determines the transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, and finally to the transmission channel Each node device sends a transmission channel, and any node device receives the transmission channel sent by the control device. When receiving the data sent by the previous node device in the transmission channel of the local node device, the data is sent to this node. The end node device is the next node device in the transmission channel to aggregate the data to the destination node device on the transmission channel. In this way, the configuration information can be customized through the input configuration command, so that the control device determines the transmission channel according to the configuration information customized by the technician, and the data aggregation is performed through the transmission channel, so the configuration information can be updated according to business needs, thereby Changing the node equipment in the data aggregation system expands the business capabilities of the data aggregation system and improves flexibility.

It should be noted that the data aggregation method provided in the embodiments of the present application can be applied in the scenarios shown in FIGS. 5-7. Next, the embodiments of the present application will explain these several scenarios one by one.

7 is a schematic diagram of a data aggregation scenario provided by an embodiment of the present disclosure. Referring to FIG. 7, the start node device and the destination node device are both located in the first data aggregation system, and the method shown in the embodiment shown in FIG. 4 can be used. Converge the data of the starting node device to the destination node device.

8 is a schematic diagram of another data aggregation scenario provided by an embodiment of the present disclosure. Referring to FIG. 8, the starting node device is located in the first data aggregation system, the destination node device is located in the second data aggregation system, and the first data aggregation system is The two data aggregation systems are in different networks, and the first data aggregation system is connected to the second port equipment of the second data aggregation system through the first port equipment. Among them, the second data aggregation system includes multiple node devices and control devices.

At this time, the transmission channel between the originating node device and the destination node device is restricted, and it must pass through the first port device and the second port device to send data from the first data aggregation system to the second data aggregation system. That is, the data transmission process includes the process of sending data from the starting node device to the first port device, from the first port device to the second port device, and sending the data from the second port device to the destination node device process.

Therefore, based on the embodiment shown in FIG. 4 above, steps 404-408 may specifically include the following steps:

1. The data transmission process in the first data aggregation system:

Step 801: In the first data aggregation system, the starting node device sends a data aggregation instruction to the control device, where the data aggregation instruction carries the device identification of the starting node device and the device identification of the destination node device.

Step 802: When the control device receives the data aggregation instruction, the destination node device in the data aggregation instruction is changed to the first port device to determine the arrival of the first node device from the starting node device based on the modified data aggregation instruction The transmission channel of the first node device data is first aggregated into the first port device.

Second, the data transmission process between the two data aggregation systems:

Step 802: After receiving the data of the starting node device, the first port device sends the data to the second port device of the second data aggregation system.

3. The data transmission process in the second data aggregation system:

Step 804: In the second data aggregation system, the second port device sends a data aggregation instruction to the control device, where the data aggregation instruction carries the device identification of the second port device and the device identification of the destination node device.

Step 805: When the control device receives the data aggregation instruction, based on the data aggregation instruction, the data of the second port device is aggregated into the destination node device.

The control device in the second data aggregation system determines the configuration information of multiple virtual nodes and determines the transmission channel based on the data aggregation instruction, and the process of data transmission by multiple node devices on the transmission channel is the same as that shown in FIG. The method embodiments are similar and will not be repeated here.

In another embodiment, the initial node device to send data is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system passes the The one-port device is connected to the second-port device of the second data aggregation system.

At this time, in the second data aggregation system, initially the data is sent to the starting node device to send the data aggregation instruction carrying the device identification of the starting node device and the device identification of the destination node device to the control device, when the control device receives the In the data aggregation instruction, the destination node device in the data aggregation instruction is changed to the second port device to determine the transmission channel from the starting node device to the second port device based on the changed data aggregation instruction, and the starting node device The data of is first aggregated into the second port device, and the data is sent from the second port device to the first port device of the first data aggregation system. In the first data aggregation system, the first port device sends a data aggregation instruction to the control device, the data aggregation instruction carries the device identification of the first port device and the device identification of the destination node device; when the control device receives the data aggregation instruction Based on the data aggregation instruction, the data of the second port device is aggregated into the destination node device.

In the above data aggregation scenario, the first data aggregation system and the second data aggregation system can access each other through the first port device and the second port device to perform data transmission. For example, data can be aggregated from the first data aggregation system to the second data system, or data can be aggregated from the second data aggregation system to the first data aggregation system.

In other data aggregation scenarios, the first data aggregation system and the second data aggregation system in different networks may be restricted to only one-sided access, for example, data can be aggregated from the first data aggregation system to the second data System, but the data cannot be aggregated from the second data aggregation system to the first data aggregation system; or the data can be aggregated from the second data aggregation system to the first data system, but the data cannot be aggregated from the first data aggregation system to the first 2. Data aggregation system. At this time, the data in the data aggregation system that is allowed to be accessed can be aggregated into another data aggregation system according to the allowed access direction.

9 is a schematic diagram of a data aggregation scenario provided by an embodiment of the present disclosure. Referring to FIG. 9, the starting node device is located in the first data aggregation system, the destination node device is located in the second data aggregation system, and the first data aggregation system and the second The data aggregation system is in a different network, and the first data aggregation system and the second data aggregation system are in a network isolation state. Among them, the second data aggregation system includes multiple node devices and control devices.

At this time, the transmission channel between the start node device and the destination node device is restricted, and cross-network transmission must be performed through the gatekeeper device to send data from the first data aggregation system to the second data aggregation system. That is, the data transmission process includes the process of sending data from the starting node device to the gatekeeper device, and sending the data from the gatekeeper device to the destination node device.

Therefore, based on the embodiment shown in FIG. 4 above, steps 404-408 may specifically include the following steps:

1. The data transmission process in the first data aggregation system:

Step 901: In the first data aggregation system, the starting node device sends a data aggregation instruction to the control device, where the data aggregation instruction carries the device identifier of the starting node device and the device identifier of the destination node device.

Step 902: When the control device receives the data aggregation instruction, the destination node device in the data aggregation instruction is changed to a gatekeeper device to determine the transmission from the starting node device to the gatekeeper device based on the changed data aggregation instruction Channel, the data of the starting node device is first aggregated to the gatekeeper device.

Second, the data transmission process in the second data aggregation system:

Step 903: In the second data aggregation system, the gatekeeper device sends a data aggregation instruction to the control device, where the data aggregation instruction carries the device identifier of the gatekeeper device and the device identifier of the destination node device.

Step 904: When the control device receives the data aggregation instruction, based on the data aggregation instruction, the data of the second port device is aggregated into the destination node device.

The control device in the second data aggregation system determines the configuration information of multiple virtual nodes and determines the transmission channel based on the data aggregation instruction, and the process of data transmission by multiple node devices on the transmission channel is the same as that shown in FIG. 4 above The method embodiments are similar and will not be repeated here.

In another embodiment, the starting node device that first sends data is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system passes the network The gate device is isolated from the second data aggregation system.

At this time, in the second data aggregation system, the starting node device that first sends data sends a data aggregation instruction carrying the device identifier of the starting node device and the device identifier of the destination node device to the control device; when the control device receives When the data aggregation instruction is changed, the destination node device in the data aggregation instruction is changed to a gatekeeper device to determine the transmission channel from the starting node device to the gatekeeper device based on the changed data aggregation instruction The data is first aggregated into the gatekeeper device. In the first data aggregation system, the gatekeeper device sends a data aggregation instruction to the control device, the data aggregation instruction carries the device identifier of the gatekeeper device and the device identifier of the destination node device; when the control device receives the data aggregation instruction, based on The data aggregation instruction aggregates the data of the gatekeeper device to the destination node device.

In the above data aggregation scenario, the first data aggregation system and the second data aggregation system can perform data transmission with each other through the gatekeeper device. For example, data can be aggregated from the first data aggregation system to the second data system, or data can be aggregated from the second data aggregation system to the first data aggregation system.

It should be noted that the data aggregation method provided by the embodiments of the present application can be applied to the above-mentioned several different data aggregation scenarios, which can realize the data aggregation of the starting node device and the destination node device in different networks, providing more diversification The aggregation method can meet the data aggregation requirements in different scenarios and enhance the ability of the data aggregation system.

10 is a schematic structural diagram of a data aggregation device provided by an embodiment of the present application. Referring to FIG. 10, it is applied to the implementation environment of the embodiment shown in FIG. The apparatus includes: an acquisition module 1001, a first determination module 1002, a second determination module 1003, and a first sending module 1004.

The obtaining module 1001 is configured to obtain configuration information of a plurality of virtual nodes. The configuration information is determined according to a configuration instruction input on a configuration device. The configuration information includes at least a device identifier of an input device and a device identifier of an output device. There is a one-to-one correspondence between the nodes and the multiple node devices;

The first determining module 1002 is configured to, when receiving a data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, determine the starting virtual node corresponding to the starting node device based on the data aggregation instruction and The destination virtual node corresponding to the destination node device;

The second determining module 1003 is configured to determine a transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, the transmission channel is composed of at least two node devices Connected in order;

The first sending module 1004 is configured to deliver a transmission channel to each node device on the transmission channel, and each node device on the transmission channel performs data transmission, and aggregates data of the starting node device to the destination node device.

Optionally, the device further includes:

The connection module is used for any two virtual nodes among the plurality of virtual nodes, when the device identifier of the output device of one virtual node is the same as the device identifier of the input device of the other virtual node, the two virtual nodes Node connection to obtain a virtual topology structure, the virtual topology structure is composed of the connection of the plurality of virtual nodes, and the virtual topology structure is used to describe the connection relationship of the plurality of node devices;

The second determination module 1003 includes:

Find submodule, used to find the virtual transmission channel from the starting virtual node to the destination virtual node in the virtual topology;

The determination submodule is used to determine the transmission channel from the starting node device to the destination node device according to the node device corresponding to each virtual node on the virtual transmission channel.

Optionally, the first sending module 1004 includes:

The sending submodule is used to deliver the transmission channel and monitoring tasks to each node device on the transmission channel. Each node device on the transmission channel performs data transmission, monitors the data transmission process at the local end, and sends it to the control device Data transfer results.

Optionally, the configuration information further includes a data processing chain, which is used to specify at least one data processing method when the corresponding node device receives the data to be transmitted; the apparatus further includes:

The second sending module is used to send the data processing chain of the corresponding virtual node to each node device, so that each node device adopts the data processing method in the data processing chain to receive the data to be transmitted Be processed.

Optionally, the device further includes:

The first modification module is used when the starting node device is the node device in the first data aggregation system, the destination node device is the node device in the second data aggregation system, and the first data aggregation system passes the first port device and the first When the second port device of the second data aggregation system is connected, the device identification of the destination node device in the data aggregation instruction is changed to the device identification of the first port device to determine the arrival from the starting node device based on the changed data aggregation instruction The transmission channel of the first port device aggregates the data of the starting node device into the first port device, and then sends the data to the second port device through the first port device, and then aggregates it into the second data aggregation system through the second port device The destination node device.

Optionally, the device further includes:

The second modification module is used when the starting node device is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system passes the first port device and the first When the second port device of the second data aggregation system is connected, it receives the first data aggregation instruction sent by the first port device, and the device identifier of the starting node device in the first data aggregation instruction is the device identifier of the first port device;

The control device in the second data aggregation system is used to change the device identification of the destination node device to the second port device when receiving the second data aggregation instruction including the device identification of the start node device and the device identification of the destination node device Device identification to determine the transmission channel from the starting node device to the second port device based on the modified data aggregation instruction, and aggregate the data of the starting node device into the second port device, which will be used by the second port device The data is sent to the first port device, and the first port device sends the first data aggregation instruction to the control device in the first data aggregation system.

Optionally, the device further includes:

The third modification module is used when the starting node device is the node device in the first data aggregation system, the destination node device is the node device in the second data aggregation system, and the first data aggregation system passes the gatekeeper device and the second When the data aggregation system is isolated, the destination node device in the data aggregation instruction is changed to a gatekeeper device to determine the transmission channel from the starting node device to the gatekeeper device based on the changed data aggregation instruction, and the starting node device The data of is first aggregated into the gatekeeper device, and then the data is aggregated to the destination node device in the second data aggregation system through the gatekeeper device.

Optionally, the device further includes:

The fourth modification module is used when the starting node device is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system communicates with the second device through the gatekeeper device. When the data aggregation system is isolated, the first data aggregation instruction sent by the gatekeeper device is received, and the device identifier of the starting node device in the first data aggregation instruction is the device identifier of the gatekeeper device;

The control device in the second data aggregation system is used to change the device identification of the destination node device to the gatekeeper device when receiving the second data aggregation instruction including the device identification of the start node device and the device identification of the destination node device Device identification, to determine the transmission channel from the starting node device to the gatekeeper device based on the changed data aggregation instruction, to aggregate the data of the starting node device into the gatekeeper device, and the gatekeeper device to aggregate the first data The control device in the system sends the first data aggregation instruction.

In summary, in the data aggregation apparatus provided in the embodiments of the present application, the control device obtains configuration information of multiple virtual nodes, and the configuration information is determined according to the configuration instruction input on the configuration device, so the data aggregation instruction can be received At the time, determine the starting virtual node and the destination virtual node according to the data aggregation instruction, and determine the transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, The transmission channel is delivered to each node device of the transmission channel, and each node device on the transmission channel performs data transmission, and aggregates the data of the starting node device to the destination node device. The configuration information can be customized through the input configuration instruction, so that the control device determines the transmission channel according to the customized configuration information, and the data aggregation is performed through the transmission channel, so the configuration information can be updated according to the business needs, thereby changing the data aggregation system The node equipment in the system expands the business capability of the data aggregation system and improves flexibility.

FIG. 11 is a schematic structural diagram of another data aggregation device provided by an embodiment of the present application. Referring to FIG. 11, the device includes: a receiving module 1101 and a first sending module 1102.

The receiving module 1101 is configured to receive a transmission channel delivered by the control device. The transmission channel is determined by the control device according to the configuration information of multiple virtual nodes and the node device corresponding to each virtual node. The configuration information is based on the input on the configuration device The configuration instruction determines that at least the device identifier of the input device and the device identifier of the output device are included. The transmission channel is formed by connecting at least two node devices in sequence and includes the local node device;

The first sending module 1102 is configured to, when receiving data sent by the previous node device of the local node device in the transmission channel, send the data to the next node device of the local node device in the transmission channel, so as to This data is aggregated into the destination node device on the transmission channel.

Optionally, the configuration information further includes a data processing chain, which is used to specify at least one data processing method when the corresponding node device receives the data to be transmitted; the first sending module 1102 includes:

The processing sub-module is used to process the data by using the data processing method in the data processing chain when receiving the data;

The sending submodule is used to send the processed data to the next node device in the transmission channel of the local node device.

Optionally, the device further includes:

The second sending module is used to send a data aggregation instruction to the control device, where the data aggregation instruction includes the device identification of the starting node device and the device identification of the destination node device, and the starting node device is the local node device, so that the control device Determine the transmission channel from the local node device to the destination node device.

Optionally, the device further includes:

The storage module is used for storing the data when the local node device is the destination node device in the transmission channel when it receives the data sent by the previous node device in the transmission channel of the local node device.

In summary, in the data aggregation apparatus provided in the embodiments of the present application, any node device receives the transmission channel delivered by the control device. When receiving the data sent by the previous node device in the transmission channel of the local node device, Send the data to the next node device in the transmission channel of the local node device to converge the data to the destination node device on the transmission channel. The configuration information can be customized through the input configuration instruction, so that the control device determines the transmission channel according to the customized configuration information and sends it to each node device in the transmission channel. Any node device can be based on the transmission channel Data aggregation is performed, so configuration information can be updated according to business requirements, thereby changing node devices in the data aggregation system, expanding the business capabilities of the data aggregation system, and increasing flexibility.

It should be noted that when the data aggregation device provided in the above embodiment performs data aggregation, only the above-mentioned division of each functional module is used as an example for illustration. In practical applications, the above-mentioned functions may be allocated by different functional modules according to needs. That is, the internal structure of the control device and the node device is divided into different functional modules to complete all or part of the functions described above. In addition, the data aggregation device and the data aggregation method embodiment provided in the above embodiments belong to the same concept. For the specific implementation process, see the method embodiment, and details are not described here.

FIG. 12 is a schematic structural diagram of a server provided by an embodiment of the present application. The server 1200 may have a relatively large difference due to different configurations or performances, and may include one or more than one central processing unit (CPU) 1201 and one Or more than one memory 1202, wherein at least one instruction is stored in the memory 1202, and the at least one instruction is loaded and executed by the processor 1201. Of course, the server 1200 may also have components such as a wired or wireless network interface, a keyboard, and an input-output interface for input and output. The server 1200 may also include other components for implementing device functions, which will not be repeated here.

The server 1200 is used to perform the operations performed by the control device or the node device in the above data aggregation method.

In an exemplary embodiment, a computer-readable storage medium is also provided, for example, a memory including instructions that can be executed by the processor in the terminal or server to complete the data aggregation method in the above embodiments. For example, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, or the like.

A person of ordinary skill in the art may understand that all or part of the steps to implement the above-described embodiments may be completed by hardware, or may be completed by a program instructing related hardware. The program may be stored in a computer-readable storage medium. The mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above are only preferred embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the protection of this application Within range.

Claims (29)

1. A data aggregation method, characterized in that it is applied to a control device of a first data aggregation system, the first data aggregation system includes a plurality of node devices and the control device, the control device and the plurality of Node device connection, the method includes:

   Acquiring configuration information of a plurality of virtual nodes, the configuration information is determined according to a configuration instruction input on a configuration device connected to the control device, the configuration information includes at least a device identification of an input device and a device identification of an output device The multiple virtual nodes correspond to the multiple node devices in one-to-one correspondence;

   When receiving a data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, based on the data aggregation instruction, the starting virtual node corresponding to the starting node device and the destination node device are determined The corresponding destination virtual node;

   According to the configuration information of the plurality of virtual nodes and the node device corresponding to each virtual node, determine a transmission channel from the starting node device to the destination node device, the transmission channel is in order by at least two node devices Connected

   Deliver the transmission channel to each node device on the transmission channel, and each node device on the transmission channel performs data transmission to aggregate the data of the starting node device to the destination node device in.
2. The method according to claim 1, wherein after acquiring configuration information of a plurality of virtual nodes, the method further comprises:

   For any two virtual nodes of the plurality of virtual nodes, when the device identification of the output device of one virtual node is the same as the device identification of the input device of the other virtual node, connect the two virtual nodes, Obtain a virtual topology structure, the virtual topology structure is formed by the connection of the multiple virtual nodes, and the virtual topology structure is used to describe the connection relationship of the multiple node devices;

   The determining the transmission channel from the starting node device to the destination node device according to the configuration information of the plurality of virtual nodes and the node device corresponding to each virtual node includes:

   In the virtual topology structure, search for a virtual transmission channel from the starting virtual node to the destination virtual node;

   The transmission channel from the starting node device to the destination node device is determined according to the node device corresponding to each virtual node on the virtual transmission channel.
3. The method according to claim 1, wherein the delivering the transmission channel to each node device on the transmission channel comprises:

   Deliver the transmission channel and the monitoring task to each node device on the transmission channel, each node device on the transmission channel performs data transmission, and monitors the data transmission process at the local end to the control device Send data transfer results.
4. The method according to claim 1, wherein the configuration information further includes a data processing chain, and the data processing chain is used to specify at least one data processing method when the corresponding node device receives the data to be transmitted; The method also includes:

   Sending a data processing chain of the corresponding virtual node to each node device, so that each node device uses the data processing mode in the data processing chain to process the data when receiving the data to be transmitted.
5. The method according to claim 1, wherein the method further comprises:

   When the starting node device is the node device in the first data aggregation system, the destination node device is the node device in the second data aggregation system, and the first data aggregation system passes the first port device and the second When the second port device of the data aggregation system is connected, the device identification of the destination node device in the data aggregation instruction is changed to the device identification of the first port device, to determine from the data aggregation instruction based on the changed data aggregation instruction The starting node device reaches the transmission channel of the first port device, aggregates the data of the starting node device into the first port device, and then sends the data to the second port through the first port device The device is aggregated to the destination node device in the second data aggregation system through the second port device.
6. The method according to claim 1, wherein the method further comprises:

   When the starting node device is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system passes the first port device and the second When the second port device of the data aggregation system is connected, it receives the first data aggregation instruction sent by the first port device, and the device identifier of the starting node device in the first data aggregation instruction is that of the first port device Equipment Identity;

   The control device in the second data aggregation system is configured to, when receiving the second data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, The device identification is changed to the device identification of the second port device to determine the transmission channel from the starting node device to the second port device based on the changed data aggregation instruction, and change the starting node device's Data is aggregated into the second port device, the second port device sends the data to the first port device, and the first port device sends the data to the control device in the first data aggregation system Sending the first data aggregation instruction.
7. The method according to claim 1, wherein the method further comprises:

   When the starting node device is a node device in the first data aggregation system, the destination node device is a node device in the second data aggregation system, and the first data aggregation system communicates with the second data through a gatekeeper device When the aggregation system is isolated, the destination node device in the data aggregation instruction is changed to the gatekeeper device to determine the transmission channel from the starting node device to the gatekeeper device based on the changed data aggregation instruction , The data of the starting node device is first aggregated into the gatekeeper device, and then the data is aggregated to the destination node device in the second data aggregation system through the gatekeeper device.
8. The method according to claim 1, wherein the method further comprises:

   When the starting node device is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system communicates with the second data through the gatekeeper device When the aggregation system is isolated, receive the first data aggregation instruction sent by the gatekeeper device, and the device identifier of the starting node device in the first data aggregation instruction is the device identifier of the gatekeeper device;

   The control device in the second data aggregation system is configured to, when receiving the second data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, The device identification is changed to the device identification of the gatekeeper device to determine the transmission channel from the starting node device to the gatekeeper device based on the changed data aggregation instruction and to aggregate the data of the starting node device To the gatekeeper device, the gatekeeper device sends the first data aggregation instruction to the control device in the first data aggregation system.
9. A data aggregation method is characterized in that it is applied to any node device in a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device. To connect multiple node devices, the method includes:

   Receiving a transmission channel delivered by the control device, the transmission channel is determined by the control device according to configuration information of multiple virtual nodes and a node device corresponding to each virtual node, and the configuration information is based on The configuration instruction input on the connected configuration device determines that it includes at least the device identification of the input device and the device identification of the output device, the transmission channel is formed by connecting at least two node devices in sequence, and includes the local node device;

   When receiving the data sent by the last node device of the local node device on the transmission channel, send the data to the next node device of the local node device on the transmission channel, to Aggregate the data to the destination node device on the transmission channel.
10. The method according to claim 9, wherein the configuration information further includes a data processing chain, and the data processing chain is used to specify at least one data processing method when the corresponding node device receives the data to be transmitted; When receiving the data sent by the previous node device of the local node device in the transmission channel, send the data to the next node device of the local node device in the transmission channel ,include:

    When receiving the data, the data processing method in the data processing chain is used to process the data, and the processed data is sent to the next node device in the transmission channel of the local node device .
11. The method according to claim 9, wherein before the receiving the transmission channel sent by the control device, the method further comprises:

    Send a data aggregation instruction to the control device, where the data aggregation instruction includes the device identifier of the starting node device and the device identifier of the destination node device, and the starting node device is the local node device, so that the The control device determines the transmission channel from the local node device to the destination node device.
12. The method according to claim 9, wherein the method further comprises:

    If the local node device is a destination node device in the transmission channel, when the data sent by the local node device in the previous node device in the transmission channel is received, the data is stored.
13. An apparatus for data aggregation is characterized in that it is applied to a control device of a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device. Node device connection, the device includes:

    An obtaining module, configured to obtain configuration information of a plurality of virtual nodes, the configuration information is determined according to a configuration instruction input on a configuration device connected to the control device, and the configuration information includes at least a device identifier and an output device of the input device Device identification, the multiple virtual nodes are in one-to-one correspondence with the multiple node devices;

    The first determining module is configured to, when receiving a data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, determine the starting virtual corresponding to the starting node device based on the data aggregation instruction A node and a destination virtual node corresponding to the destination node device;

    The second determining module is configured to determine a transmission channel from the starting node device to the destination node device according to the configuration information of the multiple virtual nodes and the node device corresponding to each virtual node, the transmission channel is At least two node devices are connected in order;

    The first sending module is used to deliver the transmission channel to each node device on the transmission channel, and each node device on the transmission channel performs data transmission to transfer the data of the starting node device Converged into the destination node device.
14. The device according to claim 13, wherein the device further comprises:

    The connection module is configured to, for any two virtual nodes of the plurality of virtual nodes, when the device identification of the output device of one of the virtual nodes is the same as the device identification of the input device of the other virtual node, Virtual nodes are connected to obtain a virtual topology structure, the virtual topology structure is composed of the multiple virtual node connections, and the virtual topology structure is used to describe the connection relationship of the multiple node devices;

    The second determination module includes:

    A search submodule, used to search for a virtual transmission channel from the starting virtual node to the destination virtual node in the virtual topology structure;

    The determination submodule is configured to determine a transmission channel from the starting node device to the destination node device according to the node device corresponding to each virtual node on the virtual transmission channel.
15. The apparatus according to claim 13, wherein the first sending module comprises:

    The sending submodule is used to deliver the transmission channel and the monitoring task to each node device on the transmission channel, each node device on the transmission channel performs data transmission, and monitors the data transmission process at the local end To send the data transmission result to the control device.
16. The apparatus according to claim 13, wherein the configuration information further includes a data processing chain, the data processing chain is used to specify at least one data processing method when the corresponding node device receives the data to be transmitted; The device also includes:

    The second sending module is used to send the data processing chain of the corresponding virtual node to each node device, so that each node device adopts the data processing method in the data processing chain to receive the data to be transmitted The data is processed.
17. The device according to claim 13, wherein the device further comprises:

    The first modification module is used when the starting node device is a node device in the first data aggregation system, the destination node device is a node device in the second data aggregation system, and the first data is aggregated When the system is connected to the second port device of the second data aggregation system through the first port device, the node identifier of the destination node device in the data aggregation instruction is changed to the node identifier of the first port device, based on The modified data aggregation instruction determines the transmission channel from the starting node device to the first port device, and aggregates the data of the starting node device into the first port device before passing the The first port device sends to the second port device, and converges to the destination node device in the second data aggregation system through the second port device.
18. The device according to claim 13, wherein the device further comprises:

    The second modification module is used when the starting node device is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system passes the first When the port device is connected to the second port device of the second data aggregation system, it receives the first data aggregation instruction sent by the first port device, and the device identifier of the starting node device in the first data aggregation instruction is The device identification of the first port device;

    The control device in the second data aggregation system is configured to, when receiving the second data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, The device identification is changed to the device identification of the second port device to determine the transmission channel from the starting node device to the second port device based on the changed data aggregation instruction, and change the starting node device's Data is aggregated into the second port device, the second port device sends the data to the first port device, and the first port device sends the data to the control device in the first data aggregation system Sending the first data aggregation instruction.
19. The device according to claim 13, wherein the device further comprises:

    The third modification module is used when the starting node device is a node device in the first data aggregation system, the destination node device is a node device in the second data aggregation system, and the first data is aggregated When the system is isolated from the second data aggregation system by a gatekeeper device, the destination node device in the data aggregation instruction is changed to the gatekeeper device to determine the starting point from the modified data aggregation instruction The node device reaches the transmission channel of the gatekeeper device, and the data of the starting node device is first aggregated into the gatekeeper device, and then the data is aggregated to the second data aggregation through the gatekeeper device The destination node device in the system.
20. The device according to claim 13, wherein the device further comprises:

    The fourth modification module is used when the starting node device is the node device in the second data aggregation system, the destination node device is the node device in the first data aggregation system, and the first data aggregation system passes through the gatekeeper When the device is isolated from the second data aggregation system, it receives the first data aggregation instruction sent by the gatekeeper device, and the device identifier of the starting node device in the first data aggregation instruction is the device of the gatekeeper device Logo

    The control device in the second data aggregation system is configured to, when receiving the second data aggregation instruction including the device identification of the starting node device and the device identification of the destination node device, The device identification is changed to the device identification of the gatekeeper device to determine the transmission channel from the starting node device to the gatekeeper device based on the changed data aggregation instruction and to aggregate the data of the starting node device To the gatekeeper device, the gatekeeper device sends the first data aggregation instruction to the control device in the first data aggregation system.
21. An apparatus for data aggregation is characterized in that it is applied to any node device in a first data aggregation system. The first data aggregation system includes a plurality of node devices and the control device. To connect multiple node devices, the apparatus includes:

    The receiving module is configured to receive a transmission channel delivered by the control device, and the transmission channel is determined by the control device according to configuration information of multiple virtual nodes and a node device corresponding to each virtual node, and the configuration information is based on The configuration instruction input on the configuration device connected to the control device is determined to include at least the device identification of the input device and the device identification of the output device, the transmission channel is formed by connecting at least two node devices in sequence, and includes the local end Node equipment

    A first sending module, configured to send the data to the local node device in the transmission channel when receiving the data sent by the previous node device of the local node device in the transmission channel The next node device to aggregate the data into the destination node device on the transmission channel.
22. The apparatus according to claim 21, wherein the configuration information further includes a data processing chain, and the data processing chain is used to specify at least one data processing method when the corresponding node device receives the data to be transmitted; The first sending module includes:

    A processing submodule, configured to process the data by using the data processing method in the data processing chain when receiving the data;

    The sending submodule is used to send the processed data to the next node device in the transmission channel of the local node device.
23. The device according to claim 21, wherein the device further comprises:

    A second sending module, configured to send a data aggregation instruction to the control device, the data aggregation instruction includes a device identifier of a start node device and a device identifier of a destination node device, and the start node device is the local end A node device, so that the control device determines a transmission channel from the local node device to the destination node device.
24. The device according to claim 21, wherein the device further comprises:

    The storage module is used for storing the data when the local node device is the destination node device in the transmission channel when receiving the data sent by the local node device on the previous node device in the transmission channel述 数据。 The data.
25. A control device, characterized in that it is applied to a first data aggregation system, the first data aggregation system includes a plurality of node devices and the control device, the control devices are respectively connected to the plurality of node devices, The control device includes a processor and a memory, at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the data according to any one of claims 1 to 8. Convergence method.
26. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of claims 1 to 8. Data aggregation method.
27. A node device, characterized in that it is applied to a first data aggregation system, the first data aggregation system includes a plurality of node devices and the control device, and the control device is respectively connected to the plurality of node devices;

    The local node device includes a processor and a memory, and the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the data according to any one of claims 9 to 12. Convergence method.
28. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of claims 9 to 12. Data aggregation method.
29. A data aggregation system, characterized in that the data aggregation system includes multiple node devices and control devices, and the control devices are respectively connected to the multiple node devices;

    The control device is the control device according to claim 25;

    Each node device is the node device according to claim 27.

Images