WO2023193671A1 - 数据传输方法以及系统 - Google Patents

数据传输方法以及系统 Download PDF

Info

Publication number
WO2023193671A1
WO2023193671A1 PCT/CN2023/085828 CN2023085828W WO2023193671A1 WO 2023193671 A1 WO2023193671 A1 WO 2023193671A1 CN 2023085828 W CN2023085828 W CN 2023085828W WO 2023193671 A1 WO2023193671 A1 WO 2023193671A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
data
working
data acquisition
nodes
Prior art date
Application number
PCT/CN2023/085828
Other languages
English (en)
French (fr)
Inventor
何淋波
Original Assignee
阿里巴巴（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴（中国）有限公司 filed Critical 阿里巴巴（中国）有限公司
Publication of WO2023193671A1 publication Critical patent/WO2023193671A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control

Definitions

  • the embodiments of this specification relate to the field of data processing technology, and in particular, to a data transmission method.
  • Kubernetes systems are becoming more and more popular in edge computing scenarios.
  • kubernetes master is deployed in the cloud
  • kubernetes workers and project pods are deployed in the edge environment (closer to the user site)
  • K8s workers and master are connected through the public network .
  • edge nodes are generally not configured with public network addresses, the K8s master cannot directly access workers.
  • the Kubernetes cluster at the edge of the cloud exceeds a certain scale (for example: nodes > 10k, pod > 200k, service > 10k)
  • requests for endpoints/endpointslices resources from all edge nodes will be triggered, thus This leads to a significant increase in cloud-edge communication traffic.
  • embodiments of this specification provide a data transmission method.
  • One or more embodiments of this specification simultaneously relate to a communication method between a cloud master node and an edge node, a data transmission device, a computing device, a computer-readable storage medium and a computer program to solve existing problems.
  • Technical flaws in the technology are simultaneously relate to a communication method between a cloud master node and an edge node, a data transmission device, a computing device, a computer-readable storage medium and a computer program to solve existing problems.
  • a data transmission method includes a master node, at least two working nodes, and a data acquisition working node determined from the at least two working nodes. Coordinate working nodes, the method includes:
  • the master node responds to the first data acquisition instruction sent by the data acquisition working node, and sends the target data to the data acquisition working node;
  • the data acquisition working node receives the target data sent by the master node and stores the target data to the coordination working node;
  • the coordination working node responds to the second data acquisition instruction sent by other working nodes, and sends the target data to the other working nodes, wherein the other working nodes are the at least two working nodes except the coordination working node. Worker nodes outside the node.
  • a communication method between a cloud master node and an edge node is provided, which is applied to a data transmission system.
  • the data transmission system includes a cloud master node, at least two edge nodes, and a server from the cloud master node.
  • the data acquisition edge node and the coordination edge node are determined in at least two edge nodes, including:
  • the cloud master node responds to the first data acquisition instruction sent by the data acquisition edge node, and sends the target data to the data acquisition edge node;
  • the data acquisition edge node receives the target data sent by the cloud master node and stores the target data in the coordination edge node;
  • the coordinating edge node sends the target data to the other edge nodes in response to the second data acquisition instruction sent by the other edge nodes, wherein the other edge nodes are the at least two edge nodes except the coordinating edge node. Edge nodes outside the node.
  • a data transmission system including a master node, at least two working nodes, and a data acquisition working node and a coordinating working node determined from the at least two working nodes, wherein ,
  • the master node is configured to send target data to the data acquisition working node in response to the first data acquisition instruction sent by the data acquisition working node;
  • the data acquisition working node is configured to receive the target data sent by the master node and store the target data in the coordination working node;
  • the coordinating working node is configured to send the target data to the other working nodes in response to the second data acquisition instruction sent by the other working nodes, wherein the other working nodes are all except the at least two working nodes.
  • a data transmission device which is applied to a data transmission system.
  • the data transmission system includes a master node, at least two working nodes, and a data transmission device determined from the at least two working nodes.
  • Data acquisition working node and coordination working node, the device includes:
  • the main module is configured to send target data to the data acquisition working node in response to the first data acquisition instruction sent by the data acquisition working node;
  • the data acquisition module is configured to receive the target data sent by the master node and store the target data in the coordination working node;
  • the coordination module is configured to send the target data to the other working nodes in response to the second data acquisition instruction sent by the other working nodes, wherein the other working nodes are all the working nodes except the at least two working nodes. Work nodes other than the coordination work node.
  • a computing device including:
  • the memory is used to store computer-executable instructions
  • the processor is used to execute the computer-executable instructions
  • the computer-executable instructions implement the steps of the above data transmission method when executed by the processor.
  • a computer-readable storage medium which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the above data transmission method are implemented.
  • a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to perform the steps of the above data transmission method.
  • Embodiments of this specification provide a data transmission method and system.
  • the method is applied to a data transmission system.
  • the data transmission system includes a master node, at least two working nodes, and data acquisition determined from the at least two working nodes.
  • Working nodes and coordinating working nodes the method includes: the master node responds to the first data acquisition instruction sent by the data acquiring working node, sending target data to the data acquiring working node; the data acquiring working node receives The target data sent by the master node, and the target data is stored in the coordination working node; the coordination working node responds to the second data acquisition instruction sent by other working nodes, and sends the target data to the other nodes.
  • the target data is provided, wherein the other working nodes are working nodes other than the coordinating working node among the at least two working nodes.
  • the data acquiring working node obtains the target data from the master node and stores it in the coordinating working node.
  • Each working node does not need to obtain data from the master node when acquiring data, reducing the number of master nodes. Communication traffic between nodes and worker nodes.
  • Figure 1 is a flow chart of a data transmission method provided by an embodiment of this specification
  • Figure 2 is a flow chart of a communication method between a cloud master node and an edge node provided by an embodiment of this specification;
  • Figure 3 is a schematic diagram of a communication method between a cloud master node and an edge node provided by an embodiment of this specification;
  • Figure 4 is a schematic structural diagram of a data transmission device provided by an embodiment of this specification.
  • Figure 5 is a structural block diagram of a computing device provided by an embodiment of this specification.
  • first, second, etc. may be used to describe various information in one or more embodiments of this specification, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other.
  • the first may also be referred to as the second, similarly Land, the second can also be called the first.
  • the word "if” as used herein may be interpreted as "when” or “when” or “in response to determining.”
  • Kubernetes An open source system for automating the deployment, scaling, and management of containerized applications.
  • Kubelet It is the main service on the worker node. It regularly receives new or modified Pod specifications from the kube-apiserver component and ensures that the Pod and its containers run under the expected specifications.
  • Kube-proxy maintains network rules on nodes and implements part of the Kubernetes Service concept. Its role is to load balance traffic to the Service (via ClusterIP and port) to the correct backend Pod.
  • YurtHub It is a reverse proxy for client Request.
  • Transmission Control Protocol (TCP, Transmission Control Protocol): It is a connection-oriented, reliable, byte stream-based transport layer communication protocol.
  • Pod is the smallest deployable computing unit that can be created and managed in Kubernetes, containing one or more application containers.
  • Node Pool Multiple nodes with the same attributes and characteristics (such as being deployed in the same physical area and accessible to each other) in the Kubernetes cluster and its extended cluster.
  • Pool Scope Data Each node in NodePool obtains the same data from the cloud. Data returned by list requests for resources such as Node, Endpointslices, Endpoints, and Service.
  • Node Scope Data The data obtained by each node in NodePool from the cloud is unique to this node. Data returned by get requests such as Pods, Secrets, Configmaps and other resources.
  • Edge node refers to a project platform built at the edge of the network close to the user, providing storage, computing, network and other resources, and sinking some key project applications to the edge of the access network to reduce the width of network transmission and multi-level forwarding and delay loss.
  • the edge node is located between the user and the cloud center.
  • the edge list/watch request may be triggered, thereby significantly increasing the cloud edge traffic.
  • the native list/watch requests in Kubernetes are abandoned and transformed into the cloud actively pushing data to the edge. The specific plan is described below.
  • the edge node and the cloud cluster build a TCP long connection.
  • the cloud cluster adds a controller (controller) to monitor (list/watch) the data changes of the cloud cluster in real time.
  • the controller detects the data
  • the changed data is pushed to each edge node through the TCP long connection of the cloud edge cluster.
  • the list/watch mechanism is the underlying mechanism for communication between components in the Kubernetes system, and the list is abandoned.
  • the /watch mechanism means destructive and intrusive modifications to the Kubernetes system, causing cloud-native ecological applications that use the list/watch mechanism on edge nodes to be unable to run.
  • the cloud resources such as endpoints
  • the cloud still needs to push data to each edge node. Therefore, the cloud edge traffic will not be reduced in most scenarios, but the traffic can be reduced in some scenarios.
  • the cloud The reduction effect of edge communication data is not good.
  • the cloud actively pushes data to the edge nodes, it means that the cloud does not know what type of data the edge nodes need, so the cloud needs to push all types of data to the edge nodes. Therefore, a lot of invalid data will be pushed, resulting in a waste of cloud edge traffic.
  • a data transmission method is provided.
  • This specification also relates to a communication method between a cloud master node and an edge node, a data transmission device, a computing device, and a computer-readable storage
  • the media will be described in detail one by one in the following embodiments.
  • Figure 1 shows a flow chart of a data transmission method according to an embodiment of this specification, which is applied to a data transmission system.
  • the data transmission system includes a master node, at least two working nodes, and all slave nodes.
  • the data acquisition working node and the coordination working node determined among the at least two working nodes, wherein the main node in this embodiment can be understood as the aforementioned cloud, the working node in this embodiment can be understood as the aforementioned edge node, and
  • the coordination work node is pre-configured, and the data transmission method specifically includes the following steps.
  • Step 102 The master node responds to the first data acquisition instruction sent by the data acquisition working node, and sends target data to the data acquisition working node.
  • the data acquisition working node can be any working node, and there is a communication connection between the node and the master node;
  • the first data acquisition instruction can be a list/watch request in Kubernetes;
  • the target data can be Pool Scope Data in the noun explanation , such as data returned by get requests for resources such as Pods, Secrets, Configmaps, etc.
  • all working nodes belong to the same node pool (NodePool), and each node in the node pool can transmit data to each other.
  • the data acquisition working node in the node pool sends the first data acquisition instruction to the master node, and the master node receives the For the first data acquisition instruction, the corresponding data is sent to the data acquisition working node.
  • the working node includes working node A and working node B.
  • Working node B is a data acquisition working node. Then working node B sends a first data acquisition instruction to the master node. When the master node receives the first data acquisition instruction, Send target data to worker node B.
  • the embodiment of this specification obtains the target data through one node, so that the target data can be stored later, and other nodes can obtain the target data from the node pool, thereby not requesting the target data from the master node and reducing communication traffic.
  • Step 104 The data acquisition working node receives the target data sent by the master node, and stores the target data in the coordination working node.
  • the data acquisition working node After receiving the target data, the data acquisition working node stores the target data in a pre-configured Set up the coordination work node so that other work nodes can obtain data from the coordination work node.
  • the working nodes include working node A, working node B and working node C.
  • Working node B is a data acquisition working node
  • working node C is a coordinating working node. Then working node B sends a first data acquisition instruction to the master node, and the master node When receiving the first data acquisition instruction, the target data is sent to the working node B, and the working node B stores the target data to the working node C.
  • the data acquisition working node can be obtained through election.
  • the specific implementation method is as follows.
  • the at least two working nodes send lock instructions to the coordinating working node
  • the coordination working node receives the lock instruction sent by the at least two working nodes, and in the case of receiving the lock instruction, determines the data acquisition working node according to the lock instruction.
  • the lock instruction can be understood as an instruction that modifies the data to indicate locking. For example, when the initial value of data X is 0, modifying the value of data X to 1 means that the value of data X has been locked, and It cannot be changed.
  • the locking mechanism can use conventional technical means in the art, and is not limited by the embodiments of this specification.
  • the working nodes include working node A, working node B and working node C.
  • Working node C is the coordinating working node. Then working node A and working node B send lock instructions to working node C. If the lock instruction of working node B arrives first Working node C has modified the data value of the lock mechanism, changing 0 to 1, and working node B sends a connection request to the master node. When the lock instruction of working node A reaches working node C, the data of the lock mechanism is found. If the value is already 1, it means that the election has been successful and working node B can be obtained as the data acquisition node.
  • the embodiment of this specification determines the data acquisition working node through election, and can determine the working node with a smaller communication delay as the data acquisition working node, thereby improving communication efficiency.
  • a proxy module is added to the working node.
  • the proxy module is used to intercept the data acquisition request issued by the service module to realize the integration of each working node. Data requests in are forwarded to the coordinating work node.
  • each of the at least two working nodes includes a service module and an agent module;
  • the coordinating working node Before the coordinating working node responds to the second data acquisition instruction sent by other working nodes, it also includes:
  • the service module sends a second data acquisition instruction to the master node
  • the agent module intercepts the second data acquisition instruction and sends the second data acquisition instruction to the coordination work node.
  • the service module can be Kubelet, Kube-proxy, etc.
  • the proxy module can be YurtHub, etc.
  • the service module should send the second data acquisition instruction to the master node and request the target data from the master node. Then the agent module can be added to each working node, and the agent module will transfer the service in the working node where it is located. The module intercepts the second data acquisition instruction that should be sent to the master node, and sends the second data acquisition instruction to the coordinating working node.
  • working nodes include working node A, working node B, and working node C.
  • Working node A, working node B, and working node C all include service modules and agent modules.
  • Working node C is a coordinating working node, and working node B is a data Acquire the node.
  • the service module in the working node A needs to request data from the master node, it sends a second data acquisition instruction to the master node.
  • the agent module in the working node A intercepts the second data acquisition instruction and obtains the second data.
  • the instruction is redirected to worker node C to obtain the target data.
  • the service module in working node B needs to request data from the master node, it sends a second data acquisition instruction to the master node.
  • the agent module in working node B intercepts the second data acquisition instruction and obtains the second data.
  • the command is redirected to worker node C.
  • the service module in the working node C needs to request data from the master node, it sends a second data acquisition instruction to the master node.
  • the agent module in the working node C intercepts the second data acquisition instruction and redirects the second data acquisition instruction. to local address.
  • a proxy module is added to the edge node to take over the communication traffic between the working node and the master node. No modifications are made to the Kubernetes architecture, and the application of the cloud native ecosystem can be very good. It runs on edge nodes, and all requests are initiated by the data demander (edge node). Cloud-edge communication will not generate invalid traffic.
  • the data acquisition working node in the above embodiment can be obtained through election and sending the first data acquisition instruction to the master node can be executed by the agent module.
  • the working nodes include working node A, working node B and working node C.
  • Working node C is the coordinating working node. Then the agent module of working node A and the agent module of working node B send lock instructions to working node C.
  • the lock instruction of the agent module of B reaches the working node C first, and the data value of the lock mechanism is modified, changing 0 to 1, and the agent module of the working node B sends a connection request to the master node, and the lock of the agent module of the working node A
  • the instruction reaches working node C and finds that the data value of the lock mechanism is already 1, it can be known that the election has been successful and working node B can be obtained as the data acquisition node.
  • the working node in addition to the target data, the working node will obtain data according to the needs of its own internal service module, so it needs to be obtained directly from the master node.
  • the specific implementation method is as follows.
  • the target working node among the at least two working nodes obtains demand data from the coordinating working node. If the demand data does not exist in the coordinating working node, sends the third data acquisition to the master node. instruction;
  • the master node responds to the third data acquisition instruction and sends the demand data to the target working node;
  • the target working node receives the demand data, where the demand data is other data in the master node except the target data.
  • the demand data can be Node Scope Data, such as the data returned by get requests for Pods, Secrets, Configmaps and other resources;
  • the third data acquisition instruction can be list/watch requests in Kubernetes.
  • working nodes include working node A, working node B, and working node C.
  • Working node A, working node B, and working node C all include service modules and agent modules.
  • Working node C is a coordinating working node
  • working node B is a data
  • the service module in the working node A sends a message to the master node when it needs to request data from the master node.
  • the agent module in the working node A intercepts the third data acquisition instruction and redirects the third data acquisition instruction to the working node C to obtain the corresponding data.
  • the working node C finds that the third data acquisition instruction is not stored. If the corresponding data is commanded, the working node A is notified, the working node A directly sends the third data acquisition instruction to the master node, and the master node returns the data corresponding to the third data acquisition instruction to the working node A.
  • the working node when a working node has separate needs, the working node directly requests data from the master node, without occupying the storage space of the coordination node.
  • the target data may change, or the working node needs to actively update the data, then it needs to request the master node to update the target data.
  • the specific implementation method is as follows.
  • the data acquisition working node sends a data update instruction to the master node
  • the master node responds to the data update instruction and sends update data to the data acquisition working node;
  • the data acquisition working node obtains updated data from the master node, and stores the updated data to the coordination working node, where the updated data is the changed data in the target data.
  • working node C is the coordinating working node
  • working node B is the data acquisition node.
  • Working node B is set to send data update instructions to the master node every 24 hours.
  • the master node responds to the data update instructions and sends the target to working node B.
  • For data update data after the working node B receives the updated data, it sends the updated data to the working node C, and the working node C modifies the target data according to the updated data.
  • the embodiment of this specification modifies the target data based on the update data instead of transmitting the entire target data, thereby reducing communication traffic.
  • the coordinating working node sends the target data to the other working nodes in response to the second data acquisition instruction sent by the other working nodes, wherein the other working nodes are the at least two working nodes except the Coordinate worker nodes outside of worker nodes.
  • the second data acquisition instruction can be a list/watch request in Kubernetes.
  • each proxy module forwards the request to the coordinating working node, and obtains the latest written target data from it. This increases the effect of cloud-side traffic reuse and reduces cloud-side public network communication traffic.
  • a coordination module is added to the working node to configure the working node as a coordination working node, and the coordination module is used to manage target data .
  • the specific implementation is as follows.
  • the coordination work node also includes the coordination module
  • the coordination module receives the second data acquisition instruction sent by any one of the agent modules, and sends the target data to the agent module;
  • the agent module When receiving the target data, the agent module sends the target data to the corresponding service module.
  • the coordination module can be understood as a module that manages target data, such as a node pool coordinator (pool-assistant). It should be noted that the node pool coordinator can be deployed on any working node, which is not limited to the embodiments of this specification. .
  • working nodes include working node A, working node B, and working node C.
  • Working node A, working node B, and working node C all include service modules and agent modules.
  • Working node C is a coordinating working node, and working node B is a data Acquire the node.
  • the service module in the working node A needs to request data from the master node, it sends a second data acquisition instruction to the master node.
  • the agent module in the working node A intercepts the second data acquisition instruction and obtains the second data. The instruction is redirected to working node C.
  • the coordination module of working node C sends the target data to working node A.
  • the target data passes through the agent module of working node A and is forwarded to the corresponding Service module.
  • the coordination module of working node C sends the target data to working node B.
  • the target data passes through the agent module of working node B and is forwarded to the corresponding service module.
  • the coordination module of working node C directly sends the target data to the agent module of working node C.
  • the target data passes through the agent module of working node C and is forwarded to the corresponding service. module.
  • obtaining target data from the master node, storing target data, and providing target data to the working nodes in the node pool can all be implemented through the agent module and the coordination module.
  • the specific implementation is as follows.
  • the agent module in the data acquisition working node sends the first data acquisition instruction to the master node
  • the master node sends target data to the agent module in the data acquisition working node in response to the first data acquisition instruction;
  • the agent module in the data acquisition working node receives the target data sent by the master node, and stores the target data to the coordination module in the coordination working node.
  • working nodes include working node A, working node B and working node C.
  • Working node B is a data acquisition working node
  • working node C is a coordinating working node.
  • Working node A, working node B and working node C all include service modules. and agent module, and the coordination module is also included in the worker node C.
  • the agent module of the working node B sends the first data acquisition instruction to the master node. After receiving the first data acquisition instruction, the master node sends the target data to the working node B.
  • the agent module of the working node B sends the target data to the working node. Coordination module of node C.
  • the method further includes:
  • the coordination module receives the target data sent by the agent module in the data acquisition working node, and writes the target data into storage.
  • storage can be storage media such as hard disk and memory.
  • the coordination module of working node C After the coordination module of working node C receives the target data, the coordination module writes the target data to the hard disk.
  • the embodiment of this specification provides a data transmission method, which is applied to a data transmission system.
  • the data transmission system includes a master node, at least two working nodes, and a data acquisition working node and coordination determined from the at least two working nodes.
  • Working node the method includes: the master node responds to the first data acquisition instruction sent by the data acquisition working node, sending target data to the data acquisition working node; the data acquisition working node receives the master node The target data is sent, and the target data is stored in the coordination working node; the coordination working node responds to the second data acquisition instruction sent by other working nodes, and sends the target data to the other nodes, Wherein, the other working nodes are working nodes other than the coordinating working node among the at least two working nodes.
  • the data acquiring working node obtains the target data from the master node and stores it in the coordinating working node.
  • Each working node does not need to obtain data from the master node when acquiring data, reducing the number of master nodes. Communication traffic between nodes and worker nodes.
  • Figure 2 is a flow chart of a communication method between a cloud master node and an edge node provided by an embodiment of this specification.
  • An embodiment of this specification also provides a communication method between a cloud master node and an edge node, which is applied to A data transmission system, which includes a cloud master node, at least two edge nodes, and a data acquisition edge node and a coordination edge node determined from the at least two edge nodes, including:
  • Step 202 The cloud master node responds to the first data acquisition instruction sent by the data acquisition edge node, and sends the target data to the data acquisition edge node;
  • Step 204 The data acquisition edge node receives the target data sent by the cloud master node, and stores the target data in the coordination edge node;
  • Step 206 The coordinating edge node responds to the second data acquisition instruction sent by other edge nodes, and sends the target data to the other edge nodes, wherein the other edge nodes are all of the at least two edge nodes. Describes the coordination of edge nodes other than edge nodes.
  • the at least two edge nodes send lock instructions to the coordinating edge node
  • the coordinating edge node receives the lock instruction sent by the at least two edge nodes, and in the case of receiving the lock instruction, determines the data acquisition edge node according to the lock instruction.
  • the target edge node among the at least two edge nodes obtains demand data from the coordinating edge node, and sends third data to the cloud master node if the demand data does not exist in the coordinating edge node. Get instructions;
  • the cloud master node responds to the third data acquisition instruction and sends the demand data to the target edge node;
  • the target edge node receives the demand data, where the demand data is other data in the cloud master node except the target data.
  • the data acquisition edge node sends a data update instruction to the cloud master node
  • the cloud master node responds to the data update instruction and sends update data to the data acquisition edge node;
  • the data acquisition edge node obtains update data from the cloud master node, and stores the update data to the coordination edge node, where the update data is changed data in the target data.
  • each of the at least two edge nodes includes a service module and a proxy module
  • the coordinating edge node Before the coordinating edge node responds to the second data acquisition instruction sent by other edge nodes, it also includes:
  • the service module sends a second data acquisition instruction to the cloud master node
  • the agent module intercepts the second data acquisition instruction and sends the second data acquisition instruction to the coordination edge node.
  • the coordination edge node further includes the coordination module
  • the coordination module receives the second data acquisition instruction sent by any one of the agent modules, and sends the target data to the agent module;
  • the agent module When receiving the target data, the agent module sends the target data to the corresponding service module.
  • the agent module in the data acquisition edge node sends the first data acquisition instruction to the cloud master node;
  • the cloud master node sends target data to the agent module in the data acquisition edge node in response to the first data acquisition instruction;
  • the agent module in the data acquisition edge node receives the target data sent by the cloud master node, and stores the target data to the coordination module in the coordination edge node.
  • the method further includes:
  • the coordination module receives the target data sent by the agent module in the data acquisition edge node, and writes the target data into storage.
  • the cloud master node can be understood as the master (master node) of Kubernetes in the cloud.
  • Figure 3 is a schematic diagram of a communication method between a cloud master node and an edge node provided by an embodiment of this specification.
  • the cloud includes a cloud master node, the cloud edge includes a node pool, and the node pool includes Edge node 1, edge node 2, edge node 3, edge node 4, edge node 1 is the data acquisition edge node, edge Node 4 is a coordination edge node.
  • Edge node 2, edge node 1 and edge node 4 all include a service module and an agent module.
  • Edge node 4 also includes a coordination module.
  • the agent module of edge node 1 sends a first data acquisition instruction to the master node.
  • the master node After receiving the first data acquisition instruction, the master node sends the target data to edge node 1, and the agent module of edge node 1 sends the target data to the edge.
  • Coordination module for Node 4 The coordination module receives the target data sent by the agent module in the data acquisition edge node, and writes the target data into storage.
  • the service module in edge node 2 When the service module in edge node 2 needs to request data from the master node, it sends a second data acquisition instruction to the master node.
  • the agent module in edge node 2 intercepts the second data acquisition instruction and redirects the second data acquisition instruction.
  • the coordination module of edge node 4 receives the second data acquisition instruction from edge node 2 and sends the target data to edge node 2.
  • the target data passes through the proxy module of edge node 2 and is forwarded to the service of edge node 2. module.
  • Embodiments of this specification provide a communication method between a cloud master node and an edge node.
  • the method is applied to a data transmission system.
  • the data transmission system includes a cloud master node, at least two edge nodes, and a slave node from the at least two edge nodes.
  • the method includes: the cloud master node responds to the first data acquisition instruction sent by the data acquisition edge node, and sends the target data to the data acquisition edge node;
  • the data acquisition edge node receives the target data sent by the cloud master node, and stores the target data to the coordination edge node;
  • the coordination edge node responds to the second data acquisition instruction sent by other edge nodes,
  • the target data is sent to the other nodes, where the other edge nodes are edge nodes other than the coordinating edge node among the at least two edge nodes.
  • the data acquisition edge node acquires the target data from the cloud master node and stores it to the coordinating edge node.
  • Each edge node does not need to acquire data from the cloud master node when acquiring data, reducing The communication traffic between the cloud master node and the edge node is controlled.
  • FIG. 4 shows a schematic structural diagram of a data transmission device provided by an embodiment of this specification. As shown in Figure 4, the device is applied to a data transmission system.
  • the data transmission system includes a main module, at least two sub-modules, and a data acquisition sub-module and a coordination sub-module determined from the at least two sub-modules.
  • the device include:
  • the main module 402 is configured to send target data to the data acquisition sub-module in response to the first data acquisition instruction sent by the data acquisition sub-module;
  • the data acquisition module 404 is configured to receive the target data sent by the main module and store the target data in the coordination sub-module;
  • the coordination module 406 is configured to send the target data to the other sub-modules in response to the second data acquisition instruction sent by the other sub-modules, wherein the other sub-modules are all of the at least two sub-modules. Submodules other than the coordination submodule described above.
  • the at least two sub-modules are configured to send lock instructions to the coordination sub-module;
  • the coordination sub-module is configured to receive the lock instruction sent by the at least two sub-modules. After receiving the lock instruction In the case of , the data acquisition sub-module is determined according to the lock instruction.
  • the target sub-module among the at least two sub-modules is configured to obtain demand data from the coordination sub-module, if the demand data does not exist in the coordination sub-module , sending a third data acquisition instruction to the main module;
  • the main module is configured to send the demand data to the target sub-module in response to the third data acquisition instruction;
  • the target sub-module is configured to receive the demand data, where the demand data is other data in the main module except the target data.
  • the data acquisition sub-module is configured to send a data update instruction to the main module
  • the main module is configured to send update data to the data acquisition sub-module in response to the data update instruction;
  • the data acquisition sub-module is configured to obtain update data from the main module and store the update data in the coordination sub-module, where the update data is changed data in the target data.
  • the agent module in the data acquisition sub-module sends the first data acquisition instruction to the main module
  • the main module is configured to send target data to the agent module in the data acquisition sub-module in response to the first data acquisition instruction;
  • the agent module in the data acquisition sub-module is configured to receive the target data sent by the main module and store the target data to the coordination module in the coordination sub-module.
  • the method further includes:
  • the coordination module is configured to receive the target data sent by the agent module in the data acquisition sub-module and write the target data into storage.
  • each of the at least two sub-modules includes a service module and a proxy module
  • the coordination sub-module is configured to respond to the second data acquisition instruction sent by other sub-modules, and also includes:
  • the service module is configured to send a second data acquisition instruction
  • the agent module is configured to intercept the second data acquisition instruction and send the second data acquisition instruction to the coordination sub-module.
  • the coordination sub-module further includes the coordination module
  • the coordination module is configured to receive the second data acquisition instruction sent by any one of the agent modules, and send the target data to the agent module;
  • the agent module is configured to, upon receiving the target data, send the target data to the corresponding service module.
  • Embodiments of this specification provide a data transmission device, which is applied to a data transmission system.
  • the data transmission system includes a main module, at least two sub-modules, and a data acquisition sub-module and coordination module determined from the at least two sub-modules.
  • the method includes: the main module responds to the first data acquisition instruction sent by the data acquisition sub-module, sending target data to the data acquisition sub-module; the data acquisition sub-module receives the main module The target data is sent, and the target data is stored in the coordination sub-module; the coordination sub-module responds to the second data acquisition instruction sent by other sub-modules, and sends the target data to the other modules, Wherein, the other sub-modules are sub-modules other than the coordination sub-module among the at least two sub-modules.
  • the data acquisition sub-module acquires the target data from the main module and stores it in the coordination sub-module.
  • Each sub-module does not need to obtain data from the main module when acquiring data, which reduces the number of main modules. Communication traffic with submodules.
  • the embodiment of this specification also provides a data transmission system, including a master node, at least two working nodes, and a data acquisition working node and a coordinating working node determined from the at least two working nodes, wherein,
  • the master node is configured to send target data to the data acquisition working node in response to the first data acquisition instruction sent by the data acquisition working node;
  • the data acquisition working node is configured to receive the target data sent by the master node and store the target data in the coordination working node;
  • the coordinating working node is configured to send the target data to the other working nodes in response to the second data acquisition instruction sent by the other working nodes, wherein the other working nodes are all except the at least two working nodes.
  • the at least two working nodes are configured to send lock instructions to the coordinating working node
  • the coordinating working node is configured to receive a lock instruction sent by the at least two working nodes, and when receiving the lock instruction, determine the data acquisition working node according to the lock instruction.
  • the target working node among the at least two working nodes is configured to obtain demand data from the coordinating working node, when the demand data does not exist in the coordinating working node.
  • the master node is configured to send the demand data to the target working node in response to the third data acquisition instruction;
  • the target working node is configured to receive the demand data, where the demand data is other data in the master node except the target data.
  • the data acquisition working node is configured to send a data update instruction to the master node
  • the master node is configured to send update data to the data acquisition working node in response to the data update instruction;
  • the data acquisition working node is configured to obtain updated data from the master node and store the updated data to the coordination working node, where the updated data is changed data in the target data.
  • the agent module in the data acquisition working node sends the first data acquisition instruction to the master node
  • the master node is configured to send target data to the agent module in the data acquisition working node in response to the first data acquisition instruction;
  • the agent module in the data acquisition working node is configured to receive the target data sent by the master node and store the target data in the coordination module in the coordination working node.
  • the method further includes:
  • the coordination module is configured to receive the target data sent by the agent module in the data acquisition working node, and write the target data into storage.
  • each of the at least two working nodes includes a service module and an agent module
  • the coordinating working node before being configured to respond to the second data acquisition instruction sent by other working nodes, also includes:
  • the service module is configured to send a second data acquisition instruction
  • the agent module is configured to intercept the second data acquisition instruction and send the second data acquisition instruction to the coordination working node.
  • the coordination working node further includes the coordination module
  • the coordination module is configured to receive the second data acquisition instruction sent by any one of the agent modules, and send the target data to the agent module;
  • the agent module is configured to, upon receiving the target data, send the target data to the corresponding service module.
  • the embodiment of this specification provides a data transmission system, including a master node, at least two working nodes, and a data acquisition working node and a coordinating working node determined from the at least two working nodes, wherein the master node is Configured to respond to the first data acquisition instruction sent by the data acquisition working node, sending target data to the data acquisition working node; the data acquisition working node is configured to receive the target data sent by the master node , and store the target data to the coordination working node; the coordination working node is configured to send the target data to the other nodes in response to the second data acquisition instruction sent by the other working nodes, wherein, The other working nodes are working nodes other than the coordinating working node among the at least two working nodes.
  • the node Determine a data acquisition working node, and the data acquisition working node acquires the target data from the master node and stores it to the coordinating working node.
  • Each working node does not need to obtain data from the master node when acquiring data, which reduces the number of master nodes and working nodes. communication traffic.
  • the above is a schematic solution of a data transmission device in this embodiment. It should be noted that the technical solution of the data transmission device and the technical solution of the above-mentioned data transmission method belong to the same concept. For details that are not described in detail in the technical solution of the data transmission device, please refer to the description of the technical solution of the above-mentioned data transmission method. .
  • Figure 5 shows a structural block diagram of a computing device 500 provided according to an embodiment of this specification.
  • Components of the computing device 500 include, but are not limited to, memory 510 and processor 520 .
  • the processor 520 is connected to the memory 510 through a bus 530, and the database 550 is used to save data.
  • Computing device 500 also includes an access device 540 that enables computing device 500 to communicate via one or more networks 560 .
  • networks include the Public Switched Telephone Network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communications networks such as the Internet.
  • Access device 540 may include one or more of any type of network interface (eg, a network interface card (NIC)), wired or wireless, such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, Global Interconnection for Microwave Access ( Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, etc.
  • NIC network interface card
  • the above-mentioned components of the computing device 500 and other components not shown in FIG. 5 may also be connected to each other, such as through a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 5 is for illustrative purposes only and does not limit the scope of this description. Those skilled in the art can add or replace other components as needed.
  • Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet computer, personal digital assistant, laptop computer, notebook computer, netbook, etc.), a mobile telephone (e.g., smartphone ), a wearable computing device (e.g., smart watch, smart glasses, etc.) or other type of mobile device, or a stationary computing device such as a desktop computer or PC.
  • a mobile computer or mobile computing device e.g., tablet computer, personal digital assistant, laptop computer, notebook computer, netbook, etc.
  • a mobile telephone e.g., smartphone
  • a wearable computing device e.g., smart watch, smart glasses, etc.
  • stationary computing device such as a desktop computer or PC.
  • Computing device 500 may also be a mobile or stationary server.
  • the processor 520 is configured to execute the following computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above data transmission method and the communication method between the cloud master node and the edge node are implemented.
  • the above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned data transmission method belong to the same concept. For details that are not described in detail in the technical solution of the computing device, please refer to the above-mentioned data transmission method and the cloud master node and edge node. Description of technical solutions for communication methods.
  • An embodiment of the present specification also provides a computer-readable storage medium that stores computer-executable instructions.
  • the computer-executable instructions are executed by a processor, the steps of the above data transmission method are implemented.
  • the above is a schematic solution of a computer-readable storage medium in this embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above-mentioned data transmission method, and the technical solution of the storage medium is not specified. For detailed description, please refer to the description of the technical solution of the above data transmission method and the communication method between the cloud master node and the edge node.
  • An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to perform the steps of the above-mentioned data transmission method and the communication method between the cloud master node and the edge node.
  • the computer instructions include computer program code, which may be in the form of source code, object code, executable file or some intermediate form.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electrical carrier signals telecommunications signals
  • software distribution media etc.
  • the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of legislation and patent practice in the jurisdiction.
  • the computer-readable medium Excludes electrical carrier signals and telecommunications signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本说明书实施例提供一种数据传输方法以及系统，该方法应用于数据传输系统，数据传输系统包括主节点、至少两个工作节点、以及从至少两个工作节点中确定的数据获取工作节点和协调工作节点，方法包括：主节点响应于数据获取工作节点发送的第一数据获取指令，向数据获取工作节点发送目标数据；数据获取工作节点接收主节点发送的目标数据，并将目标数据存储至协调工作节点；协调工作节点响应于其他工作节点发送的第二数据获取指令，向其他节点发送目标数据。通过由数据获取工作节点从主节点获取目标数据并存储至协调工作节点，每个工作节点在获取数据时不用向主节点进行数据获取，减少了主节点与工作节点的通信流量。

Description

数据传输方法以及系统
本申请要求于2022年04月06日提交中国专利局、申请号为202210356688.1、申请名称为“数据传输方法以及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本说明书实施例涉及数据处理技术领域，特别涉及一种数据传输方法。
背景技术
Kubernetes系统在边缘计算场景越来越流行，在云管边架构中，kubernetes master部署在云端，kubernetes worker以及项目pods部署在边缘环境(离用户现场比较近)，K8s的worker和master通过公网连接。并且由于边缘节点一般没有配置公网地址，因此K8s master无法直接访问worker。当云管边的Kubernetes集群超过一定规模时(如：节点>10k,pod>200k,service>10k)时，当边缘项目Pod删除重建时，会触发所有边缘节点针对endpoints/endpointslices资源的请求，从而导致云边通信流量大幅度增加。同时边缘节点的上下线，也会触发所有边缘节点对Node资源的请求，也会导致云边流量大幅提升。云边流量的瞬时大幅度提升，甚至有可能达到云端流量出口(SLB)的流量限制，造成边缘请求的超过和出错。因此如何减少云边的公网通信流量，成为一个急需解决的问题。
发明内容
有鉴于此，本说明书实施例提供了一种数据传输方法。本说明书一个或者多个实施例同时涉及一种云端主节点与边缘节点的通信方法，一种数据传输装置，一种计算设备，一种计算机可读存储介质以及一种计算机程序，以解决现有技术中存在的技术缺陷。
根据本说明书实施例的第一方面，提供了一种数据传输方法，所述数据传输系统包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，所述方法包括：
所述主节点响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；
所述数据获取工作节点接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；
所述协调工作节点响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
根据本说明书实施例的第二方面，提供了一种云端主节点与边缘节点的通信方法，应用于数据传输系统，所述数据传输系统包括云端主节点、至少两个边缘节点、以及从所述至少两个边缘节点中确定的数据获取边缘节点和协调边缘节点，包括：
所述云端主节点响应于所述数据获取边缘节点发送的第一数据获取指令，向所述数据获取边缘节点发送目标数据；
所述数据获取边缘节点接收所述云端主节点发送的所述目标数据，并将所述目标数据存储至所述协调边缘节点；
所述协调边缘节点响应于其他边缘节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他边缘节点为所述至少两个边缘节点中除所述协调边缘节点之外的边缘节点。
根据本说明书实施例的第三方面，提供了一种数据传输系统，包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，其中，
所述主节点，被配置为响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；
所述数据获取工作节点，被配置为接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；
所述协调工作节点，被配置为响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
根据本说明书实施例的第四方面，提供了一种数据传输装置，应用于数据传输系统，所述数据传输系统包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，所述装置包括：
所述主模块，被配置为响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；
所述数据获取模块，被配置为接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；
所述协调模块，被配置为响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
根据本说明书实施例的第五方面，提供了一种计算设备，包括：
存储器和处理器；
所述存储器用于存储计算机可执行指令，所述处理器用于执行所述计算机可执行指令， 该计算机可执行指令被处理器执行时实现上述数据传输方法的步骤。
根据本说明书实施例的第六方面，提供了一种计算机可读存储介质，其存储有计算机可执行指令，该指令被处理器执行时实现上述数据传输方法的步骤。
根据本说明书实施例的第七方面，提供了一种计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行上述数据传输方法的步骤。
本说明书实施例提供一种数据传输方法以及系统，该方法应用于数据传输系统，所述数据传输系统包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，所述方法包括：所述主节点响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；所述数据获取工作节点接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；所述协调工作节点响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。通过在工作节点中确定一个数据获取工作节点，由数据获取工作节点从主节点获取目标数据并存储至协调工作节点，每个工作节点在获取数据时不用都向主节点进行数据获取，减少了主节点与工作节点的通信流量。
附图说明
图1是本说明书一个实施例提供的一种数据传输方法的流程图；
图2是本说明书一个实施例提供的一种云端主节点与边缘节点的通信方法的流程图；
图3是本说明书一个实施例提供的一种云端主节点与边缘节点的通信方法的示意图；
图4是本说明书一个实施例提供的一种数据传输装置的结构示意图；
图5是本说明书一个实施例提供的一种计算设备的结构框图。
具体实施方式
在下面的描述中阐述了很多具体细节以便于充分理解本说明书。但是本说明书能够以很多不同于在此描述的其它方式来实施，本领域技术人员可以在不违背本说明书内涵的情况下做类似推广，因此本说明书不受下面公开的具体实施的限制。
在本说明书一个或多个实施例中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本说明书一个或多个实施例。在本说明书一个或多个实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本说明书一个或多个实施例中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本说明书一个或多个实施例中可能采用术语第一、第二等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本说明书一个或多个实施例范围的情况下，第一也可以被称为第二，类似 地，第二也可以被称为第一。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
首先，对本说明书一个或多个实施例涉及的名词术语进行解释。
Kubernetes：用于自动部署、扩展和管理容器化应用程序的开源系统。
list/watch：Kubernetes中的数据访问机制。
Kubelet：是工作节点上的主要服务，定期从kube-apiserver组件接收新的或修改的Pod规范，并确保Pod及其容器在期望规范下运行。
Kube-proxy：维护节点上的网络规则，实现了Kubernetes Service概念的一部分。它的作用是使发往Service的流量(通过ClusterIP和端口)负载均衡到正确的后端Pod。
YurtHub：是对客户端Request的反向代理。
传输控制协议(TCP，Transmission Control Protocol)：是一种面向连接的、可靠的、基于字节流的传输层通信协议。
Lease：Kubernetes节点发送的心跳以确定每个节点的可用性，是一种轻量级资源。
Pod：Pod是可以在Kubernetes中创建和管理的、最小的可部署的计算单元，其中包含一个或多个应用容器。
节点池(NodePool)：Kubernetes集群以及其扩展集群中，具备同一属性特征(如部署在同一物理区域且可互相访问)的多个节点。
Pool Scope Data：NodePool中的各个节点从云端获取的数据是相同的。如Node，Endpointslices，Endpoints，Service等资源的list请求返回的数据。
Node Scope Data:NodePool中的各个节点从云端获取的数据是本节点独有的。如Pods，Secrets，Configmaps等资源的get请求返回的数据。
边缘节点：指在靠近用户的网络边缘侧构建的项目平台，提供存储、计算、网络等资源，将部分关键项目应用下沉到接入网络边缘，以减少网络传输和多级转发带来的宽度和时延损耗。边缘节点位置介于用户和云中心之间。
选举：是一种决策过程。
为了防止在云边(云端与边缘)网络长时间断连后再恢复时，可能触发边缘的list/watch请求，从而云边流量大幅度提升。在一些云管边架构中，为了减少云边流量，摒弃了Kubernetes中原生的list/watch请求，改造为云端主动往边缘推送数据。具体方案如下所述。
首先边缘节点接入云端集群时，边缘节点和云端集群构建一条tcp长连接，同时云端集群增加一个控制器(controller)，实时监测(list/watch)云端集群的数据变化，当控制器监测到数据变化时，通过云边集群的tcp长连接，把变化数据推送到各个边缘节点。通过云端的数据主动推送，从而减少边缘的主动list操作，达到云边通信数据的消减。
该方案虽然可以减少部分场景(云边网络长时间断连再恢复)下list请求的流量，但是存在不足的地方，如，list/watch机制是Kubernetes系统中组件间通信的底层机制，摒弃了list/watch机制，意味对Kubernetes系统有破坏性的侵入式修改，导致边缘节点上使用list/watch机制的云原生生态的应用都无法运行。并且大部分场景下，当云端的资源(如endpoints)变化时，云端还是需要向每个边缘节点推送数据，因此大部分场景下云边流量并不会减少，只是部分场景下可以减少流量，云边通信数据的消减的效果不好。又因为是云端主动往边缘节点推送数据，意味边缘节点需要什么类型数据，云端并不知情，所以云端需要往边缘节点推送所有类型数据。因此会产生很多无效数据的推送，由此带来云边流量的浪费。
基于此，在本说明书中，提供了一种数据传输方法，本说明书同时涉及一种云端主节点与边缘节点的通信方法，一种数据传输装置，一种计算设备，以及一种计算机可读存储介质，在下面的实施例中逐一进行详细说明。
参见图1，图1示出了根据本说明书一个实施例提供的一种数据传输方法的流程图，应用于数据传输系统，所述数据传输系统包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，其中，本实施例中的主节点可以理解为前述的云端，本实施例中的工作节点可以理解为前述的边缘节点，且其中的协调工作节点为预先配置好的，该数据传输方法包括具体包括以下步骤。
步骤102：所述主节点响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据。
其中，数据获取工作节点可以为任何一个工作节点，该节点与主节点之间存在通信连接；第一数据获取指令可以为Kubernetes中的list/watch请求；目标数据可以为名词解释中的Pool Scope Data，如，Pods，Secrets，Configmaps等资源的get请求返回的数据。
在实际应用中，所有工作节点属于同一个节点池(NodePool)，节点池内的各个节点可以相互传输数据，节点池中的数据获取工作节点向主节点发送第一数据获取指令，主节点在收到第一数据获取指令的，向数据获取工作节点发送相应的数据。
例如，工作节点包括工作节点A和工作节点B，工作节点B为数据获取工作节点，则工作节点B向主节点发送第一数据获取指令，主节点在接收到第一数据获取指令的情况下，向工作节点B发送目标数据。
本说明书实施例通过一个节点获取目标数据，以使后续可以存储目标数据，使其他节点从节点池内获取目标数据，从而不向主节点请求目标数据，减少了通信流量。
步骤104：所述数据获取工作节点接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点。
在实际应用中，数据获取工作节点在接收到目标数据之后，将目标数据存储在预先配 置好的协调工作节点，以便其他工作节点从协调工作节点中获取数据。
例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点B为数据获取工作节点，工作节点C为协调工作节点，则工作节点B向主节点发送第一数据获取指令，主节点在接收到第一数据获取指令的情况下，向工作节点B发送目标数据，工作节点B将目标数据存储至工作节点C。
在一种可实现的方式中，数据获取工作节点可以通过选举的方式得到，具体实现方式如下所述。
所述至少两个工作节点向所述协调工作节点发送锁指令；
所述协调工作节点接收所述至少两个工作节点发送的锁指令，在接收到锁指令的情况下，根据所述锁指令确定所述数据获取工作节点。
其中，锁指令可以理解为对数据进行修改以表示锁定的指令，例如，在数据X初始值为0的情况下，将数据X的值修改为1，则表示数据X的值已被锁定，且不可更改，该锁定机制可以使用本领域的常规技术手段，本说明书实施例不进行限定。
例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点C为协调工作节点，则工作节点A、工作节点B向工作节点C发送锁指令，若工作节点B的锁指令先到达工作节点C，且修改了锁机制的数据值，将0变为1，并且工作节点B向主节点发送连接请求，工作节点A的锁指令在到达工作节点C的情况下，发现锁机制的数据值已经为1，则可知已经选举成功，并可以获取到工作节点B为数据获取节点。
需要说明的是，为了保证工作节点和云端的网络是连通的，工作节点参加选举的必要条件是和云端网络连通。
本说明书实施例通过选举的方式确定数据获取工作节点，可以确定通信延迟较小的工作节点为数据获取工作节点，提高了通信效率。
在一种可实现的方式中，在每个工作节点中存在服务模块的基础上，在工作节点中添加代理模块，代理模块用于拦截服务模块的发出的数据获取请求，以实现将各工作节点中的数据请求转发至协调工作节点，具体实现方式如下所述。
具体地，所述至少两个工作节点中每个工作节点均包括服务模块和代理模块；
所述协调工作节点响应于其他工作节点发送的第二数据获取指令之前，还包括：
所述服务模块向所述主节点发送第二数据获取指令；
所述代理模块拦截所述第二数据获取指令，并将所述第二数据获取指令发送至所述协调工作节点。
其中，服务模块可以为Kubelet、Kube-proxy等；代理模块可以为YurtHub等。
在实际应用中，服务模块本应该将第二数据获取指令发送至主节点，向主节点请求目标数据，那么可以在各个工作节点中加入代理模块，代理模块将所在的工作节点中的服务 模块本应该向主节点发送的第二数据获取指令拦截下来，并将该第二数据获取指令发送至协调工作节点。
例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点A、工作节点B和工作节点C中均包括服务模块和代理模块，工作节点C为协调工作节点，工作节点B为数据获取节点，工作节点A中的服务模块在需要向主节点请求数据的情况下，向主节点发送第二数据获取指令，工作节点A中代理模块拦截第二数据获取指令，并将第二数据获取指令重定向至工作节点C，以获取目标数据。相应地，工作节点B中的服务模块在需要向主节点请求数据的情况下，向主节点发送第二数据获取指令，工作节点B中代理模块拦截第二数据获取指令，并将第二数据获取指令重定向至工作节点C。工作节点C中的服务模块在需要向主节点请求数据的情况下，向主节点发送第二数据获取指令，工作节点C中代理模块拦截第二数据获取指令，并将第二数据获取指令重定向至本地地址。
本说明书实施例在原生Kubernetes的架构中，边缘节点上增加一个代理模块，用来接管工作节点和主节点之间的通信流量，对Kubernetes的架构未做任何修改，云原生生态的应用可以很好的运行在边缘节点，并且所有请求由数据需求方(边缘节点)自行发起，云边通信不会产生无效的流量。
相应地，上述实施例中的数据获取工作节点可以通过选举的方式得到和向主节点发送第一数据获取指令可以为代理模块执行。例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点C为协调工作节点，则工作节点A的代理模块、工作节点B的代理模块向工作节点C发送锁指令，若工作节点B的代理模块的锁指令先到达工作节点C，且修改了锁机制的数据值，将0变为1，并且工作节点B的代理模块向主节点发送连接请求，工作节点A的代理模块的锁指令在到达工作节点C的情况下，发现锁机制的数据值已经为1，则可知已经选举成功，并可以获取到工作节点B为数据获取节点。
在一种可实现的方式中，工作节点除了目标数据之外，会根据自己的内部的服务模块的需求获取数据，那么就需要直接从主节点获取。具体实现方式如下所述。
所述至少两个工作节点中的目标工作节点，从所述协调工作节点获取需求数据，在所述协调工作节点中不存在所述需求数据的情况下，向所述主节点发送第三数据获取指令；
所述主节点响应于所述第三数据获取指令，向所述目标工作节点发送所述需求数据；
所述目标工作节点接收所述需求数据，其中，所述需求数据为所述主节点中除所述目标数据之外的其他数据。
其中，需求数据可以为Node Scope Data，例如Pods，Secrets，Configmaps等资源的get请求返回的数据；第三数据获取指令可以为Kubernetes中的list/watch请求。
例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点A、工作节点B和工作节点C中均包括服务模块和代理模块，工作节点C为协调工作节点，工作节点B为数据获取节点，工作节点A中的服务模块在需要向主节点请求数据的情况下，向主节点发 送第三数据获取指令，工作节点A中代理模块拦截第三数据获取指令，并将第三数据获取指令重定向至工作节点C，以获取相应的数据，工作节点C发现未存储第三数据获取指令相应的数据，则通知工作节点A，工作节点A直接向主节点发送第三数据获取指令，主节点向工作节点A返回第三数据获取指令相应的数据。
本说明书实施例在工作节点有单独的需求的情况下，该工作节点直接向主节点请求数据，不占用协调节点的存储空间。
在一种可实现的方式中，目标数据可能存在变化的情况，或者工作节点需要主动更新数据，那么就需要向主节点请求更新目标数据，具体实施方式如下所述。
所述数据获取工作节点向所述主节点发送数据更新指令；
所述主节点响应于所述数据更新指令，向所述数据获取工作节点发送更新数据；
所述数据获取工作节点从所述主节点获取更新数据，并将所述更新数据存储至所述协调工作节点，其中，所述更新数据为所述目标数据中变更的数据。
例如，工作节点C为协调工作节点，工作节点B为数据获取节点，工作节点B被设置为每隔24小时向主节点发送数据更新指令，主节点响应于数据更新指令，向工作节点B发送目标数据的更新数据，在工作节点B接收到更新数据之后，向工作节点C发送更新数据，工作节点C根据更新数据修改目标数据。
本说明书实施例根据更新数据对目标数据进行修改，而不是传输整个目标数据，减少了通信流量。
步骤106所述协调工作节点响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
其中，第二数据获取指令可以为Kubernetes中的list/watch请求。
在实际应用中，各个工作节点对目标数据的数据请求经过各个代理模块时，代理模块把请求转发到协调工作节点，从其中获取到写入的最新的目标数据。从而增加云边流量复用的效果，减少云边公网通信流量。
在一种可实现的方式中，在每个工作节点中存在服务模块和代理模块的基础上，在工作节点中添加协调模块，以配置该工作节点为协调工作节点，协调模块用于管理目标数据。具体实施方式如下所述。
所述协调工作节点还包括所述协调模块；
所述协调模块接收任意一个所述代理模块发送的第二数据获取指令，并向所述代理模块发送所述目标数据；
所述代理模块在接收到所述目标数据的情况下，将所述目标数据发送至对应的服务模块。
其中，协调模块可以理解为管理目标数据的模块，例如，节点池协调器(pool-assistant)，需要说明的是，节点池协调器可以部署在任何一个工作节点上，本说明书实施例不进行限定。
例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点A、工作节点B和工作节点C中均包括服务模块和代理模块，工作节点C为协调工作节点，工作节点B为数据获取节点，工作节点A中的服务模块在需要向主节点请求数据的情况下，向主节点发送第二数据获取指令，工作节点A中代理模块拦截第二数据获取指令，并将第二数据获取指令重定向至工作节点C，工作节点C的协调模块在接收到工作节点A的二数据获取指令，向工作节点A发送目标数据，该目标数据经过工作节点A的代理模块，并转发至相应的服务模块。相应地，工作节点C的协调模块在接收到工作节点B的二数据获取指令，向工作节点B发送目标数据，该目标数据经过工作节点B的代理模块，并转发至相应的服务模块。工作节点C的协调模块在接收到工作节点C的代理模块的二数据获取指令，直接向工作节点C的代理模块发送目标数据，该目标数据经过工作节点C的代理模块，并转发至相应的服务模块。
本说明书实施例对Kubernetes未做任何修改，云原生生态的应用可以很好的运行在边缘，并且共享到流量复用的能力。
在一种可实现的方式中，从主节点获取目标数据、存储目标数据以及向节点池内的工作节点提供目标数据均可以通过代理模块和协调模块实现，具体实施方式如下所述。
所述数据获取工作节点中的代理模块向所述主节点发送所述第一数据获取指令；
所述主节点向所述响应于所述第一数据获取指令，向所述数据获取工作节点中的代理模块发送目标数据；
所述数据获取工作节点中的代理模块接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点中的协调模块。
例如，工作节点包括工作节点A、工作节点B和工作节点C，工作节点B为数据获取工作节点，工作节点C为协调工作节点，工作节点A、工作节点B和工作节点C中均包括服务模块和代理模块，工作节点C中还包括协调模块。工作节点B的代理模块向主节点发送第一数据获取指令，主节点在接收到第一数据获取指令的情况下，向工作节点B发送目标数据，工作节点B的代理模块将目标数据发送至工作节点C的协调模块。
进一步地，所述将所述目标数据存储至所述协调工作节点中的协调模块之后，还包括：
所述协调模块接收所述数据获取工作节点中的代理模块发送的所述目标数据，并将所述目标数据写入存储中。
其中，存储可以为硬盘，内存等存储介质。
沿用上例，工作节点C的协调模块在接收到目标数据之后，协调模块将目标数据写入到硬盘中。
需要说明的是，即使在工作节点断网的情况下，所有节点池内的工作节点依然能够从协调节点中获取目标数据。
本说明书实施例提供一种数据传输方法，应用于数据传输系统，所述数据传输系统包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，所述方法包括：所述主节点响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；所述数据获取工作节点接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；所述协调工作节点响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。通过在工作节点中确定一个数据获取工作节点，由数据获取工作节点从主节点获取目标数据并存储至协调工作节点，每个工作节点在获取数据时不用都向主节点进行数据获取，减少了主节点与工作节点的通信流量。
参见图2，图2是本说明书一个实施例提供的一种云端主节点与边缘节点的通信方法的流程图，本说明书实施例还提供了一种云端主节点与边缘节点的通信方法，应用于数据传输系统，所述数据传输系统包括云端主节点、至少两个边缘节点、以及从所述至少两个边缘节点中确定的数据获取边缘节点和协调边缘节点，包括：
步骤202：所述云端主节点响应于所述数据获取边缘节点发送的第一数据获取指令，向所述数据获取边缘节点发送目标数据；
步骤204：所述数据获取边缘节点接收所述云端主节点发送的所述目标数据，并将所述目标数据存储至所述协调边缘节点；
步骤206：所述协调边缘节点响应于其他边缘节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他边缘节点为所述至少两个边缘节点中除所述协调边缘节点之外的边缘节点。
在一种可实现的方式中，还包括：
所述至少两个边缘节点向所述协调边缘节点发送锁指令；
所述协调边缘节点接收所述至少两个边缘节点发送的锁指令，在接收到锁指令的情况下，根据所述锁指令确定所述数据获取边缘节点。
在一种可实现的方式中，还包括：
所述至少两个边缘节点中的目标边缘节点，从所述协调边缘节点获取需求数据，在所述协调边缘节点中不存在所述需求数据的情况下，向所述云端主节点发送第三数据获取指令；
所述云端主节点响应于所述第三数据获取指令，向所述目标边缘节点发送所述需求数据；
所述目标边缘节点接收所述需求数据，其中，所述需求数据为所述云端主节点中除所述目标数据之外的其他数据。
在一种可实现的方式中，还包括：
所述数据获取边缘节点向所述云端主节点发送数据更新指令；
所述云端主节点响应于所述数据更新指令，向所述数据获取边缘节点发送更新数据；
所述数据获取边缘节点从所述云端主节点获取更新数据，并将所述更新数据存储至所述协调边缘节点，其中，所述更新数据为所述目标数据中变更的数据。
在一种可实现的方式中，所述至少两个边缘节点中每个边缘节点均包括服务模块和代理模块；
所述协调边缘节点响应于其他边缘节点发送的第二数据获取指令之前，还包括：
所述服务模块向所述云端主节点发送第二数据获取指令；
所述代理模块拦截所述第二数据获取指令，并将所述第二数据获取指令发送至所述协调边缘节点。
在一种可实现的方式中，所述协调边缘节点还包括所述协调模块；
所述协调模块接收任意一个所述代理模块发送的第二数据获取指令，并向所述代理模块发送所述目标数据；
所述代理模块在接收到所述目标数据的情况下，将所述目标数据发送至对应的服务模块。
在一种可实现的方式中，还包括：
所述数据获取边缘节点中的代理模块向所述云端主节点发送所述第一数据获取指令；
所述云端主节点向所述响应于所述第一数据获取指令，向所述数据获取边缘节点中的代理模块发送目标数据；
所述数据获取边缘节点中的代理模块接收所述云端主节点发送的所述目标数据，并将所述目标数据存储至所述协调边缘节点中的协调模块。
在一种可实现的方式中，所述将所述目标数据存储至所述协调边缘节点中的协调模块之后，还包括：
所述协调模块接收所述数据获取边缘节点中的代理模块发送的所述目标数据，并将所述目标数据写入存储中。
其中，云端主节点可以理解为在云端的Kubernetes的master(主节点)。
举例来说，参见图3，图3是本说明书一个实施例提供的一种云端主节点与边缘节点的通信方法的示意图，其中，在云端包括云端主节点，云边包括节点池，节点池内包括边缘节点1、边缘节点2、边缘节点3、边缘节点4，边缘节点1为数据获取边缘节点，边缘 节点4为协调边缘节点，边缘节点2、边缘节点1和边缘节点4中均包括服务模块和代理模块，边缘节点4中还包括协调模块。边缘节点1的代理模块向主节点发送第一数据获取指令，主节点在接收到第一数据获取指令的情况下，向边缘节点1发送目标数据，边缘节点1的代理模块将目标数据发送至边缘节点4的协调模块。所述协调模块接收所述数据获取边缘节点中的代理模块发送的所述目标数据，并将所述目标数据写入存储中。
边缘节点2中的服务模块在需要向主节点请求数据的情况下，向主节点发送第二数据获取指令，边缘节点2中代理模块拦截第二数据获取指令，并将第二数据获取指令重定向至边缘节点4，边缘节点4的协调模块在接收到边缘节点2的二数据获取指令，向边缘节点2发送目标数据，该目标数据经过边缘节点2的代理模块，并转发至边缘节点2的服务模块。
本说明书实施例提供一种云端主节点与边缘节点的通信方法，该方法应用于数据传输系统，所述数据传输系统包括云端主节点、至少两个边缘节点、以及从所述至少两个边缘节点中确定的数据获取边缘节点和协调边缘节点，所述方法包括：所述云端主节点响应于所述数据获取边缘节点发送的第一数据获取指令，向所述数据获取边缘节点发送目标数据；所述数据获取边缘节点接收所述云端主节点发送的所述目标数据，并将所述目标数据存储至所述协调边缘节点；所述协调边缘节点响应于其他边缘节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他边缘节点为所述至少两个边缘节点中除所述协调边缘节点之外的边缘节点。通过在边缘节点中确定一个数据获取边缘节点，由数据获取边缘节点从云端主节点获取目标数据并存储至协调边缘节点，每个边缘节点在获取数据时不用都向云端主节点进行数据获取，减少了云端主节点与边缘节点的通信流量。
与上述方法实施例相对应，本说明书还提供了数据传输装置实施例，图4示出了本说明书一个实施例提供的一种数据传输装置的结构示意图。如图4所示，该装置应用于数据传输系统，所述数据传输系统包括主模块、至少两个子模块、以及从所述至少两个子模块中确定的数据获取子模块和协调子模块，该装置包括：
所述主模块402，被配置为响应于所述数据获取子模块发送的第一数据获取指令，向所述数据获取子模块发送目标数据；
所述数据获取模块404，被配置为接收所述主模块发送的所述目标数据，并将所述目标数据存储至所述协调子模块；
所述协调模块406，被配置为响应于其他子模块发送的第二数据获取指令，向所述其他模块发送所述目标数据，其中，所述其他子模块为所述至少两个子模块中除所述协调子模块之外的子模块。
在一种可实现的方式中，所述至少两个子模块，被配置为向所述协调子模块发送锁指令；
所述协调子模块，被配置为接收所述至少两个子模块发送的锁指令，在接收到锁指令 的情况下，根据所述锁指令确定所述数据获取子模块。
在一种可实现的方式中，所述至少两个子模块中的目标子模块，被配置为从所述协调子模块获取需求数据，在所述协调子模块中不存在所述需求数据的情况下，向所述主模块发送第三数据获取指令；
所述主模块，被配置为响应于所述第三数据获取指令，向所述目标子模块发送所述需求数据；
所述目标子模块，被配置为接收所述需求数据，其中，所述需求数据为所述主模块中除所述目标数据之外的其他数据。
在一种可实现的方式中，所述数据获取子模块，被配置为向所述主模块发送数据更新指令；
所述主模块，被配置为响应于所述数据更新指令，向所述数据获取子模块发送更新数据；
所述数据获取子模块，被配置为从所述主模块获取更新数据，并将所述更新数据存储至所述协调子模块，其中，所述更新数据为所述目标数据中变更的数据。
在一种可实现的方式中，所述数据获取子模块中的代理模块向所述主模块发送所述第一数据获取指令；
所述主模块，被配置为向所述响应于所述第一数据获取指令，向所述数据获取子模块中的代理模块发送目标数据；
所述数据获取子模块中的代理模块，被配置为接收所述主模块发送的所述目标数据，并将所述目标数据存储至所述协调子模块中的协调模块。
在一种可实现的方式中，所述将所述目标数据存储至所述协调子模块中的协调模块之后，还包括：
所述协调模块，被配置为接收所述数据获取子模块中的代理模块发送的所述目标数据，并将所述目标数据写入存储中。
在一种可实现的方式中，所述至少两个子模块中每个子模块均包括服务模块和代理模块；
所述协调子模块，被配置为响应于其他子模块发送的第二数据获取指令之前，还包括：
所述服务模块，被配置为发送第二数据获取指令；
所述代理模块，被配置为拦截所述第二数据获取指令，并将所述第二数据获取指令发送至所述协调子模块。
在一种可实现的方式中，所述协调子模块还包括所述协调模块；
所述协调模块，被配置为接收任意一个所述代理模块发送的第二数据获取指令，并向所述代理模块发送所述目标数据；
所述代理模块，被配置为在接收到所述目标数据的情况下，将所述目标数据发送至对应的服务模块。
本说明书实施例提供一种数据传输装置，该装置应用于数据传输系统，所述数据传输系统包括主模块、至少两个子模块、以及从所述至少两个子模块中确定的数据获取子模块和协调子模块，所述方法包括：所述主模块响应于所述数据获取子模块发送的第一数据获取指令，向所述数据获取子模块发送目标数据；所述数据获取子模块接收所述主模块发送的所述目标数据，并将所述目标数据存储至所述协调子模块；所述协调子模块响应于其他子模块发送的第二数据获取指令，向所述其他模块发送所述目标数据，其中，所述其他子模块为所述至少两个子模块中除所述协调子模块之外的子模块。通过在子模块中确定一个数据获取子模块，由数据获取子模块从主模块获取目标数据并存储至协调子模块，每个子模块在获取数据时不用都向主模块进行数据获取，减少了主模块与子模块的通信流量。
本说明书实施例还提供了一种数据传输系统，包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，其中，
所述主节点，被配置为响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；
所述数据获取工作节点，被配置为接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；
所述协调工作节点，被配置为响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
在一种可实现的方式中，所述至少两个工作节点，被配置为向所述协调工作节点发送锁指令；
所述协调工作节点，被配置为接收所述至少两个工作节点发送的锁指令，在接收到锁指令的情况下，根据所述锁指令确定所述数据获取工作节点。
在一种可实现的方式中，所述至少两个工作节点中的目标工作节点，被配置为从所述协调工作节点获取需求数据，在所述协调工作节点中不存在所述需求数据的情况下，向所述主节点发送第三数据获取指令；
所述主节点，被配置为响应于所述第三数据获取指令，向所述目标工作节点发送所述需求数据；
所述目标工作节点，被配置为接收所述需求数据，其中，所述需求数据为所述主节点中除所述目标数据之外的其他数据。
在一种可实现的方式中，所述数据获取工作节点，被配置为向所述主节点发送数据更新指令；
所述主节点，被配置为响应于所述数据更新指令，向所述数据获取工作节点发送更新数据；
所述数据获取工作节点，被配置为从所述主节点获取更新数据，并将所述更新数据存储至所述协调工作节点，其中，所述更新数据为所述目标数据中变更的数据。
在一种可实现的方式中，所述数据获取工作节点中的代理模块向所述主节点发送所述第一数据获取指令；
所述主节点，被配置为向所述响应于所述第一数据获取指令，向所述数据获取工作节点中的代理模块发送目标数据；
所述数据获取工作节点中的代理模块，被配置为接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点中的协调模块。
在一种可实现的方式中，所述将所述目标数据存储至所述协调工作节点中的协调模块之后，还包括：
所述协调模块，被配置为接收所述数据获取工作节点中的代理模块发送的所述目标数据，并将所述目标数据写入存储中。
在一种可实现的方式中，所述至少两个工作节点中每个工作节点均包括服务模块和代理模块；
所述协调工作节点，被配置为响应于其他工作节点发送的第二数据获取指令之前，还包括：
所述服务模块，被配置为发送第二数据获取指令；
所述代理模块，被配置为拦截所述第二数据获取指令，并将所述第二数据获取指令发送至所述协调工作节点。
在一种可实现的方式中，所述协调工作节点还包括所述协调模块；
所述协调模块，被配置为接收任意一个所述代理模块发送的第二数据获取指令，并向所述代理模块发送所述目标数据；
所述代理模块，被配置为在接收到所述目标数据的情况下，将所述目标数据发送至对应的服务模块。
本说明书实施例提供一种数据传输系统，包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，其中，所述主节点，被配置为响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；所述数据获取工作节点，被配置为接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；所述协调工作节点，被配置为响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。通过在工作节点 中确定一个数据获取工作节点，由数据获取工作节点从主节点获取目标数据并存储至协调工作节点，每个工作节点在获取数据时不用都向主节点进行数据获取，减少了主节点与工作节点的通信流量。
上述为本实施例的一种数据传输装置的示意性方案。需要说明的是，该数据传输装置的技术方案与上述的数据传输方法的技术方案属于同一构思，数据传输装置的技术方案未详细描述的细节内容，均可以参见上述数据传输方法的技术方案的描述。
图5示出了根据本说明书一个实施例提供的一种计算设备500的结构框图。该计算设备500的部件包括但不限于存储器510和处理器520。处理器520与存储器510通过总线530相连接，数据库550用于保存数据。
计算设备500还包括接入设备540，接入设备540使得计算设备500能够经由一个或多个网络560通信。这些网络的示例包括公用交换电话网(PSTN)、局域网(LAN)、广域网(WAN)、个域网(PAN)或诸如因特网的通信网络的组合。接入设备540可以包括有线或无线的任何类型的网络接口(例如，网络接口卡(NIC))中的一个或多个，诸如IEEE802.11无线局域网(WLAN)无线接口、全球微波互联接入(Wi-MAX)接口、以太网接口、通用串行总线(USB)接口、蜂窝网络接口、蓝牙接口、近场通信(NFC)接口，等等。
在本说明书的一个实施例中，计算设备500的上述部件以及图5中未示出的其他部件也可以彼此相连接，例如通过总线。应当理解，图5所示的计算设备结构框图仅仅是出于示例的目的，而不是对本说明书范围的限制。本领域技术人员可以根据需要，增添或替换其他部件。
计算设备500可以是任何类型的静止或移动计算设备，包括移动计算机或移动计算设备(例如，平板计算机、个人数字助理、膝上型计算机、笔记本计算机、上网本等)、移动电话(例如，智能手机)、可佩戴的计算设备(例如，智能手表、智能眼镜等)或其他类型的移动设备，或者诸如台式计算机或PC的静止计算设备。计算设备500还可以是移动式或静止式的服务器。
其中，处理器520用于执行如下计算机可执行指令，该计算机可执行指令被处理器执行时实现上述数据传输方法和云端主节点与边缘节点的通信方法的步骤。
上述为本实施例的一种计算设备的示意性方案。需要说明的是，该计算设备的技术方案与上述的数据传输方法的技术方案属于同一构思，计算设备的技术方案未详细描述的细节内容，均可以参见上述数据传输方法和云端主节点与边缘节点的通信方法的技术方案的描述。
本说明书一实施例还提供一种计算机可读存储介质，其存储有计算机可执行指令，该计算机可执行指令被处理器执行时实现上述数据传输方法的步骤。
上述为本实施例的一种计算机可读存储介质的示意性方案。需要说明的是，该存储介质的技术方案与上述的数据传输方法的技术方案属于同一构思，存储介质的技术方案未详 细描述的细节内容，均可以参见上述数据传输方法和云端主节点与边缘节点的通信方法的技术方案的描述。
本说明书一实施例还提供一种计算机程序，其中，当所述计算机程序在计算机中执行时，令计算机执行上述数据传输方法和云端主节点与边缘节点的通信方法的步骤。
上述为本实施例的一种计算机程序的示意性方案。需要说明的是，该计算机程序的技术方案与上述的数据传输方法的技术方案属于同一构思，计算机程序的技术方案未详细描述的细节内容，均可以参见上述数据传输方法和云端主节点与边缘节点的通信方法的技术方案的描述。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
所述计算机指令包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
需要说明的是，对于前述的各方法实施例，为了简便描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本说明书实施例并不受所描述的动作顺序的限制，因为依据本说明书实施例，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定都是本说明书实施例所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其它实施例的相关描述。
以上公开的本说明书优选实施例只是用于帮助阐述本说明书。可选实施例并没有详尽叙述所有的细节，也不限制该发明仅为所述的具体实施方式。显然，根据本说明书实施例的内容，可作很多的修改和变化。本说明书选取并具体描述这些实施例，是为了更好地解释本说明书实施例的原理和实际应用，从而使所属技术领域技术人员能很好地理解和利用本说明书。本说明书仅受权利要求书及其全部范围和等效物的限制。

Claims (14)

  1. 一种数据传输方法，应用于数据传输系统，所述数据传输系统包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，所述方法包括：
    所述主节点响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；
    所述数据获取工作节点接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点；
    所述协调工作节点响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
  2. 根据权利要求1所述的方法，还包括：
    在所述至少两个工作节点中通过选举机制确定所述数据获取工作节点。
  3. 根据权利要求2所述的方法，所述在所述至少两个工作节点中通过选举机制确定所述数据获取工作节点，包括：
    所述至少两个工作节点向所述协调工作节点发送锁指令；
    所述协调工作节点接收所述至少两个工作节点发送的锁指令，在接收到锁指令的情况下，根据所述锁指令确定所述数据获取工作节点。
  4. 根据权利要求1所述的方法，还包括：
    所述至少两个工作节点中的目标工作节点，从所述协调工作节点获取需求数据，在所述协调工作节点中不存在所述需求数据的情况下，向所述主节点发送第三数据获取指令；
    所述主节点响应于所述第三数据获取指令，向所述目标工作节点发送所述需求数据；
    所述目标工作节点接收所述需求数据，其中，所述需求数据为所述主节点中除所述目标数据之外的其他数据。
  5. 根据权利要求1所述的方法，还包括：
    所述数据获取工作节点向所述主节点发送数据更新指令；
    所述主节点响应于所述数据更新指令，向所述数据获取工作节点发送更新数据；
    所述数据获取工作节点从所述主节点获取更新数据，并将所述更新数据存储至所述协调工作节点，其中，所述更新数据为所述目标数据中变更的数据。
  6. 根据权利要求1所述的方法，所述至少两个工作节点中每个工作节点均包括服务模块和代理模块；
    所述协调工作节点响应于其他工作节点发送的第二数据获取指令之前，还包括：
    所述服务模块向所述主节点发送第二数据获取指令；
    所述代理模块拦截所述第二数据获取指令，并将所述第二数据获取指令发送至所述协调工作节点。
  7. 根据权利要求6所述的方法，所述协调工作节点还包括所述协调模块；
    所述协调模块接收任意一个所述代理模块发送的第二数据获取指令，并向所述代理模块发送所述目标数据；
    所述代理模块在接收到所述目标数据的情况下，将所述目标数据发送至对应的服务模块。
  8. 根据权利要求7所述的方法，还包括：
    所述数据获取工作节点中的代理模块向所述主节点发送所述第一数据获取指令；
    所述主节点向所述响应于所述第一数据获取指令，向所述数据获取工作节点中的代理模块发送目标数据；
    所述数据获取工作节点中的代理模块接收所述主节点发送的所述目标数据，并将所述目标数据存储至所述协调工作节点中的协调模块。
  9. 根据权利要求8所述的方法，所述将所述目标数据存储至所述协调工作节点中的协调模块之后，还包括：
    所述协调模块接收所述数据获取工作节点中的代理模块发送的所述目标数据，并将所述目标数据写入存储中。
  10. 一种云端主节点与边缘节点的通信方法，应用于数据传输系统，所述数据传输系统包括云端主节点、至少两个边缘节点、以及从所述至少两个边缘节点中确定的数据获取边缘节点和协调边缘节点，包括：
    所述云端主节点响应于所述数据获取边缘节点发送的第一数据获取指令，向所述数据获取边缘节点发送目标数据；
    所述数据获取边缘节点接收所述云端主节点发送的所述目标数据，并将所述目标数据存储至所述协调边缘节点；
    所述协调边缘节点响应于其他边缘节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他边缘节点为所述至少两个边缘节点中除所述协调边缘节点之外的边缘节点。
  11. 一种数据传输系统，包括主节点、至少两个工作节点、以及从所述至少两个工作节点中确定的数据获取工作节点和协调工作节点，其中，
    所述主节点，被配置为响应于所述数据获取工作节点发送的第一数据获取指令，向所述数据获取工作节点发送目标数据；
    所述数据获取工作节点，被配置为接收所述主节点发送的所述目标数据，并将所述目 标数据存储至所述协调工作节点；
    所述协调工作节点，被配置为响应于其他工作节点发送的第二数据获取指令，向所述其他节点发送所述目标数据，其中，所述其他工作节点为所述至少两个工作节点中除所述协调工作节点之外的工作节点。
  12. 根据权利要求11所述的系统，所述至少两个工作节点中每个工作节点均包括服务模块和代理模块；
    所述协调工作节点，被配置为响应于其他工作节点发送的第二数据获取指令之前，还包括：
    所述服务模块，被配置为发送第二数据获取指令；
    所述代理模块，被配置为拦截所述第二数据获取指令，并将所述第二数据获取指令发送至所述协调工作节点。
  13. 根据权利要求12所述的系统，所述协调工作节点还包括所述协调模块；
    所述协调模块，被配置为接收任意一个所述代理模块发送的第二数据获取指令，并向所述代理模块发送所述目标数据；
    所述代理模块，被配置为在接收到所述目标数据的情况下，将所述目标数据发送至对应的服务模块。
  14. 一种计算设备，包括：
    存储器和处理器；
    所述存储器用于存储计算机可执行指令，所述处理器用于执行所述计算机可执行指令，该计算机可执行指令被处理器执行时实现权利要求1至10任意一项所述方法的步骤。
PCT/CN2023/085828 2022-04-06 2023-04-03 数据传输方法以及系统 WO2023193671A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210356688.1 2022-04-06
CN202210356688.1A CN114884880B (zh) 2022-04-06 2022-04-06 数据传输方法以及系统

Publications (1)

Publication Number Publication Date
WO2023193671A1 true WO2023193671A1 (zh) 2023-10-12

Family

ID=82668933

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/085828 WO2023193671A1 (zh) 2022-04-06 2023-04-03 数据传输方法以及系统

Country Status (2)

Country Link
CN (1) CN114884880B (zh)
WO (1) WO2023193671A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114884880B (zh) * 2022-04-06 2024-03-08 阿里巴巴（中国）有限公司 数据传输方法以及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107566505A (zh) * 2017-09-15 2018-01-09 郑州云海信息技术有限公司 数据存储资源管理方法、主节点、系统、装置及存储介质
WO2019184164A1 (zh) * 2018-03-30 2019-10-03 平安科技（深圳）有限公司 自动部署Kubernetes从节点的方法、装置、终端设备及可读存储介质
CN113946287A (zh) * 2021-09-08 2022-01-18 广州虎牙科技有限公司 分布式存储系统及其数据处理方法、相关装置
CN114884880A (zh) * 2022-04-06 2022-08-09 阿里巴巴（中国）有限公司 数据传输方法以及系统

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9817703B1 (en) * 2013-12-04 2017-11-14 Amazon Technologies, Inc. Distributed lock management using conditional updates to a distributed key value data store
CN110896404B (zh) * 2018-09-12 2021-09-14 华为技术有限公司 数据处理的方法、装置和计算节点
CN111614702B (zh) * 2019-02-22 2022-03-25 华为云计算技术有限公司 一种边缘计算方法以及边缘计算系统
CN113497817B (zh) * 2020-03-20 2023-04-07 北京金山云网络技术有限公司 流量调度方法、装置、cdn网络的边缘节点及服务器
CN111897638B (zh) * 2020-07-27 2024-04-19 广州虎牙科技有限公司 分布式任务调度方法及系统
CN112076464B (zh) * 2020-09-04 2022-06-21 腾讯科技（深圳）有限公司 一种数据请求处理方法、装置、计算机设备及存储介质
US20210014133A1 (en) * 2020-09-25 2021-01-14 Intel Corporation Methods and apparatus to coordinate edge platforms
CN113676564B (zh) * 2021-09-28 2022-11-22 深信服科技股份有限公司 数据传输方法、装置及存储介质
CN114241002B (zh) * 2021-12-14 2024-02-02 中国电信股份有限公司 基于云边协同的目标跟踪方法、系统、设备和介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107566505A (zh) * 2017-09-15 2018-01-09 郑州云海信息技术有限公司 数据存储资源管理方法、主节点、系统、装置及存储介质
WO2019184164A1 (zh) * 2018-03-30 2019-10-03 平安科技（深圳）有限公司 自动部署Kubernetes从节点的方法、装置、终端设备及可读存储介质
CN113946287A (zh) * 2021-09-08 2022-01-18 广州虎牙科技有限公司 分布式存储系统及其数据处理方法、相关装置
CN114884880A (zh) * 2022-04-06 2022-08-09 阿里巴巴（中国）有限公司 数据传输方法以及系统

Also Published As

Publication number Publication date
CN114884880A (zh) 2022-08-09
CN114884880B (zh) 2024-03-08

Similar Documents

Publication Publication Date Title
CN111277616B (zh) 一种基于rdma的数据传输方法和分布式共享内存系统
US7831734B2 (en) Method and system for remote configuration of network devices
US20130036236A1 (en) Method and system for creating peer-to-peer geographical routing and multi-attribute similarity routing
WO2023193671A1 (zh) 数据传输方法以及系统
US10228930B2 (en) Offloading network connectivity and execution tasks to an assistant device
CN113452592A (zh) 混合云架构下的跨云数据访问方法及装置
CN111352716B (zh) 一种基于大数据的任务请求方法、装置、系统及存储介质
CN110932876B (zh) 一种通信系统、方法及装置
WO2023185934A1 (zh) 数据处理方法及装置
CN111857772B (zh) 终端边缘应用程序远程传输及动态加载方法及设备
CN114025002A (zh) 一种基于mqtt信息传输的方法、系统及通信设备
WO2011087584A2 (en) Fault tolerant and scalable load distribution of resources
CN116048673A (zh) 业务处理方法及平台
JP7081014B2 (ja) インスタンス数を調整するための方法および装置、電子機器、記憶媒体並びにコンピュータプログラム
US20130013892A1 (en) Hierarchical multi-core processor, multi-core processor system, and computer product
JP6988973B2 (ja) Ｏｐｃ ｕａを用いたシステム処理、ｏｐｃ ｕａを用いた通信方法、及び負荷分散装置
EP3939236B1 (en) Node and cluster management on distributed self-governed ecosystem
EP3712768B1 (en) Management of services in an edge computing system
CN102427474B (zh) 云存储中的数据传输系统
CN112491935A (zh) 一种用于区块链的水波式广播方法及系统
CN112801795B (zh) 区块链多链管理方法、装置、电子设备及可读存储介质
CN114760360B (zh) 请求响应方法、装置、电子设备及计算机可读存储介质
CA3172917A1 (en) Tiered application pattern
Han et al. Online data caching in edge computing
Huh et al. oneM2M: Extension of protocol binding: Reuse of binding protocol's legacy services

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23784234

Country of ref document: EP

Kind code of ref document: A1