CN114143039B - Global multistage unified secure data transmission method and server cluster - Google Patents
Global multistage unified secure data transmission method and server cluster Download PDFInfo
- Publication number
- CN114143039B CN114143039B CN202111307940.1A CN202111307940A CN114143039B CN 114143039 B CN114143039 B CN 114143039B CN 202111307940 A CN202111307940 A CN 202111307940A CN 114143039 B CN114143039 B CN 114143039B
- Authority
- CN
- China
- Prior art keywords
- circulation
- data transmission
- data
- task
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 90
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000007246 mechanism Effects 0.000 claims abstract description 15
- 238000012546 transfer Methods 0.000 claims description 28
- 238000012795 verification Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 3
- 239000000758 substrate Substances 0.000 claims 1
- 230000002457 bidirectional effect Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001915 proofreading effect Effects 0.000 description 1
- 230000008521 reorganization Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/101—Access control lists [ACL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/321—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority
- H04L9/3213—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority using tickets or tokens, e.g. Kerberos
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a global multi-level unified safe data transmission method and a server cluster, which are characterized in that a message service SDK is created, cluster nodes of cascade transmission are dynamically recombined through a cascade data transmission structure of a forest topology structure, a circulation task is created, then the created circulation task is subjected to data transmission through the message service SDK, and the safety of data transmission among multi-level multi-center clusters is ensured through ACL authority control and Token mechanism in the data transmission process, so that reliable bidirectional data circulation among the dynamic customized multi-level clusters is realized, and finally the data circulation efficiency among the multi-level clusters is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a global multi-level unified and secure data transmission method and a server cluster.
Background
When data transmission is performed in a multi-center and multi-stage complex computer cluster environment, for example, as shown in fig. 1, a data center is deployed in a cluster 0, lower computer devices of the data center are respectively deployed on different computer clusters, for example, a computer device 1 is deployed on a cluster 1, a computer device 2 is deployed on a cluster 2, if at this time, when the data center performs data transmission with its lower computer devices, data transfer needs to be performed synchronously between the clusters, that is, when the data center needs to send data to the device 1, the cluster 0 needs to transfer the data synchronous transfer to the cluster 1, and similarly, when the device 2 needs to send data to the data center, the cluster 2 needs to transfer the data synchronous transfer to the cluster 0, and the principle of multi-stage and multi-center inter-cluster data cascade transfer is also the same.
The message middleware is used as an important subsystem for providing high-efficiency and reliable data transmission service in the current distributed system, the existing main stream message middleware technology comprises Kafka, pulsar, rocketMQ and the like, and has an implementation scheme for providing different message cross-cluster synchronization functions, and the method focuses on providing a technical scheme for full data synchronization among computer clusters distributed in different regions, so that the data consistency among regions is ensured, such as a gear replying cross-cluster message copying scheme of Pulsar, a MirrorMaker cross-cluster message copying scheme of Kafka and the like. However, the existing cross-cluster message synchronization technology cannot provide a service for data transmission through dynamic reorganization of multi-level multi-center computer clusters, so that reliable bidirectional data flow among the multi-level clusters cannot be dynamically customized, and the data flow efficiency among the multi-level clusters is reduced.
Disclosure of Invention
The invention provides a global multi-stage unified and safe data transmission method and a server cluster, which are used for solving the problem that reliable bidirectional data flow between multi-stage clusters cannot be efficiently carried out in the prior art.
In a first aspect, the present invention provides a global multi-level unified secure data transmission method, the method comprising: developing a message service SDK to provide messaging capability between message clients; designing a cascade data transmission structure based on forest topology, dynamically recombining cluster nodes of cascade transmission according to the cascade transmission structure, customizing a circulation relation, and creating a circulation task; performing data transmission on the created circulation task based on the message service SDK; meanwhile, the security of data transmission among the multi-level multi-center clusters is ensured through ACL authority control and Token mechanism.
Optionally, the method further comprises: and dynamically reorganizing the cascade transmission nodes based on the user instruction, and customizing the streaming relationship.
Optionally, the plurality of circulation tasks are included, and according to each circulation task and circulation relation thereof, corresponding relation among the circulation tasks is established by starting multithreading, so that customization of supporting a plurality of circulation tasks by one service is realized.
And creating a message service client by using multithreading according to the cluster nodes of the circulation tasks and the circulation relation thereof by accessing the message service SDK, and establishing a corresponding relation between the circulation tasks so as to realize cross-cluster data transmission of the circulation tasks.
Before data transmission, detecting whether a circulation task has a loop or not, wherein the loop checking of a single task and the loop checking of multiple tasks are included, the loop checking of the single task is to check whether data in two circulation directions are repeated or not, namely if the data is in the reported circulation direction and in the issued circulation direction, the circulation task has a loop path; the multi-task loop checking is based on all task circulation relations and circulation data.
By constructing a directed graph of all tasks in a contiguous matrix manner, whether loop paths exist in the directed graph or not is detected based on depth-first traversal DFS.
The Token mechanism adopts the client IP encrypted by the MD5 as a Token identifier to carry out identity authentication.
The ACL authority control comprises two parts, namely authority analysis and authority verification;
the authority analysis comprises the following steps: the server analyzes the request of the client to obtain an attribute field needing authentication;
the authority verification includes: checking whether the global IP full list is hit, checking whether the user IP full list is hit, checking the Access Key and the secretKey signature, and checking whether the rights required by the request and the rights owned by the user are matched.
In a second aspect, the present invention provides a server cluster, where a plurality of server clusters are provided, and any one of the methods described above is adopted to perform data transmission between a plurality of server clusters.
In a third aspect, the present invention provides a computer readable storage medium storing a computer program of signal mapping, which when executed by at least one processor, implements the steps of any of the above-described global multi-level unified secure data transmission methods.
The invention has the following beneficial effects:
the invention designs message service SDK to provide data receiving and transmitting capability and designs cascade data transmission structure. Creating a circulation task by dynamically reorganizing cluster nodes of cascade transmission according to the cascade data transmission structure and customizing a data circulation relation; then, carrying out data transmission on the created circulation task through a message service SDK; and the safety of data transmission among the multistage multi-center clusters is ensured through ACL authority control and Token mechanism, so that reliable bidirectional data flow among the multistage clusters is dynamically customized, and finally the data flow efficiency among the multistage clusters is improved.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a schematic diagram of data transmission among multiple multi-level multi-center computer clusters in the prior art;
FIG. 2 is a flow chart of a global multi-level unified secure data transmission method according to an embodiment of the invention;
FIG. 3 is a schematic diagram of an internal interface of a message service SDK according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a cascade transmission structure design based on forest topology according to an embodiment of the present invention;
FIG. 5 is a flow chart of a method for transmitting data in a global multi-stage multi-center according to an embodiment of the present invention;
fig. 6 is a schematic diagram of data transmission based on the data flow service according to an embodiment of the present invention;
fig. 7 is a schematic diagram of performing data flow service by using a cross-cluster node according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of the implementation principle of a depth-first traversal directed graph according to an embodiment of the invention;
FIG. 9 is a flow chart of an implementation of data transfer between cascaded clusters based on a message service SDK according to an embodiment of the present invention;
FIG. 10 is a flowchart illustrating a client authentication based on Token mechanism according to an embodiment of the present invention;
fig. 11 is a logic diagram of access control of a client request authority through an ACL for a message service according to an embodiment of the present invention.
Detailed Description
Aiming at the problem that reliable bidirectional data transfer among multistage clusters cannot be efficiently performed in the prior art, the embodiment of the invention dynamically reorganizes cluster nodes of cascade transmission through a cascade data transmission structure of a forest topology structure, customizes a transfer relation, creates a transfer task, then performs data transmission on the created transfer task through a message service SDK, and ensures the safety of data transmission among multistage multi-center clusters through ACL authority control and Token mechanism in the data transmission process, thereby realizing the reliable bidirectional data transfer among the dynamically customized multistage clusters and finally improving the data transfer efficiency among the multistage clusters. The present invention will be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
A first embodiment of the present invention provides a global multi-level unified secure data transmission method, referring to fig. 2, the method includes:
s201, creating a message service SDK to provide messaging capability between message clients;
in particular, embodiments of the present invention are directed to designing and developing a messaging service SDK that provides messaging capability between messaging clients. Where a messaging client refers to a program that uses a messaging service for messaging, whether a service or a client program. The message service provides access to the SDK, and the internal interface schematic diagram of the message service SDK is shown in FIG. 3;
s202, designing a cascade data transmission structure of a forest topological structure, dynamically reorganizing cluster nodes according to the cascade data transmission interface and customizing a data circulation relation, so as to create a circulation task; and carrying out data transmission on the created circulation task based on the message service SDK, and ensuring the safety of data transmission among the multistage multi-center clusters through ACL authority control and Token mechanism in the data transmission process.
Namely, according to the demand characteristics of data transmission among the multistage multi-center clusters, the embodiment of the invention designs a cascade data transmission architecture based on a forest topology structure so as to improve the high efficiency and reliability of data transmission, dynamically reorganizes cluster nodes of the cascade transmission and customizes a circulation relation through the cascade data transmission structure, creates a circulation task, realizes the data transmission of the circulation task based on the message service SDK, and ensures the safety of the data transmission among the multistage multi-center clusters through ACL authority control and Token mechanism.
Specifically, the embodiment of the invention designs a cascade data transmission structure based on forest topology, as shown in fig. 4. Compared with other topological structures, such as a star structure and a ring structure, the forest topological structure is considered to be a hierarchical multi-center centralized control type network interface, and the forest topological structure has the advantages of being high in reliability, low in data transmission delay and the like. Therefore, aiming at the demand characteristics of data transmission among multi-level multi-center clusters, in order to ensure the reliability and high efficiency of the data transmission in the environment, the embodiment of the invention designs a cascade transmission structure based on forest topology;
in addition, the embodiment of the invention also creates a circulation task by dynamically reorganizing the father domain cluster node and the subdomain cluster node according to the cascade transmission structure and customizing the data circulation relation, including reporting and issuing two circulation directions; in addition, one circulation task can customize two circulation directions simultaneously. The child domain cluster reports the data to the parent domain cluster, and the parent domain cluster transmits the data to the child domain cluster;
through the creation of the circulation tasks, the inter-cluster data circulation service provided by the invention supports customization of a plurality of circulation tasks at the same time;
according to the creation of the streaming task, in the embodiment of the present invention, before data streaming is performed between the parent domain and the child domain cluster node, loop checking processing is performed on the streaming task, including:
a) The loop checking of the single task, before the single task performs data circulation, checking whether the data in two circulation directions are repeated, and if so, the loop path exists for the task;
b) And (3) performing loop checking on a plurality of tasks, wherein the multitasks store a directed graph data structure represented by an adjacent matrix on the data content before data circulation, and perform loop checking processing on the plurality of circulation tasks by using a directed graph loop algorithm based on depth-first search.
In the embodiment of the invention, based on the message service SDK, the message service client is created through multithreading according to the reporting or issuing relation of the circulation task, and the message receiving and transmitting interface provided by the SDK is called to realize the data circulation among the multistage clusters.
In the specific implementation, in order to ensure the security of data transfer among multi-level multi-center clusters, the security and controllability of the data transfer are ensured by using a Token mechanism and ACL authority control;
in the embodiment of the invention, the client identity authentication is performed by using the client IP address encrypted by the MD5 as a Token; after the ACL verification is opened by the server, the client requests are subjected to authority analysis and authority verification, and access control of the resource level is performed.
The method according to the embodiments of the present invention will be explained and illustrated in detail with reference to fig. 5 to 11:
a) Message middleware is a type of middleware for communication by taking messages as carriers, and a high-efficiency and reliable message mechanism is utilized to realize a large amount of data exchange among different applications, so that the message middleware is an important supporting transmission component in a distributed application system. In large distributed systems, message middleware typically employs a communication model of message queues. Message middleware of the current mainstream includes RocketMQ, kafaka, rabbitMQ and the like.
b) Forest structures, common topological structures mainly comprise star structures, ring structures, forest structures and the like. The star-shaped structure connects all the work stations into a network in a star-shaped mode, the network has only one central node, other nodes are directly connected with the central node, the network has simple structure, convenient management and small network delay, but the defects are obvious, the central node is highly depended, and once the central node collapses, the whole data transmission structure collapses, so that the reliability is low; the ring structure is formed by connecting a plurality of nodes in a network end to end through point-to-point links, a central node is not arranged, and only an upstream node is required to receive data and send the data to a downstream node during data transmission, but the biggest defect is that depending on the reliability of all the nodes, when any one node breaks down or any one link breaks down, the whole structure breaks down; when the number of nodes in the ring is excessive, the data is generally required to be forwarded from one node to the other node for multiple times, and higher delay is probably caused;
compared with the star-shaped structure and the ring-shaped structure, the forest structure has more advantages. a) The hierarchical centralized control network can take the upper node as an authoritative data proofreading node so as to ensure the correctness of data; b) If the forest structure is properly established, the average time complexity of forwarding data from one node to the other nodes is O (log), which is relatively better; c) Even if the upper node crashes, the subtrees of the upper node can be independently networked to carry out data forwarding, so that the breakdown of the whole structure is avoided, and the method is relatively reliable. Therefore, the forest structure is more suitable for a multi-stage multi-center data transmission scheme.
d) Token mechanism: token, which is a string of characters generated by the server to be used as an identifier (Token) for the client to request, when the client requests, the server generates a Token, and the identity of the client is verified through the Token; there are two common ways to verify identity using Token: the first is to use the device number/device mac address as Token identification; the second is to use the session value as Token identification.
c) ACL: the access control list (Access Control Lists, ACL) is an access control technology based on packet filtering, and can filter the data packet on the interface according to the set condition, and allow the data packet to pass or be discarded. The ACL can relate to concepts of users, resources, rights, roles and the like, and can effectively control the access of the users to the network by means of the ACL, so that the network security is guaranteed to the greatest extent.
e) Directed graph: a graph is a special data structure, where each data element can be arbitrarily associated, and a typical graph structure is composed of vertices (data elements) and edges (lines connecting vertices). A directed graph is a graph with directionality on each side.
Fig. 5 is a flow chart of the multi-level multi-center secure unified data transmission method. As shown in fig. 5, the multi-level multi-center secure unified data transmission method of the present invention includes the steps of:
in step 501, creating a circulation task by dynamically reorganizing a parent domain cluster node and a child domain cluster node of cascade transmission and customizing a data circulation relationship (including reporting and issuing);
wherein, according to some embodiments of the invention, as shown in FIG. 6, one circulation task can customize a bidirectional circulation relation; in addition, the data transfer service supports the customization of a plurality of transfer tasks at the same time, and according to each transfer task and the transfer relation thereof, the customization of a service supporting a plurality of transfer tasks is realized by starting multithreading to establish the corresponding relation between the transfer tasks;
and then, in step 502, a multithreading message service client is created according to the task circulation relation based on the message service SDK according to the task circulation relation, so as to realize the cross-cluster data transmission of the circulation task. This step 502 comprises the sub-steps of:
according to the created circulation task, whether loop path checking exists or not is carried out on the circulation task, wherein the loop checking comprises loop checking of a single task and loop checking of a plurality of tasks;
the loop checking of a single circulation task is to check whether the data in two circulation directions are repeated or not, namely if the data is in the reported circulation direction and in the down circulation direction, the circulation task has a loop path;
the loop checking of a plurality of circulation tasks, according to all task circulation relations and circulation data, constructing a directed graph of all tasks in an adjacent matrix mode for improving the retrieval efficiency, and detecting whether a loop path exists in the directed graph or not based on depth-first traversal (DFS), as shown in figures 7-8;
wherein, in the adjacency matrix, rows represent parent domain clusters (source addresses) and columns represent child domain clusters (destination addresses); the DFS has the following steps:
a) Assuming that all vertexes in the initial state are not accessed, starting from each vertex v, accessing the vertex;
b) Sequentially performing depth-first traversal on the graph from each non-accessed adjacent point of v until all vertexes communicated with v in the graph with paths are accessed;
c) If other vertexes in the graph are not accessed, selecting one non-accessed vertex as a starting point for starting, and performing depth-first traversal again;
d) The above process is repeated until all vertices in the graph have been accessed.
Based on the message service SDK, establishing a message service client through multithreading according to the transfer tasks and the transfer relation thereof, and establishing a corresponding relation between the transfer tasks to realize cross-cluster data transmission of the transfer tasks, as shown in fig. 9;
the message service is an efficient, reliable, safe, convenient and extensible distributed message service, provides message transmission capability between message clients, currently supports Java language and C++ language access SDK, takes Java version of message service SDK as an example, takes 'ac.nci.xt4b.message client.client' as interface definition of the message clients, and main interface definition and function description are as follows:
the "ac.nci.xt4b.message client.image.clustermqclient" class is an interface implementation of a messaging client.
In addition, according to the embodiment of the invention, in order to enable the unified data transmission technical scheme of the multistage multi-center to have safety controllability, the safety of data transmission is ensured through a Token mechanism and ACL authority control, as shown in fig. 10-11.
When the message client requests connection service, the Token transmitted by the client is compared with the Token acquired by the server, if the connection service passes through the verification of the connection service, if the connection service of the client is successful, and if the connection service of the client is refused, the connection service is refused.
ACL rights control primarily provides resource-level user access control for message services. The message service injects an Access Key and a secretKey signature into a client through RPCHook, when a server opens ACL verification, and the message client after successful connection service requests to send/subscribe a message to the server, the server checks the authority owned by the Access Key in the client request, and if the verification is not passed, the server throws out an exception. In addition, the main ACL flow mainly comprises two parts of authority analysis and authority verification, and specific logic is shown in fig. 11. The specific flow mainly comprises the following steps: the server analyzes the request of the client to obtain an attribute field needing authentication; the server performs a series of checks on the authority, including: checking whether a global IP full list is hit; checking whether the user IP full list is hit; checking the Access Key and the secretKey signature; checking whether the rights required by the request and the rights possessed by the user match or not, and the like.
In general, embodiments of the present invention design and develop a messaging service SDK that provides messaging capabilities; designing a cascade data transmission path based on a forest topology structure; creating a circulation task by dynamically reorganizing a parent domain cluster and a child domain cluster and customizing a data circulation relation (including reporting and downloading) according to the cascade transmission structure; through the circulation tasks, unified data circulation among cascade clusters is realized based on the message service SDK according to circulation relations; the security in the inter-cluster data transmission process is ensured through a Token mechanism and ACL authority control; meanwhile, in the data transmission process, one task is supported to customize the bidirectional circulation direction at the same time, in addition, all circulation tasks are subjected to loop checking processing based on a directed graph loop algorithm of depth-first retrieval, so that safe and controllable multi-level multi-center unified data circulation is realized, and effective and reliable service is provided for data transmission among multi-level multi-center clusters.
The second embodiment of the present invention provides a plurality of server clusters, and the data transmission is performed between the plurality of server clusters by using the global multi-level unified and secure data transmission method according to any one of the first embodiments of the present invention. The relevant content of the embodiments of the present invention can be understood with reference to the first embodiment of the present invention, and will not be discussed in detail herein.
A third embodiment of the present invention provides a computer readable storage medium storing a computer program of signal mapping, which when executed by at least one processor, implements the global multi-level unified secure data transmission method according to any one of the first embodiments of the present invention. The relevant content of the embodiments of the present invention can be understood with reference to the first embodiment of the present invention, and will not be discussed in detail herein.
Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, and accordingly the scope of the invention is not limited to the embodiments described above.
Claims (8)
1. A global multi-level unified secure data transmission method, the method comprising:
creating a message service SDK to provide messaging capability between message clients;
the method comprises the steps of dynamically recombining cluster nodes of cascade transmission through a cascade data transmission structure of a forest topological structure, customizing a circulation relation, creating a circulation task, carrying out data transmission on the created circulation task through a message service SDK, and guaranteeing the safety of data transmission among multi-level multi-center clusters through ACL authority control and Token mechanism in the data transmission process;
according to the cascade transmission structure, dynamically reorganizing a parent domain cluster node and a child domain cluster node, customizing a data transfer relation, and reporting and issuing two transfer directions, wherein the child domain cluster reports the data transferred by the parent domain cluster, and the parent domain cluster issues the data transferred by the child domain cluster;
the circulation tasks comprise a plurality of circulation tasks, and according to each circulation task and the circulation relation thereof, the corresponding relation among the circulation tasks is established by starting multithreading so as to realize the customization of supporting a plurality of circulation tasks by one service;
and creating a message service client by using multithreading according to the cluster nodes of the circulation tasks and the circulation relation thereof by accessing the message service SDK, and establishing a corresponding relation between the circulation tasks so as to realize cross-cluster data transmission of the circulation tasks.
2. The method according to claim 1, wherein the method further comprises:
and dynamically reorganizing the cascade transmission nodes based on the user instruction, and customizing the streaming relationship.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises,
before data transmission, detecting whether a circulation task has a loop or not, wherein the loop checking of a single task and the loop checking of multiple tasks are included, wherein the loop checking of the single task is to check whether data in two circulation directions are repeated or not, namely if the data is in the reported circulation direction and in the issued circulation direction, the circulation task has a loop path; the multi-task loop checking is based on all task circulation relations and circulation data.
4. The method of claim 3, wherein the step of,
by constructing a directed graph of all tasks in a contiguous matrix manner, whether loop paths exist in the directed graph or not is detected based on depth-first traversal DFS.
5. The method according to any one of claims 1 to 4, wherein,
the Token mechanism adopts the client IP encrypted by the MD5 as a Token identifier to carry out identity authentication.
6. The method according to any one of claims 1 to 4, wherein,
the ACL authority control comprises two parts of authority analysis and authority verification, wherein the authority analysis comprises the following steps: the server analyzes the request of the client to obtain an attribute field needing authentication; the authority verification includes: checking whether the global IP full list is hit, checking whether the user IP full list is hit, checking the Access Key and the secretKey signature, and checking whether the rights required by the request and the rights owned by the user are matched.
7. A server cluster, comprising: the server clusters are multiple, and the data transmission is performed among the multiple server clusters by adopting the global multi-level unified and safe data transmission method according to any one of claims 1-6.
8. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, which when executed by a processor, implements the steps of the global multi-level unified secure data transfer method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111307940.1A CN114143039B (en) | 2021-11-05 | 2021-11-05 | Global multistage unified secure data transmission method and server cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111307940.1A CN114143039B (en) | 2021-11-05 | 2021-11-05 | Global multistage unified secure data transmission method and server cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114143039A CN114143039A (en) | 2022-03-04 |
| CN114143039B true CN114143039B (en) | 2024-04-16 |
Family
ID=80392398
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111307940.1A Active CN114143039B (en) | 2021-11-05 | 2021-11-05 | Global multistage unified secure data transmission method and server cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114143039B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101252506A (en) * | 2007-12-29 | 2008-08-27 | 中国建设银行股份有限公司 | Data transmission system |
| CN110611621A (en) * | 2019-09-26 | 2019-12-24 | 上海依图网络科技有限公司 | Tree-structured multi-cluster routing control method and cluster forest |
| CN111865935A (en) * | 2020-06-30 | 2020-10-30 | 北京天融信网络安全技术有限公司 | Data transmission system |
| CN112235285A (en) * | 2020-10-12 | 2021-01-15 | 山东健康医疗大数据有限公司 | Method and system for user authentication mode and service based on non-session |
| US11165859B1 (en) * | 2021-04-09 | 2021-11-02 | Area 1 Security, Inc. | Scaling stateful services hosted in a cluster of server nodes |
-
2021
- 2021-11-05 CN CN202111307940.1A patent/CN114143039B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101252506A (en) * | 2007-12-29 | 2008-08-27 | 中国建设银行股份有限公司 | Data transmission system |
| CN110611621A (en) * | 2019-09-26 | 2019-12-24 | 上海依图网络科技有限公司 | Tree-structured multi-cluster routing control method and cluster forest |
| CN111865935A (en) * | 2020-06-30 | 2020-10-30 | 北京天融信网络安全技术有限公司 | Data transmission system |
| CN112235285A (en) * | 2020-10-12 | 2021-01-15 | 山东健康医疗大数据有限公司 | Method and system for user authentication mode and service based on non-session |
| US11165859B1 (en) * | 2021-04-09 | 2021-11-02 | Area 1 Security, Inc. | Scaling stateful services hosted in a cluster of server nodes |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114143039A (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alfandi et al. | Blockchain solution for iot-based critical infrastructures: Byzantine fault tolerance | |
| KR102566892B1 (en) | Blockchain consensus method, device and system | |
| US9495392B2 (en) | System and method for parallel multiplexing between servers in a cluster | |
| EP3718285B1 (en) | Computer-implemented system and method for propagation and communication of data in a network such as a blockchain network | |
| Guo et al. | Exploiting efficient and scalable shuffle transfers in future data center networks | |
| US10341138B2 (en) | Spanning tree protocol enabled N-node link aggregation system | |
| US20040179481A1 (en) | Overlay network for location-independent communication between computer systems | |
| CN113596176A (en) | Self-selection method and device of Internet of things center node, Internet of things equipment and system | |
| Alvisi et al. | How robust are gossip-based communication protocols? | |
| Biswas et al. | A novel leader election algorithm based on resources for ring networks | |
| Huang et al. | Blockchain based log system | |
| Newport | Consensus with an abstract MAC layer | |
| Rodrigues et al. | A distributed k-mutual exclusion algorithm based on autonomic spanning trees | |
| CN114143039B (en) | Global multistage unified secure data transmission method and server cluster | |
| Lim et al. | Unstructured deadlock detection technique with scalability and complexity‐efficiency in clouds | |
| Herault et al. | A model for large scale self-stabilization | |
| Mehrabian et al. | Randomized rumor spreading in poorly connected small‐world networks | |
| US10397099B2 (en) | Spanning tree protocol enabled N-node link aggregation system | |
| Haeupler et al. | Optimal gossip with direct addressing | |
| US11711220B1 (en) | System and methods for computation, storage, and consensus in distributed systems | |
| CN113746894A (en) | Method and device for realizing load balancing based on HDFS RBF routing layer | |
| Coulson et al. | The design of a generalised approach to the programming of systems of systems | |
| CN101521597B (en) | Data statistical approach and system of mixed P2P network | |
| Wang et al. | A physical topology for optimizing partition tolerance in consortium blockchains to reach CAP guarantee bound | |
| Zhai et al. | An improved DDS publish/subscribe automatic discovery algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |