CN114422493A

CN114422493A - Data transmission method, device, equipment and storage medium of distributed system

Info

Publication number: CN114422493A
Application number: CN202210060266.XA
Authority: CN
Inventors: 孙飞
Original assignee: Ping An E Wallet Electronic Commerce Co Ltd
Current assignee: Ping An E Wallet Electronic Commerce Co Ltd
Priority date: 2022-01-19
Filing date: 2022-01-19
Publication date: 2022-04-29

Abstract

The invention relates to the technical field of big data, and discloses a data transmission method, a data transmission device, data transmission equipment and a data transmission storage medium of a distributed system, which are used for simplifying the deployment process of the distributed system and reducing the deployment cost. The data transmission method of the distributed system comprises the following steps: an entry node in the distributed system receives a data access request from an external network, and determines whether the data access request is an effective request of the distributed system according to the format of the data access request; and if the first access request is an effective request of the distributed system, sending the first access request to a target routing node corresponding to the target service node, forwarding the first access request to the target service node through the target routing node, and responding to the first access request by the target service node. In addition, the invention also relates to a block chain technology, and data of the distributed system can be stored in the block chain nodes.

Description

Data transmission method, device, equipment and storage medium of distributed system

Technical Field

The present invention relates to the field of big data technologies, and in particular, to a data transmission method, apparatus, device, and storage medium for a distributed system.

Background

In computer system services, the deployment of distributed systems has become the mainstream. The distributed system is composed of a plurality of computers, the computers provide services to the outside in a unified mode, and hardware nodes among the computers in the system are independent and not shared. When a plurality of computers in the system cooperate, a communication mechanism is needed to ensure the division of labor and assistance among the computers, so that the external service is provided uniformly.

The network call of the distributed system may also be referred to as an RPC remote service call scheme, and in the related art, the basic service communication of the distributed system is currently completed through components such as dubbo, springclosed and the like. However, these methods rely on other components, for example, dubbo needs to rely on zookeeper for service registration, and some functions also need to be developed secondarily, such as monitoring, logging, current limiting, tracking, and the like, and are complex to deploy and high in deployment cost.

Disclosure of Invention

The invention provides a data transmission method, a data transmission device, data transmission equipment and a storage medium of a distributed system, which are used for simplifying the deployment process of the distributed system and reducing the deployment cost.

In order to achieve the above object, a first aspect of the present invention provides a data transmission method for a distributed system, where the method is applied to an entry node in the distributed system; the distributed system also comprises a routing node and a service node; the entry node and the service node are respectively connected with the routing node; the method comprises the following steps: receiving a data access request from an external network, and determining whether the data access request is an effective request of a distributed system according to the format of the data access request; if the first access request is an effective request of the distributed system, according to a target service node accessed by the first access request, the first access request is sent to a target routing node corresponding to the target service node, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request.

Optionally, in a first implementation manner of the first aspect of the present invention, a legal format corresponding to the distributed system is pre-stored in the entry node; the step of determining whether the first access request is a valid request for the distributed system based on the format of the first access request comprises: and judging whether the format of the first access request is matched with the legal format, and if so, determining that the first access request is a valid request of the distributed system.

Optionally, in a second implementation manner of the first aspect of the present invention, the method further includes: monitoring real-time request traffic of a first access request from an external network; and determining whether the real-time request flow is higher than a preset flow threshold, and if the real-time request flow is higher than the flow threshold, putting the received first access request into a preset queue component so as to perform flow limitation processing on the first access request through the queue component.

The second aspect of the present invention provides another data transmission method for a distributed system, where the method is applied to an egress node in the distributed system; the distributed system also comprises a routing node and a service node; the exit node and the service node are respectively connected with the routing node; the method comprises the following steps: receiving a second access request forwarded by the routing node; the second access request is sent to the routing node by the service node; the second access request is used for accessing an external network; determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request; and if the second access request is a valid request of the distributed system, sending the second access request to the external network so as to send the second access request to the target URL address through the external network.

Optionally, in a first implementation manner of the second aspect of the present invention, a service node having an access right to access a specified URL address is stored in advance in the egress node; the step of determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request comprises the following steps: judging whether the service node sending the second access request has an access right for accessing the target URL address; if the access right to the target URL address is available, the second access request is determined to be a valid request of the distributed system.

Optionally, in a second implementation manner of the second aspect of the present invention, after the step of sending the second access request to the external network if the second access request is a valid request of the distributed system, the method further includes: counting request response parameters of the access request corresponding to the target URL address; wherein the request response parameters include: the response duration, response success times and response failure times of the target URL address responding to the access request; based on the request response parameter, a processing priority of the access request to access the target URL address is adjusted.

The third aspect of the present invention provides a data transmission device for a distributed system, where the device is disposed at an entry node in the distributed system; the distributed system also comprises a routing node and a service node; the entry node and the service node are respectively connected with the routing node; the device comprises a first receiving module, a second receiving module and a third receiving module, wherein the first receiving module is used for receiving a data access request from an external network and determining whether the data access request is a valid request of the distributed system or not according to the format of the data access request; and the first sending module is used for sending the first access request to a target routing node corresponding to the target service node according to the target service node accessed by the first access request if the first access request is an effective request of the distributed system, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request.

Optionally, in a first implementation manner of the third aspect of the present invention, a legal format corresponding to a distributed system is pre-stored in the entry node; the first receiving module is specifically configured to: and judging whether the format of the first access request is matched with the legal format, and if so, determining that the first access request is a valid request of the distributed system.

Optionally, in a second implementation manner of the third aspect of the present invention, the data transmission apparatus of the distributed system further includes a monitoring module, configured to monitor a real-time request traffic of the first access request from the external network; and determining whether the real-time request flow is higher than a preset flow threshold, and if the real-time request flow is higher than the flow threshold, putting the received first access request into a preset queue component so as to perform flow limitation processing on the first access request through the queue component.

The fourth aspect of the present invention provides another data transmission apparatus for a distributed system, where the apparatus is disposed at an egress node in the distributed system; the distributed system also comprises a routing node and a service node; the exit node and the service node are respectively connected with the routing node; the device comprises a second receiving module, a first access module and a second access module, wherein the second receiving module is used for receiving a second access request forwarded by the routing node; the second access request is sent to the routing node by the service node; the second access request is used for accessing an external network; the validity determining module is used for determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request; and the second sending module is used for sending the second access request to the external network if the second access request is a valid request of the distributed system so as to send the second access request to the target URL address through the external network.

Optionally, in a first implementation manner of the fourth aspect of the present invention, a service node having an access right to access a specified URL address is stored in advance in the egress node; the validity determination module is specifically configured to: judging whether the service node sending the second access request has an access right for accessing the target URL address; if the access right to the target URL address is available, the second access request is determined to be a valid request of the distributed system.

Optionally, in a second implementation manner of the fourth aspect of the present invention, the data transmission apparatus of the distributed system further includes a statistics module, configured to count request response parameters of the access request corresponding to the target URL address; wherein the request response parameters include: the response duration, response success times and response failure times of the target URL address responding to the access request; based on the request response parameter, a processing priority of the access request to access the target URL address is adjusted.

A fifth aspect of the present invention provides a data transmission device for a distributed system, including: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes instructions in the memory to cause the data transfer device of the distributed system to perform the data transfer method of the distributed system described above.

A sixth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which when run on a computer, cause the computer to execute the above-mentioned data transmission method of the industrial distributed system.

In the technical scheme provided by the invention, a data access request from an external network is received, and whether the data access request is an effective request of a distributed system is determined according to the format of the data access request; if the first access request is an effective request of the distributed system, according to a target service node accessed by the first access request, the first access request is sent to a target routing node corresponding to the target service node, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request. In the distributed system, an ingress node, an egress node, a routing node and a service node are provided, where the routing node is configured to forward information from each node, receive a request from an external network through the ingress node, and process the request by the service node. The distributed system can realize the division of labor cooperation of the multiple service nodes without deploying a large number of components, and is simple in deployment and low in cost.

Drawings

FIG. 1 is a schematic structural diagram of a distributed system according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an embodiment of a data transmission method of a distributed system in an embodiment of the present invention;

fig. 3 is a schematic diagram of another embodiment of a data transmission method of a distributed system according to an embodiment of the present invention;

fig. 4 is a schematic diagram of another embodiment of a data transmission method of a distributed system in the embodiment of the present invention;

fig. 5 is a schematic diagram of an embodiment of a data transmission apparatus of a distributed system according to an embodiment of the present invention;

fig. 6 is a schematic diagram of another embodiment of a data transmission apparatus of a distributed system according to an embodiment of the present invention;

fig. 7 is a schematic diagram of another embodiment of a data transmission apparatus of a distributed system according to an embodiment of the present invention;

fig. 8 is a schematic diagram of another embodiment of a data transmission apparatus of a distributed system according to an embodiment of the present invention;

fig. 9 is a schematic diagram of an embodiment of a data transmission device of a distributed system in an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides a data transmission method, a data transmission device, data transmission equipment and a storage medium of a distributed system, which are used for uniformly providing services for the outside.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, please refer to fig. 1, which is a schematic structural diagram of a distributed system according to an embodiment of the present invention, in which a plurality of service nodes are provided, and fig. 1 takes three service nodes as an example for description, and further provides a routing node, an ingress node, and an egress node, where the service nodes implement communication between the service nodes through the routing node; the service node realizes the access to the external system through the routing node and the exit node; the external system accesses the service node inside the distributed system through the entry node and the routing node. The distributed system structure is easy to deploy, does not depend on third-party components or containers, and is low in deployment cost and easy to implement.

Referring to fig. 2, a specific flow of a data transmission method for a distributed system implemented by an ingress node in the distributed system in the embodiment of the present invention is described below, where an embodiment of the data transmission method for a distributed system in the embodiment of the present invention includes:

201. and receiving a data access request from an external network, and determining whether the data access request is a valid request of the distributed system according to the format of the data access request.

The distributed system is provided with a plurality of service nodes, routing nodes, inlet nodes and outlet nodes; each service node is respectively in communication connection with the routing node; the inlet node and the outlet node are respectively connected with the routing node in a communication mode. The service nodes realize communication among the service nodes through the routing nodes; the service node realizes the access to the external system through the routing node and the exit node; the external system accesses the service node inside the distributed system through the entry node and the routing node.

Specifically, the entry node monitors the format of an access request sent by an external network, and identifies and rejects invalid requests and malicious requests; one or more legal formats or format rules of the access request can be stored in the entry node; after receiving a data access request from an external network, judging whether the access request is legal or not according to the legal format or the format rule, if so, forwarding the access request to a routing node, and executing step 202; if not, the access request is denied and the service that sent the access request is notified.

In another mode, if the distributed system only processes the access request sent by the specified service, the format of the access request sent by the specified service is determined to be a legal format, and if the access request received by the ingress node does not match the legal format, it may be determined that the access request is not sent by the specified service, and the access request is rejected.

It is to be understood that the execution subject of the present invention may be a data transmission apparatus of a distributed system, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a terminal as an execution subject.

202. If the first access request is a valid request of the distributed system, according to a target service node accessed by the first access request, the first access request is sent to a target routing node corresponding to the target service node, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request.

Specifically, when the target routing node receives the access request, the routing node firstly recombines the message of the access request according to the access address of the access request, so that the format of the message of the access request is matched with the format required by the access address. And then forwarding the access request to a target service node corresponding to the access address according to the access address of the access request.

When the number of service nodes in the distributed system is large, a plurality of routing nodes can be set, and each routing node is used for forwarding access requests corresponding to a part of service nodes, so that the forwarding amount of each routing node is reduced. In other manners, one service node may also be connected to multiple routing nodes at the same time, and each connected routing node may forward an access request corresponding to the service node, that is, the service nodes forwarded between the routing nodes may overlap. Therefore, the problem that a certain routing node fails to process the request of part of service nodes is avoided.

The portal node provides an access interface to an external network, which can invoke the portal node through http or https requests. In the distributed system of the present embodiment, the ingress node and the egress node have different functions. And the ingress node is only used for processing the access request sent by the external network. Based on this, the portal node provides an access interface for the external network, and the access interface may specifically be address information such as a website or a URL (Uniform Resource Locator). Besides the entry node processing receiving the access request, in order to ensure the stability and security of the distributed system, the entry node also needs to perform processing such as format conversion, request monitoring, access flow management and control, access request distribution, log recording, encryption, signature and the like on the access request.

Some operations of the ingress node may be recorded in the log, such as which access requests were forwarded, which access requests were denied, and so on. If the distributed system has encryption requirements, the access request can be encrypted, signed and the like through a preset encryption algorithm, then the access request is forwarded to the routing node, and the access request is sent to the service node after the routing node carries out decryption, signature verification and the like. Of course, the service node may perform decryption, signature verification, and the like.

Further, the terminal stores the legal format of the access request in the blockchain database, which is not limited herein.

In the embodiment of the invention, a distributed system is provided with a plurality of service nodes, routing nodes, inlet nodes and outlet nodes; each service node is respectively in communication connection with the routing node; the inlet node and the outlet node are respectively connected with the routing node in a communication mode. And the service node accesses the access request of the external network through the entrance node. The ingress node is also used to monitor and identify the format of access requests sent by external networks. The access request is identified and rejected at the entry node aiming at invalid requests and malicious requests, in addition, the entry node can also monitor the flow of access sent by an external network, and when the flow is higher than a preset flow threshold value, the flow limiting processing is carried out on the access request, so that the service efficiency is improved, and the safety of service and data is protected.

The data transmission method of the distributed system receives a data access request from an external network, and determines whether the data access request is an effective request of the distributed system according to the format of the data access request; if the first access request is an effective request of the distributed system, according to a target service node accessed by the first access request, the first access request is sent to a target routing node corresponding to the target service node, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request. In the distributed system, an ingress node, an egress node, a routing node and a service node are provided, where the routing node is configured to forward information from each node, receive a request from an external network through the ingress node, and process the request by the service node. The distributed system can realize the division of labor cooperation of the multiple service nodes without deploying a large number of components, and is simple in deployment and low in cost.

Optionally, the entry node determines whether the format of the first access request matches a legal format, and if so, determines that the first access request is an effective request of the distributed system.

It will be appreciated that one or more legal or regular formats of the access request may be maintained at the ingress node, for example: the http request format includes: request method URI agreement/version, request head and request body, the request method includes: get, post, put, delete, options, trace, connect, head, the request header contains many useful information about the client environment and the request body, for example: the request header can be the language used by the life browser, the length of the request body, and the request body can include the query string information submitted by the client. Based on this, the entry node can judge whether the format of the first access request is matched with the legal format, determine whether the first access request is an effective request of the distributed system, and improve the data transmission efficiency and the security of the distributed system.

Further, the entry node monitors real-time request traffic of a first access request from an external network; and determining whether the real-time request flow is higher than a preset flow threshold, and if the real-time request flow is higher than the flow threshold, putting the received first access request into a preset queue component so as to perform flow limitation processing on the first access request through the queue component. For example, a traffic threshold may be set, and if the traffic of the access request is lower than the threshold, the access request is forwarded normally, and if the traffic of the access request is higher than the threshold, the throttling process is performed. Specifically, the access requests may be queued according to the arrival order of the access requests, and when the queue is long, some access requests may be delayed to be forwarded. Or, the processing priority of the access request is set, a part of the access request is filtered according to the request type of the access request, the service type of sending the access request, the service type of receiving the access request and other factors, and the filtered access request is not responded.

Referring to fig. 3, a specific flow of a data transmission method for a distributed system implemented by an egress node in the distributed system in the embodiment of the present invention is described below, where another embodiment of the data transmission method for a distributed system in the embodiment of the present invention includes:

301. receiving a second access request forwarded by the routing node; the second access request is sent to the routing node by the service node; the second access request is for accessing an external network.

Specifically, the egress node receives a second access request forwarded by the routing node, where the access request is used to access an external network and is sent to the routing node by the service node. When the routing node receives the access request sent by the service node, the message of the access request is recombined according to the access address of the access request, so that the message format of the access request is matched with the format required by the access address. And then forwarding the access request to an egress node corresponding to the access address.

302. And determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request.

The URL is a compact representation of the location and access method of the resource obtained on the internet, and is the address of the standard resource on the internet. Each file on the internet has a unique URL, which includes: resource type, host domain name and resource file name of resource storage. Through the URL, the required resources such as files, databases, images and the like can be found on the Internet.

Further, the exit node registers the URL address called by the access request in advance, allocates the authority for calling a certain URL address for the service node in the distributed system, and rejects the request if the certain service node does not have the authority for accessing the specific URL address; specifically, the URL address to be accessed by each service in the distributed system is registered in the exit node, the corresponding relation between the service and the URL address is stored, and whether the access request has the authority to access a certain URL address is determined according to the corresponding relation. By the method, the service in the distributed system can communicate with the specific URL, on one hand, the reliability of communication can be guaranteed, and on the other hand, network congestion or access abnormity caused by sending an abnormal access request when the service is abnormal can be avoided.

303. And if the second access request is a valid request of the distributed system, sending the second access request to the external network so as to send the second access request to the target URL address through the external network.

Specifically, after determining an effective request of the distributed system in the second access request based on a URL address called by an access request registered in advance by the egress node, the second access request is sent to an external network, where the external network generally refers to the internet or a wide area network, for example: public network, extranet, are a network throughout the world. The second request may be sent to the target URL address over an external network.

Before sending the access request to the external network, the egress node may also encrypt the access request and send the encrypted access request to the external network. The egress node may be pre-configured with an encryption algorithm by which to encrypt the access request. Meanwhile, a decryption algorithm corresponding to the encryption algorithm needs to be preset in the service receiving the access request, and after the service receives the access request, decryption is carried out through the decryption algorithm. Thereby improving the security of the transmission of the access request in the external network.

In the embodiment of the invention, a distributed system is provided with a plurality of service nodes, routing nodes, inlet nodes and outlet nodes; each service node is respectively in communication connection with the routing node; the inlet node and the outlet node are respectively connected with the routing node in a communication mode. The service node accesses the external network through the exit node, the exit node registers a URL address in advance, whether the access request has access authority is determined according to the pre-registered URL information, if so, the access request can be sent to the external network after being encrypted, and if not, the access request is rejected. The exit node is only used for processing the access request of the service node in the distributed system for accessing the external network, and the exit node and the entrance node are separately arranged, so that the problems of low request processing efficiency and even request blocking caused by the fact that the same node simultaneously processes the two-way access request can be avoided.

Optionally, the service node having an access right to access the specified URL address is stored in advance in the egress node; the exit node judges whether the service node sending the second access request has the access right of accessing the target URL address; if the access right to the target URL address is available, the second access request is determined to be a valid request of the distributed system.

It is understood that a plurality of service nodes are provided in the distributed system for providing various services, such as login, payment, batch processing, interface service, etc., the functions of the service nodes are usually associated with the functions of the distributed system, and if the distributed system is a shopping website, the service nodes may include login nodes, payment nodes, communication nodes, etc.; if the distributed system is an educational website, the service nodes may include login nodes, check-in nodes, test nodes, and the like. The service nodes have different functions, resulting in different access rights. Therefore, after receiving the second access request, the exit node firstly judges whether the service node sending the second access request has the access authority for accessing the target URL address according to the URL address called by the pre-registered access request; if the access right of the target URL address is available, the second access request is determined to be a valid request of the distributed system, and then the subsequent steps are executed.

Further, the exit node counts request response parameters of the access request corresponding to the target URL address; wherein the request response parameters include: the response duration, response success times and response failure times of the target URL address responding to the access request; based on the request response parameter, a processing priority of the access request to access the target URL address is adjusted.

That is, when a certain URL in the external network is accessed for multiple times and is timed out, it may be determined that the network accessing the URL fails or the URL itself is inaccessible, and at this time, in order to improve the efficiency of the egress node in processing the access request, reduce the access priority of the access request accessing the URL, preferentially process the access requests accessing other normal URLs.

In other ways, after a period of time, the access request may be sent to the URL again to determine whether the network of the URL or the URL itself is normal. If normal, the priority of the access request accessing the URL is increased.

For convenience of understanding, a specific flow of the data transmission method of the distributed system according to the embodiment of the present invention is described below:

firstly, an entry node receives a data access request from an external network, and determines whether the data access request is a valid request of a distributed system according to the format of the data access request.

Specifically, the step of determining whether the first access request is an effective request of the distributed system according to the format of the first access request includes: and judging whether the format of the first access request is matched with the legal format, and if so, determining that the first access request is a valid request of the distributed system.

If the first access request is an effective request of the distributed system, according to a target service node accessed by the first access request, the first access request is sent to a target routing node corresponding to the target service node, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request.

Optionally, the ingress node monitors a real-time request traffic of the first access request from the external network; and determining whether the real-time request flow is higher than a preset flow threshold, and if the real-time request flow is higher than the flow threshold, putting the received first access request into a preset queue component so as to perform flow limitation processing on the first access request through the queue component. For example, a traffic threshold may be set, and if the traffic of the access request is lower than the threshold, the access request is forwarded normally, and if the traffic of the access request is higher than the threshold, the throttling process is performed. Specifically, the access requests may be queued according to the arrival order of the access requests, and when the queue is long, some access requests may be delayed to be forwarded. Or, the processing priority of the access request is set, a part of the access request is filtered according to the request type of the access request, the service type of sending the access request, the service type of receiving the access request and other factors, and the filtered access request is not responded.

Further, when receiving the access request, the routing node recombines the message of the first access request according to the access address of the first access request, so that the format of the message of the access request is matched with the format required by the access address.

And the routing node forwards the access request to a service node corresponding to the access address according to the access address of the access request.

Further, the service node generates a second access request, searches a corresponding routing node from the routing node list according to an access address of the second access request, and sends the second access request to the routing node, so that the second access request is forwarded to a node corresponding to the access address through the routing node.

Before the second access request is sent, the message format of the access request is also assembled. When the message format is assembled, the assembly can be carried out according to the message format requirement of the routing node, and the assembly can also be carried out according to the message format requirement of the service receiving the access request.

And after the service node sends the access request to the routing node, timing is started, whether a return result is received within a preset time period or not is judged, if the return result is received, the return result is analyzed, if the return result is not received, the access is determined to be overtime, and the access request can be initiated again.

Further, when receiving the second access request, the routing node recombines the message of the access request according to the access address of the second access request, so that the format of the message of the access request is matched with the format required by the access address.

And forwarding the access request to an exit node corresponding to the access address according to the access address of the access request.

Further, the exit node receives a second access request forwarded by the routing node; the second access request is sent to the routing node by the service node; the second access request is for accessing an external network.

And determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request.

Optionally, the egress node determines whether the service node sending the second access request has an access right to access the target URL address; if the access right to the target URL address is available, the second access request is determined to be a valid request of the distributed system.

And if the second access request is a valid request of the distributed system, sending the second access request to the external network so as to send the second access request to the target URL address through the external network.

In addition, the exit node also counts request response parameters of the access request corresponding to the target URL address; wherein the request response parameters include: the response duration, response success times and response failure times of the target URL address responding to the access request; based on the request response parameter, a processing priority of the access request to access the target URL address is adjusted.

Further, after the external network performs service processing, the return result is sent to the egress node, the egress node judges whether the request result is processed normally, and if the judgment result is processed normally, the request result is sent to the routing node.

Further, the routing node receives a return result of the access request, acquires a service node sending the access request, and sends the return result to the service node; the message format also needs to be reassembled for the returned result before sending it to the service node.

Further, after sending the second access request to the routing node, the service node starts timing, determines whether a return result is received within a preset time period, if so, analyzes the return result, and if not, determines that the access is overtime, and may reinitiate the access request.

Optionally, the service node is provided with multiple threads, and after the access request is sent, the service node controls the thread corresponding to the access request to be suspended, and if a return result is received within a preset time period, the service node wakes up the thread.

Specifically, after the service node sends the access request, if the return result of the access request is not received within a certain period of time, it may be considered that the access is failed, and at this time, in order to improve the service processing efficiency, the access request is immediately re-initiated to return the result as soon as possible. In addition, the service node can realize that multiple threads simultaneously process multiple access requests, the threads are not influenced, when a certain access request does not receive a return result, the thread corresponding to the access request is suspended, time slices are not distributed to the thread any more, and therefore system overhead is saved.

The service node is used for providing various services, such as login, payment, batch processing, interface service and the like; each service node can be dispersedly arranged on a plurality of servers, and each service node is in communication connection with the routing node. The service node can send the access request to other service nodes or external networks, and can also receive the access request and provide related return results.

The functions of the service node, which are usually associated with the functions of the distributed system, may include a login node, a payment node, a communication node, etc. if the distributed system is a shopping website; if the distributed system is an educational website, the service nodes may include login nodes, check-in nodes, test nodes, and the like.

It should be noted that, if the data amount or the computation amount of a certain function is large, the same function may be implemented by multiple service nodes, and at this time, the multiple service nodes may need to implement a load balancing function, so as to improve the service processing efficiency.

Referring to fig. 4, a specific implementation flow of the internal call external interface according to an embodiment of the present invention is described below by taking Lpms-api (application program interface) as an example of a target service node, where another embodiment of a data transmission method of a distributed system according to an embodiment of the present invention includes:

first, the Lpms-api (application program interface) starts the bis-client, where Lpms-api is some predefined interface, such as: functions, http interfaces, interfaces are a tool, specification and protocol for adaptation between two things.

Then, call BIS-client through BIS RPC Service (remote Service call scheme), wherein RPC adopts client/server mode, the request program is a client, the Service program is a server, the client calls the process to send a call message with process parameters to the Service process, wait for response message, then the Service end waits until the call message arrives, when the call message arrives, the server obtains the parameters, calculates the result, sends back the reply message, continues to wait for the next message to arrive, finally, the client receives the reply message, obtains the reply result, and continues to execute.

Then, a Bis Client Server Process (Client/Server Process) is called, and the Process is a program already running in the computer and is a container of a thread. The thread is the minimum unit that the operating system can perform operation scheduling, and is the actual operation unit in the process. A thread refers to a single sequential control flow in a process, where multiple threads can be concurrently executed, each performing a different task.

After the Bis Client Server Process is called, the message is assembled by the Lpms-api of the Client, and when the message format is assembled, the message can be assembled according to the message format requirement of the routing node, or according to the message format requirement of the service receiving the access request. And searching a Bis-client list, sending the request message to a Bis-router (router) and waiting for 60s to return a result.

Further, after receiving the message Request, the Bis-router parses the Request data through the Bis Route Process, then searches for Request receiver information according to the service Id by a Req Process Action (Request Process Action Request Process operation), reassembles the message, and sends the message to the Bis-client-out (client outlet) after the message assembly is completed.

Further, the Bis-Client-out uses a Bis Client Process (Client Process) to analyze the request message sent by the Bis-router, and then checks the configured processing class information according to the analyzed service Id, dynamically loads the service implementation class, and calls the external system. In the Service interface, only the statement of each method is defined, and the implementation of the method is completed in the Service implementation class of the server, and the Service implementation class must inherit the Remote Service Servlet class and implement the corresponding Service interface. Each Service is finally realized through servlets, but the Service realization class does not directly inherit the Http Servlet class, but realizes the interaction and serialization with the client by inheriting the Remote Service Servlet class.

That is, after the Bis-client-out analyzes the request message sent by the Bis-router, the configured processing class information is checked according to the analyzed Service Id, the Remote Service Servlet class is inherited in the running process of the distributed system, the corresponding Service interface is realized, and then the external network is called.

Further, the external system performs service processing and returns a result to the Bis-client-out.

After receiving the returned result, the Bis-client-out processes the returned result, then judges whether the request is processed normally, if not, the process is finished; if the processing is normal, the request result is sent to the Bis-router through the Bis Client process.

After receiving the request result, the Bis-router analyzes the request content of the Bis-client-out through a Res Process Action (Response Process operation), searches the original sender of the request, reassembles the message, and returns the result to the service requester.

The Lpms-api analyzes Bis Trace Num according to a return result, finds an initial thread through a Bis Client Server Process and wakes up the initial thread, then judges whether the thread is woken up within 60s, if not, the interface access is overtime and ends; and if the message is awakened within 60s, analyzing the message and returning, and ending.

In the embodiment of the invention, a distributed system is provided with a plurality of service nodes, routing nodes, inlet nodes and outlet nodes; each service node is respectively in communication connection with the routing node; the inlet node and the outlet node are respectively connected with the routing node in a communication mode. And the service node accesses the access request of the external network through the entrance node. The ingress node is also used to monitor and identify the format of access requests sent by external networks. The access request is identified and rejected at the entry node aiming at invalid requests and malicious requests, in addition, the entry node can also monitor the flow of access sent by an external network, and when the flow is higher than a preset flow threshold value, the flow limiting processing is carried out on the access request, so that the service efficiency is improved, and the safety of service and data is protected. The service node accesses the external network through the exit node, the exit node registers a URL address in advance, whether the access request has access authority is determined according to the pre-registered URL information, if so, the access request can be sent to the external network after being encrypted, and if not, the access request is rejected. The exit node is only used for processing the access request of the service node in the distributed system for accessing the external network, and the exit node and the entrance node are separately arranged, so that the problems of low request processing efficiency and even request blocking caused by the fact that the same node simultaneously processes the two-way access request can be avoided.

With reference to fig. 5, the data transmission method of the distributed system in the embodiment of the present invention is described above, and a data transmission device of the distributed system in the embodiment of the present invention is described below, where an embodiment of the data transmission device of the distributed system in the embodiment of the present invention includes:

a first receiving module 501, configured to receive a data access request from an external network, and determine whether the data access request is a valid request of a distributed system according to a format of the data access request;

a first sending module 502, configured to, if the first access request is a valid request of the distributed system, send the first access request to a target routing node corresponding to the target service node according to the target service node accessed by the first access request, forward the first access request to the target service node through the target routing node, and respond to the first access request by the target service node.

Referring to fig. 6, another embodiment of the data transmission apparatus of the distributed system in the embodiment of the present invention includes:

a first sending module 502, configured to, if the first access request is a valid request of the distributed system, send the first access request to a target routing node corresponding to the target service node according to the target service node accessed by the first access request, forward the first access request to the target service node through the target routing node, and respond to the first access request by the target service node. Optionally, the first receiving module 501 may be further specifically configured to:

and judging whether the format of the first access request is matched with the legal format, and if so, determining that the first access request is a valid request of the distributed system.

Optionally, the data transmission apparatus of the distributed system may further include:

a monitoring module 503, configured to monitor a real-time request traffic of a first access request from an external network; and determining whether the real-time request flow is higher than a preset flow threshold, and if the real-time request flow is higher than the flow threshold, putting the received first access request into a preset queue component so as to perform flow limitation processing on the first access request through the queue component.

Referring to fig. 7, another embodiment of the data transmission apparatus of the distributed system in the embodiment of the present invention includes:

a second receiving module 701, configured to receive a second access request forwarded by the routing node; the second access request is sent to the routing node by the service node; the second access request is used for accessing an external network;

a validity determination module 702, configured to determine whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request;

a second sending module 703, configured to send the second access request to the external network if the second access request is a valid request of the distributed system, so as to send the second access request to the target URL address through the external network.

Referring to fig. 8, another embodiment of the data transmission decoration of the distributed system in the embodiment of the present invention includes:

Optionally, the validity determining module 702 may be further specifically configured to:

judging whether the service node sending the second access request has an access right for accessing the target URL address; and if the target URL address has the access right, determining that the second access request is a valid request of the distributed system.

a counting module 704, configured to count request response parameters of the access request corresponding to the target URL address; wherein the request response parameters include: the target URL address responds to the response duration, response success times and response failure times of the access request; adjusting a processing priority of an access request accessing the target URL address based on the request response parameter.

Fig. 5 to 8 describe the data transmission apparatus of the distributed system in the embodiment of the present invention in detail, and the data transmission device of the distributed system in the embodiment of the present invention is described in detail in terms of hardware processing.

Fig. 9 is a schematic structural diagram of a data transmission device of a distributed system according to an embodiment of the present invention, where the data transmission device 900 of the distributed system may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) storing an application 933 or data 932. Memory 920 and storage media 930 may be, among other things, transient storage or persistent storage. The program stored in the storage medium 930 may include one or more modules (not shown), each of which may include a series of instruction operations in the data transmission device 900 of the distributed system. Still further, the processor 910 may be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the distributed system data transmission apparatus 900.

The data transmission apparatus 900 of the distributed system may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input-output interfaces 960, and/or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and so forth. Those skilled in the art will appreciate that the data transmission device architecture of the distributed system shown in fig. 9 does not constitute a limitation of the data transmission device of the distributed system, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when executed on a computer, cause the computer to perform the steps of the data transmission method of the distributed system.

The present invention further provides a data transmission device of a distributed system, where the data transmission device of the distributed system includes a memory and a processor, and the memory stores instructions, and the instructions, when executed by the processor, cause the processor to execute the steps of the data transmission method of the distributed system in the foregoing embodiments.

Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A data transmission method of a distributed system is characterized in that the method is applied to an entrance node in the distributed system; the distributed system also comprises a routing node and a service node; the entry node and the service node are respectively connected with the routing node; the data transmission method of the distributed system comprises the following steps:

receiving a data access request from an external network, and determining whether the data access request is a valid request of the distributed system according to the format of the data access request;

if the first access request is a valid request of the distributed system, according to a target service node accessed by the first access request, the first access request is sent to a target routing node corresponding to the target service node, so that the first access request is forwarded to the target service node through the target routing node, and the target service node responds to the first access request.

2. The data transmission method of the distributed system according to claim 1, wherein a legal format corresponding to the distributed system is pre-stored in the ingress node;

the step of determining whether the first access request is a valid request of the distributed system according to the format of the first access request comprises:

3. The data transmission method of the distributed system according to claim 1, wherein the method further comprises:

monitoring real-time request traffic of a first access request from an external network;

and determining whether the real-time request flow is higher than a preset flow threshold, and if the real-time request flow is higher than the flow threshold, putting the received first access request into a preset queue component so as to perform flow limitation processing on the first access request through the queue component.

4. The data transmission method of a distributed system, characterized by, the said method is applied to the exit node in the distributed system; the distributed system also comprises a routing node and a service node; the exit node and the service node are respectively connected with the routing node; the data transmission method of the distributed system comprises the following steps:

receiving a second access request forwarded by the routing node; wherein the second access request is sent by the serving node to the routing node; the second access request is used for accessing an external network;

determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request;

and if the second access request is a valid request of the distributed system, sending the second access request to an external network so as to send the second access request to the target URL address through the external network.

5. The data transmission method of the distributed system according to claim 4, wherein the egress node stores in advance a service node having an access right to access a specified URL address;

the step of determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request includes:

judging whether the service node sending the second access request has an access right for accessing the target URL address;

and if the target URL address has the access right, determining that the second access request is a valid request of the distributed system.

6. The data transmission method of the distributed system according to claim 4, wherein after the step of sending the second access request to an external network if the second access request is a valid request of the distributed system, the method further comprises:

counting request response parameters of the access request corresponding to the target URL address; wherein the request response parameters include: the target URL address responds to the response duration, response success times and response failure times of the access request;

adjusting a processing priority of an access request accessing the target URL address based on the request response parameter.

7. The data transmission device of the distributed system is characterized in that the device is arranged at an entrance node in the distributed system; the distributed system also comprises a routing node and a service node; the entry node and the service node are respectively connected with the routing node; the data transmission device of the distributed system comprises:

the first receiving module is used for receiving a data access request from an external network and determining whether the data access request is a valid request of the distributed system or not according to the format of the data access request;

a first sending module, configured to, if the first access request is a valid request of the distributed system, send the first access request to a target routing node corresponding to a target service node according to the target service node accessed by the first access request, forward the first access request to the target service node through the target routing node, and respond to the first access request by the target service node.

8. The data transmission device of the distributed system is characterized in that the device is arranged at an exit node in the distributed system; the distributed system also comprises a routing node and a service node; the exit node and the service node are respectively connected with the routing node; the data transmission device of the distributed system comprises:

a second receiving module, configured to receive a second access request forwarded by the routing node; wherein the second access request is sent by the serving node to the routing node; the second access request is used for accessing an external network;

the validity determining module is used for determining whether the second access request is a valid request of the distributed system according to the target URL address accessed by the second access request;

and the second sending module is used for sending the second access request to an external network if the second access request is a valid request of the distributed system so as to send the second access request to the target URL address through the external network.

9. A data transmission device of a distributed system, characterized in that the data transmission device of the distributed system comprises: a memory and at least one processor, the memory having instructions stored therein;

the at least one processor invokes the instructions in the memory to cause a data transfer device of the distributed system to perform the data transfer method of the distributed system of any of claims 1-6.

10. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement a data transmission method of a distributed system according to any one of claims 1 to 6.