CN118138622A - Node. Js RPC framework implementation method based on Fury serialization and soft communication protocol - Google Patents
Node. Js RPC framework implementation method based on Fury serialization and soft communication protocol Download PDFInfo
- Publication number
- CN118138622A CN118138622A CN202410422409.6A CN202410422409A CN118138622A CN 118138622 A CN118138622 A CN 118138622A CN 202410422409 A CN202410422409 A CN 202410422409A CN 118138622 A CN118138622 A CN 118138622A
- Authority
- CN
- China
- Prior art keywords
- serialization
- request
- service
- fury
- communication protocol
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 64
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000004044 response Effects 0.000 claims abstract description 70
- 238000005538 encapsulation Methods 0.000 claims abstract description 8
- 230000006835 compression Effects 0.000 claims description 31
- 238000007906 compression Methods 0.000 claims description 31
- 241000533950 Leucojum Species 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 238000011161 development Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 9
- 238000012360 testing method Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 101001068027 Homo sapiens Serine/threonine-protein phosphatase 2A catalytic subunit alpha isoform Proteins 0.000 description 1
- 102100034464 Serine/threonine-protein phosphatase 2A catalytic subunit alpha isoform Human genes 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/133—Protocols for remote procedure calls [RPC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/03—Protocol definition or specification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Communication Control (AREA)
Abstract
The invention discloses a node. Js RPC framework implementation method based on Fury serialization and soft communication protocol, which comprises the following steps: s1, a service provider registers a service to a local place, and a service calling party establishes connection with the service provider based on the registered service; s2, the service calling party obtains a request message sent to the service provider based on a soft communication protocol Fury in a serialization encapsulation mode; s3, the service provider calls a service method based on the received request message, and obtains a response message sent to the service caller based on soft communication protocol and Fury serialization encapsulation; s4, the service calling party acquires a response body from the response message based on the soft communication protocol and Fury serialization, and then a service calling result is obtained. The invention provides a node. Js RPC framework implementation method based on Fury serialization and soft communication protocol, which adopts a custom soft protocol to greatly reduce the volume of a request message or a response message, improves the transmission efficiency, and greatly improves the development efficiency of users through Fury serialization.
Description
Technical Field
The present invention relates to the field of distributed systems. More particularly, the present invention relates to a node. Js RPC framework implementation based on Fury serialization and soft communication protocols.
Background
Currently, the development of distributed systems is increasingly rapid, and node. Js becomes an important choice for building high-performance, scalable distributed applications. In this context, the RPC framework has received attention as a core technology for implementing inter-service communication in a distributed system. However, the existing RPC framework of node. Js still faces a series of challenges in large-scale distributed systems. Particularly, in the aspects of serialization and network communication, the traditional framework has low usability, is difficult to meet the requirements of convenient development and efficient communication, is most commonly used as gRPC framework developed by google and threft. Js framework developed by Facebook, the first bottom layer is based on a protobuf serialization framework, the second bottom layer is based on a threft serialization framework, and the two frames are static serialization frameworks, and are required to be written in advance. The two serialization modes can not directly generate serialization description according to the object to develop cross-language and process for the application, so that the use cost and the learning cost of developers are greatly increased.
In addition, for network communication protocols, the method for implementing the RPC framework compatible with HTTP requests proposes to use HTTP protocols in a compatible manner, but the HTTP protocols and the HTTPs protocols have the defects of high delay, larger resource consumption, higher serialization overhead and the like compared with the communication based on binary protocols, and the use of such protocols by the RPC framework causes efficiency problems.
To sum up, the current node. Js RPC framework implementation method mainly has the following problems:
In the existing cross-language RPC framework based on node. Js, protobuf serialization or threft serialization is mainly used for serialization, and the two serialization modes can increase the use cost and learning cost of developers.
Whereas HTTP protocol, HTTPs protocol, etc. are common problems compared to binary protocol based communications: high delay, larger resource consumption, higher serialization cost and the like.
Disclosure of Invention
It is an object of the present invention to address at least the above problems and/or disadvantages and to provide at least the advantages described below.
To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, there is provided a node. Js RPC framework implementation method based on Fury serialization and soft communication protocols, including:
s1, a service provider registers a service to a local place, and a service calling party establishes connection with the service provider based on the registered service;
s2, the service calling party obtains a request message sent to the service provider based on a soft communication protocol Fury in a serialization encapsulation mode;
s3, the service provider calls a service method based on the received request message, and obtains a response message sent to the service caller based on soft communication protocol and Fury serialization encapsulation;
S4, the service calling party acquires a response body from the response message based on the soft communication protocol and Fury serialization, and then a service calling result is obtained.
Preferably, the soft communication protocol is a customized protocol based on TCP, and the header information in the soft communication protocol includes: magic fingers of 4 bytes, protocol version number of 1 byte, header length of 2 bytes, total length of 4 bytes, request type of 1 byte, serialization type of 1 byte, compression type of 1 byte, request or response id of 8 bytes, description length of 4 bytes.
Preferably, the request message includes a request header and a request body; the response message comprises a response head and a response body;
Wherein, the request head and the response head both comprise head information;
The request body and the response body comprise: a service name for characterizing a specific service in the service provider, a method name for characterizing a mode in which the service provider finds the specific service, and a description string for deserializing serialization;
The request body further comprises: providing a parameter list of specific service incoming parameters called by a service provider;
the response body further comprises: and the returned value of the remote calling result of the service calling party is represented.
Preferably, in S2, the acquiring procedure of the request message includes:
S21, a writing request body service calling party writes a request body;
S22, the service calling party creates a Proxy object based on Proxy to intercept a method of service call, and when the Proxy object intercepts the called method, the following request header is constructed:
Request numbers generated by a snowflake algorithm;
The request type is used for distinguishing the common type from other types in a marked manner;
Serialization type, carrying out Fury serialization type and other serialization types by a marking mode;
Compression types, which are distinguished by a marking mode;
a serialization description number generated by a snowflake algorithm is adopted;
S23, the proxy object of the service calling party encapsulates the request header of S22 and the request body of S21 according to the soft protocol to obtain a request message.
Preferably, in S23, the manner of encapsulating the request message is as follows:
s230, a Buffer module based on node. Js allocates a Buffer area with 38 bytes as a request head;
s231, sequentially writing soft magic fingers, protocol version numbers and head lengths in the first 7 bytes of the buffer area;
s232, sequentially writing a request type, a serialization type, a compression type, a request id and a description id in the 12 th to 30 th bytes of the buffer;
s233, generating a description character string of the request body based on Fury serialization, obtaining a byte stream A by the description character string serialization request body, and obtaining a byte stream B by Fury serialization of the description character string;
S234, splicing the byte stream B to the rear of the byte stream A and compressing to obtain a byte stream C;
S235, splicing the byte stream C to the back of the request header message;
s236, writing the length of the uncompressed byte stream A in the 31 st to 34 th bytes of the request head;
s237, writing the total length of the message to complete encapsulation in the 7 th to 11 th bytes of the request head.
Preferably, in S3, the service provider invokes the service method based on the received request message by:
S30, analyzing the magic fingers in the request message to judge whether to adopt a soft protocol;
s31, analyzing the version number to judge whether the version number accords with the current version of the protocol;
S32, respectively analyzing the length of the header, the total length, the request type, the serialization type, the compression type, the request id, the serialization description number and the serialization description length;
s33, intercepting byte streams with the length of the serialization description from the 35 th byte in the request message, and generating description character strings by utilizing Fury deserialization;
s34, intercepting the rest all byte streams, and deserializing the rest all byte streams by adopting the description character strings obtained in the S33 to obtain a request body.
Preferably, in S3, the manner of encapsulating the response message is identical to the manner of encapsulating the request message.
Preferably, in S4, the proxy object of the service caller returns the returnValue value in the response body to get the RPC remote call value.
The invention at least comprises the following beneficial effects: firstly, the invention provides an RPC framework implementation method based on Fury serialization, which defines and implements a set of basic capability of serialization through Fury serialization, can quickly construct different multilingual serialization protocols based on the set of capability, and has high performance through optimization such as compiling acceleration.
Secondly, the invention designs and realizes a communication protocol soft based on TCP, which has great advantages in performance, flexibility and size compared with HTTP protocol and HTTPS protocol, and is suitable for a distributed system needing efficient communication.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
FIG. 1 is a flow chart of an RPC framework implementation method based on Fury serialization and node. Js;
FIG. 2 is a schematic diagram of a front 1/3 portion of a soft communication protocol message according to the present invention;
FIG. 3 is a schematic diagram of a 1/3 portion of a soft communication protocol message according to the present invention;
FIG. 4 is a schematic diagram of a 1/3 portion of a soft communication protocol message according to the present invention;
FIG. 5 is a schematic diagram of a soft communication protocol requester structure according to the present invention;
FIG. 6 is a schematic diagram of a soft communication protocol response body structure of the present invention;
FIG. 7 is a schematic diagram illustrating a first half generation flow of a character string according to the present invention;
FIG. 8 is a schematic diagram illustrating a second half of a string generation process according to the present invention;
FIG. 9 is a diagram illustrating byte codes of a request message according to the present invention;
FIG. 10 is a diagram of byte codes of a response message according to the present invention.
Detailed Description
The present invention is described in further detail below with reference to the drawings to enable those skilled in the art to practice the invention by referring to the description.
The invention provides a node. Js RPC framework implementation method based on Fury serialization and soft communication protocol, which mainly relates to protocol formulation, network communication and serialization technology. The method is used for solving the problems of low operation efficiency, insufficient usability and the like of the conventional node. Js RPC framework in cross-language and cross-process transmission.
The service provider registers the service locally (or may be a registry), the service caller establishes connection with the service provider according to the required service, and then the service caller packages the message according to the soft communication protocol, serializes and compresses the request body Fury and sends the request body to the service provider. After receiving the request message of the service calling party, the service provider performs decompression, fury deserialization and service execution, encapsulates the message according to the soft communication protocol after the service execution is completed, and transmits Fury serialization and compression of the service execution result to the service calling party, and the service calling party decompresses and Fury deserializes to obtain the result. The detailed steps are shown in fig. 1, and the related technology is as follows:
1. Serialization and deserialization
Serialization and deserialization are processes in which data is transferred or stored between different levels or between different systems.
Serialization is the process of converting a data structure or object into a format that can be stored or transmitted, typically converting data into a byte stream. In this process, the state, structure, and possibly type information of the object is encoded as a sequence of bytes. The serialized data may be transmitted to other systems over a network.
Deserialization is the process of re-restoring serialized data to the original data structure or object. In this process, the byte stream is parsed and restored into a corresponding data structure for reuse in the program.
Implementations are mainly through Fury serialization and deserialization.
Fury is a multi-language serialization framework that combines JIT dynamic compilation and zero-copy techniques through a set of highly optimized serialization primitives, while meeting performance, functional, and ease-of-use requirements, and providing exceptional performance.
In addition, key features of Fury are:
(1) Serializing cross-language object graphs: the cross-language automatic serialization of any common type does not need to create IDL files, manually compile schema generation codes and convert objects into an intermediate format like protobuf or thorft, can automatically serialize shared references and cyclic references, does not need to care about data repetition or recursive errors, can support object references and polymorphic functions which are not supported by protobuf and thorft, and can automatically generate corresponding serialization descriptions and serialize even complex objects according to the objects.
(2) Multiprotocol support: fury also supports the cross-language capability of static serialization when considering the flexibility and usability of dynamic serialization, i.e. IDL files can be created first like protobuf or thraft, and then objects are serialized according to the IDL files.
The service caller and service provider may use Fury serialization to dynamically convert objects into byte streams, or may directly convert Fury serialized byte streams into objects across languages.
2. Soft communication protocol based on TCP
TCP is a connection-oriented, reliable, byte-stream based transport layer protocol. It is located above the network layer in the OSI model and is responsible for providing reliable data transfer services, ensuring data integrity, sequencing and reliability. Whereas the conventional HTTP/HTTPs protocol based on TCP is usually performed in a text format or a more complex binary format, and the default is to be short-chained, three handshakes are required to be performed for each request, and the service call scenario has higher requirements on the efficiency and the size of data transmission, so that some performance loss may be caused compared with the custom protocol based on TCP. In the RPC framework, the service call scene has higher requirements on transmission efficiency, and the direct use of a custom protocol based on TCP is more efficient.
To solve this problem, a TCP-based soft communication protocol is designed, which removes any header information and text format that are irrelevant to service call, and adds fields such as request type, serialization type, compression type, request/response id, etc. so as to facilitate remote call by service caller and service provider better. The soft communication protocol request message includes a request header and a request body. The header information includes a magic finger (4 bytes), a version number (1 byte), a header length (2 bytes), a total length (4 bytes), a request type (1 byte), a serialization type (1 byte), a compression type (1 byte), a request/response id (8 bytes), a description id (8 bytes), and a description length (4 bytes). The request body only contains service names, method names, parameter lists and corresponding serialized anti-serialized description strings. The response body only contains the service name, the method name, the return value and the corresponding serialized anti-serialized description string. As shown in particular in figures 2,3,4,5 and 6.
The meaning of the various fields of the soft communication protocol request header is explained in detail below:
Magic finger field: 4 bytes to distinguish soft from other protocols.
1. Version number field: takes up 1 byte for distinguishing versions of the soft communication protocol.
2. Header length field: and occupies 2 bytes, for recording the request header length of the soft communication protocol.
3. Total length field: 4 bytes for recording the total length of the request header and the request body.
4. Request type field: and 1 byte is used for distinguishing whether the normal request is a normal request for remotely calling the service or a heartbeat request, wherein the heartbeat request is a request for judging whether the socket long connection of the service calling party and the service provider is disconnected.
5. Serialization type field: when a service caller invokes the service of a service provider, the field is used for telling the service provider the serialization mode used by the binary stream of the request body transmitted by the service provider, so that the service provider can use the correct serialization mode to deserialize the request body.
6. Compression type field: when a service caller calls the service of a service provider, the field is used for telling the service provider the compression mode used by the binary stream of the request body transmitted by the service provider, so that the service provider can conveniently decompress the request body by using the correct compression mode.
7. Request/response id field: 8 bytes, used in the asynchronous RPC call, to match the client response to the original request.
8. Description id field: and 8 bytes used for searching the function corresponding to the serialization from the Fury serialization description factory by the server or adding the serialization function to the serialization description factory.
9. Description length field: 4 bytes for storing Fury the length of the serialization description.
The meaning of the various fields of the soft communication protocol request body or response body is explained in detail below:
1. Description string: the binary stream excluding the description character string is used for deserializing the received request body binary stream or the response body binary stream by the service provider or the service caller.
2. Service name: for the service provider to find a specific service.
3. The method comprises the following steps: a specific method for a service provider to find a specific service.
4. Parameter list: and the parameters are transmitted when the service provider calls the specific method of the specific service.
5. Return value: and (5) a result of remote calling by the service calling party.
The soft communication protocol is simpler and more efficient than the HTTP or HTTPS protocol, and does not need to pass through an HTTP or HTTPS proxy or gateway, so that the complexity and delay of an intermediate link can be reduced, and the soft communication protocol is more suitable for efficient call with a service.
Examples:
in a specific implementation, the present invention may be described in terms of the following steps:
1. The service caller composes a request body (requestPayload). The request body contains the following fields:
(1) Service name (serviceName): for routing to a specific service of the service provider.
(2) Method name (methodName): a specific method for routing to a service.
(3) Parameter list (argumentsList): when a specific service method is found, it is executed as a parameter entry method.
(4) Description string (description string): it is not defined here that the objects are automatically generated from the objects at the time of request body serialization for Fury serialization and serialization of the objects.
2. The Proxy object is created by the service caller to intercept the method of service call, and when the Proxy object intercepts the call of the method, the request is constructed. The request contains a request header and a request body, specifically the following fields:
(1) Request number (requestId): and generating a unique request number by adopting a snowflake algorithm, so as to ensure that no repetition occurs.
(2) Request type (requestType): with 1 representing a normal request, other numbers represent other types of requests, such as 2 representing heartbeat requests, etc., in order to guarantee request extensibility.
(3) Serialization type (serializeType): to ensure serialization scalability, the use of 2 indicates Fury serialization types, and other numbers indicate other types of serialization, such as1 indicates JSON serialization, etc.
(4) Compression type (compressType): and 1 is used for indicating the GZIP compression type and reducing the volume of a request message, and other numbers indicate other compression modes, such as 2 indicates a deflate compression type and the like, in order to ensure the expansibility of the compression modes.
(5) Serialization description number (descriptionId): and a snowflake algorithm is adopted to generate a serialization description number, so that the uniqueness of the serialization description is ensured.
(6) Request body (requestPayload): is the requestor we define in step 1.
3. The proxy object of the service calling party encapsulates the request message according to the soft communication protocol. The specific operation is as follows:
(1) The Buffer module of node. Js is utilized to allocate 38 bytes of Buffer as the request header.
(2) The first 4 bytes of the buffer are written to soft magic finger, the 5 th byte is written to the protocol version number, and the 6 th and 7 th bytes are written to the header length 38.
(3) Skip 4 bytes, 12 th byte write request type, 13 th byte write Fury serialization type 1, 14 th byte write GZIP compression type 1, 15 th to 22 th byte write request id, 23 rd byte to 30 th byte write description id.
(4) And generating a description character string of the request body by Fury serialization, and serializing the request body into a byte stream A according to the description character string, wherein the specific description character string generation mode is shown in fig. 7-8.
(5) The string Fury is also serialized into byte stream B as described.
(6) Splicing the byte stream B to the rear of the byte stream A, performing GZIP compression, and splicing the compressed byte stream to the rear of the request header message.
(7) The length of uncompressed byte stream a is written at the 31 st to 34 th bytes of the request header.
(8) The total length of the message is written in bytes 7 to 11 of the request header.
4. The proxy object of the service caller sends the request message stream to the service provider.
5. And the service provider receives the request message stream of the service caller and then analyzes the message. The specific operation is as follows:
(1) Reading the 1 st to 4 th bytes, analyzing the magic finger and judging whether the magic finger belongs to the soft communication protocol.
(2) Reading the 5 th byte, analyzing the version number and judging whether the current version of the protocol is met.
(3) Reading the 6 th byte and the 7 th byte, and analyzing the length of the head; reading the 8 th to 11 th bytes, and analyzing the total length; reading the 12 th byte and analyzing the request type; reading the 13 th byte and analyzing the serialization type; reading the 14 th byte and analyzing the compression type; reading the 15 th to 22 th bytes, and analyzing the request id; reading the 23 rd to 30 th bytes, and analyzing the serial description numbers; the 31 st to 34 th bytes are read, and the serialization description length is parsed.
(4) A byte stream of the serialized description length size is intercepted starting at byte 35 and deserialized with Fury to generate a description string.
(5) The remaining full byte stream is intercepted and de-sequenced into the request body using the description string of step 4.
6. The service provider finds the corresponding service and method according to the service name and method name in the request body, and transmits the parameters in the request body into the method of the service, so that the service calling result can be obtained.
7. The service provider composes a response body (responsePayload). The request body contains the following fields:
(1) Service name (serviceName): for telling the service caller which service to call results.
(2) Method name (methodName): for telling the service caller what method of the service the call result is.
(3) Return value (returnValue): after the service method is executed, a specific return value is obtained.
(4) Description string (description string): there is no definition here, which is automatically generated from objects when the response body is serialized.
8. The service provider composes the response. The request contains a response header and a response body, specifically the following fields:
(1) Response number (requestId): and the request number of the request body is consistent.
(2) Response type (responseType): with 20 representing a successful response, other numbers represent other types of requests, such as 21 representing a heartbeat response, etc., in order to ensure response extensibility.
(3) Serialization type (serializeType): to ensure serialization scalability, the use of 2 indicates Fury serialization types, and other numbers indicate other types of serialization, such as1 indicates JSON serialization, etc.
(4) Compression type (compressType): and 1 is used for indicating the GZIP compression type and reducing the volume of a request message, and other numbers indicate other compression modes, such as 2 indicates a deflate compression type and the like, in order to ensure the expansibility of the compression modes.
(5) Serialization description number (descriptionId): and a snowflake algorithm is adopted to generate a serialization description number, so that the uniqueness of the serialization description is ensured.
(6) Response body (responsePayload): is the response body we define in step 6.
9. The service provider encapsulates the response message according to the soft communication protocol. The specific operation is as follows:
(1) The Buffer module of node. Js is utilized to allocate 38 bytes of Buffer as the response header.
(2) The first 4 bytes of the buffer are written to soft magic finger, the 5 th byte is written to protocol version number 1, and the 6 th and 7 th bytes are written to header length 38.
(3) 4 Bytes are skipped, namely, 12 th byte write response type 1, 13 th byte write Fury serialization type 1, 14 th byte write GZIP compression type 1, 15 th to 22 th byte write response numbers, 23 rd byte to 30 th byte write description id.
(4) And generating a description character string of the request body by Fury serialization, and serializing the request body into a byte stream A according to the description character string, wherein the specific description character string generation mode is shown in fig. 7.
(5) The string Fury is also serialized into byte stream B as described.
(6) Splicing the byte stream B to the rear of the byte stream A, performing GZIP compression, and splicing the compressed byte stream to the rear of the request header message.
(7) The length of byte stream a is written at bytes 31 to 34 of the response header.
(8) The total length of the message is written in bytes 7 to 11 of the response header.
10. The service provider sends the response message stream to the service caller.
11. And the proxy object of the service calling party receives the request message stream of the service provider and then analyzes the message. The specific operation is as follows:
(1) Reading the 1 st to 4 th bytes, analyzing the magic finger and judging whether the magic finger belongs to the soft communication protocol.
(2) Reading the 5 th byte, analyzing the version number and judging whether the current version of the protocol is met.
(3) Reading the 6 th byte and the 7 th byte, and analyzing the length of the head; reading the 8 th to 11 th bytes, and analyzing the total length; reading the 12 th byte and analyzing the response type; reading the 13 th byte and analyzing the serialization type; reading the 14 th byte and analyzing the compression type; reading the 15 th to 22 th bytes, and analyzing the request id; reading the 23 rd to 30 th bytes, and analyzing the serial description numbers; the 31 st to 34 th bytes are read, and the serialization description length is parsed.
(4) Byte streams of the serialized description length size are intercepted starting at byte 34 and deserialized with Fury to generate a description string.
(5) The remaining total byte stream is intercepted and de-sequenced into a response body using the description string of step 4.
12. And returning returnValue values in the response body by the proxy object of the service calling party, and obtaining the remote calling value of the R PC.
The following is a specific embodiment of the method.
1. Assuming that the service provider (with ip address 192.168.1.1 and port 8088) provides a service (here, the service is a class) with service name com. Nodejs. Test. Hellon oomiRpc, a method is defined in the service, the method name sayHello, which requires that the incoming parameter be a string type message, and the method returns the incoming message. Then define a collection as key, store the service name as value into the collection, define a TCP server finally and start up this server.
2. If the parameter passed by the service caller is a "Hello, RPC" string, the service caller packages the request body according to invention step 1 as { serviceName: "com.nodes.test.hellonoomi RPC", methodName: "sayHello", argumentsList: [ "Hello, RPC" ] }.
3. The service calling party firstly generates a request number by utilizing a snowflake algorithm, wherein the request number is as follows: 286220748778127360 the resulting request according to invention step 2 is {requestId:286220748778127360,reques tType:1,serializeType:2,compressType:1,descriptionId:286220748778127361,requestPayload:{serviceName:"com.nodejs.Test.HelloNoomiRpc",methodName:sayHello",argumentsList:["Hello,RPC]}}, where requestType:1 represents a normal request, serializeType:2 represents Fury serialization, compressType:1 represents GZIP compression.
4. The service caller generates Fury a description string for the request body according to invention step 3 as ：{"typ e":256,"label":"object","options":{"props":{"serviceName":{"type":13,"label":"string"},"methodName":{"type":13,"label":"string"},"ar gumentsList":{"type":25,"label":"array","options":{"inner":{"type":13,"label":"string"}}}},"tag":"286220748778127361"}}, the request body sequences into byte codes according to the description string Fury and the string Fury also sequences into byte codes, encapsulates the request message according to soft communication protocol, and sends the message to the service provider, the request message is shown in fig. 9.
5. The service provider analyzes the message according to the soft communication protocol and the invention step 5, and packages the request of the service caller again, and the result is as follows ：{requestId:286220748778127360,requ estType:1,serializeType:2,compressType:1,descriptionId:286220748778127361,requestPayload:{serviceName:"com.nodejs.Test.HelloNoomiRpc",methodName:"sayHello",argumentsList:["Hello,RPC"]}}.
6. The service provider finds the corresponding class in the set according to the serviceName of requestPayload, "com. Nodejs. Test. HellonomiRpc," finds the corresponding method of the class according to methodName, "sayHello," and sends the parameters into the method according to argumentsList: [ "Hello, RPC" ], thus obtaining the result "Hello, RPC".
7. The service provider writes a response body according to the step 7 of the invention as follows: { serviceName: "com.node js.test.HelloNoominRpc", methodName: "sayHello", returnValue: "Hello, RP C" }.
8. The service provider gets the response according to the invention step 8 as ：{{responseId:286220748778127360,{serializeType:2,compressType:1,responseType:20,responseBo dy:{serviceName:'com.nodejs.Test.HelloNoomiRpc',methodName:'sayHel lo',returnValue:'Hello,RPC'}}.
9. The service provider generates a description character string ：{"type":256,"label":"object","options":{"props":{"serviceName":{"type":13,"l abel":"string"},"methodName":{"type":13,"label":"string"},"returnV alue":{"type":13,"label":"string"}},"tag":"286244247706112984"}}, for the response body according to the invention step 9, the response body is serialized into byte codes according to the description character string Fury and the character string Fury is serialized into byte codes, then packages the response message according to the soft communication protocol and the invention step 9, and sends the response message to the service caller, and the response message is shown in fig. 10.
10. The service caller parses the response message according to invention step 11, and repackages the service provider's response, resulting in the following ：{{responseId:286220748778127360,{serializeTy pe:2,compressType:1,responseType:20,responseBody:{serviceName:'com.nodejs.Test.HelloNoomiRpc',methodName:'sayHello',returnValue:'Hell o,RPC'}}.
And extracting returnValue value 'hello.RPC' of responseBody fields through the packaged response, namely the RPC calling result.
The invention has the following effects:
1. Dynamic serialization of cross-language objects: the RPC framework does not require the creation of IDL files, manual compilation of schema generated code, and conversion of objects into intermediate formats when called across languages, as does gRPC framework using protobuf or the thrft. The user only needs to give the object to the RPC framework, the framework can automatically generate Fury serialization description of the object according to the incoming object, and the object is automatically serialized according to the serialization description, so that the user does not need to care about serialization details like other serialization modes, and the development efficiency of the user is greatly improved.
2. Efficient RPC communication protocol: aiming at the problem that the HTTP/HTTPS protocol introduces a certain overhead in communication, resulting in low performance and efficiency in RPC remote call, a TCP-based soft communication protocol is designed, which removes components irrelevant to communication between services, only contains necessary information for call between services, greatly reduces the volume of request messages or response messages, and improves the transmission efficiency.
The above is merely illustrative of a preferred embodiment, but is not limited thereto. In practicing the present invention, appropriate substitutions and/or modifications may be made according to the needs of the user.
Although embodiments of the invention have been disclosed above, they are not limited to the use listed in the specification and embodiments. It can be applied to various fields suitable for the present invention. Additional modifications will readily occur to those skilled in the art. Therefore, the invention is not to be limited to the specific details and illustrations shown and described herein, without departing from the general concepts defined in the claims and their equivalents.
Claims (8)
1. The node js RPC framework implementation method based on Fury serialization and soft communication protocol is characterized by comprising the following steps:
s1, a service provider registers a service to a local place, and a service calling party establishes connection with the service provider based on the registered service;
s2, the service calling party obtains a request message sent to the service provider based on a soft communication protocol Fury in a serialization encapsulation mode;
s3, the service provider calls a service method based on the received request message, and obtains a response message sent to the service caller based on soft communication protocol and Fury serialization encapsulation;
S4, the service calling party acquires a response body from the response message based on the soft communication protocol and Fury serialization, and then a service calling result is obtained.
2. The method for implementing a node. Js RPC framework based on Fury serialization and soft communication protocol of claim 1, wherein the soft communication protocol is a custom protocol based on TCP, and the header information in the soft communication protocol includes: magic fingers of 4 bytes, protocol version number of 1 byte, header length of 2 bytes, total length of 4 bytes, request type of 1 byte, serialization type of 1 byte, compression type of 1 byte, request or response id of 8 bytes, description length of 4 bytes.
3. The method for implementing node. Js RPC framework based on Fury serialization and soft communication protocol as claimed in claim 2, wherein the request message includes a request header and a request body; the response message comprises a response head and a response body;
Wherein, the request head and the response head both comprise head information;
The request body and the response body comprise: a service name for characterizing a specific service in the service provider, a method name for characterizing a mode in which the service provider finds the specific service, and a description string for deserializing serialization;
The request body further comprises: providing a parameter list of specific service incoming parameters called by a service provider;
the response body further comprises: and the returned value of the remote calling result of the service calling party is represented.
4. The method for implementing node. Js RPC framework based on Fury serialization and soft communication protocol as claimed in claim 3, wherein in S2, the request message acquisition procedure includes:
S21, a writing request body service calling party writes a request body;
S22, the service calling party creates a Proxy object based on Proxy to intercept a method of service call, and when the Proxy object intercepts the called method, the following request header is constructed:
Request numbers generated by a snowflake algorithm;
The request type is used for distinguishing the common type from other types in a marked manner;
Serialization type, carrying out Fury serialization type and other serialization types by a marking mode;
Compression types, which are distinguished by a marking mode;
a serialization description number generated by a snowflake algorithm is adopted;
S23, the proxy object of the service calling party encapsulates the request header of S22 and the request body of S21 according to the soft protocol to obtain a request message.
5. The method for implementing node. Js RPC framework based on Fury serialization and soft communication protocol as claimed in claim 4, wherein in S23, the manner of encapsulating the request message is:
s230, a Buffer module based on node. Js allocates a Buffer area with 38 bytes as a request head;
s231, sequentially writing soft magic fingers, protocol version numbers and head lengths in the first 7 bytes of the buffer area;
s232, sequentially writing a request type, a serialization type, a compression type, a request id and a description id in the 12 th to 30 th bytes of the buffer;
s233, generating a description character string of the request body based on Fury serialization, obtaining a byte stream A by the description character string serialization request body, and obtaining a byte stream B by Fury serialization of the description character string;
S234, splicing the byte stream B to the rear of the byte stream A and compressing to obtain a byte stream C;
S235, splicing the byte stream C to the back of the request header message;
s236, writing the length of the uncompressed byte stream A in the 31 st to 34 th bytes of the request head;
s237, writing the total length of the message to complete encapsulation in the 7 th to 11 th bytes of the request head.
6. The method for implementing node. Js RPC framework based on Fury serialization and soft communication protocol according to claim 1, wherein in S3, the service provider invokes the service method based on the received request message, which is:
S30, analyzing the magic fingers in the request message to judge whether to adopt a soft protocol;
s31, analyzing the version number to judge whether the version number accords with the current version of the protocol;
S32, respectively analyzing the length of the header, the total length, the request type, the serialization type, the compression type, the request id, the serialization description number and the serialization description length;
s33, intercepting byte streams with the length of the serialization description from the 35 th byte in the request message, and generating description character strings by utilizing Fury deserialization;
s34, intercepting the rest all byte streams, and deserializing the rest all byte streams by adopting the description character strings obtained in the S33 to obtain a request body.
7. The method for implementing a node. Js RPC framework based on Fury serialization and soft communication protocol of claim 1, wherein in S3, the manner of encapsulating the response message is identical to the manner of encapsulating the request message.
8. The node. Js RPC framework implementation method based on Fury serialization and soft communication protocol of claim 1, wherein in S4, the proxy object of the service caller returns returnValue value in the response body to get the value of RPC remote invocation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410422409.6A CN118138622A (en) | 2024-04-09 | 2024-04-09 | Node. Js RPC framework implementation method based on Fury serialization and soft communication protocol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410422409.6A CN118138622A (en) | 2024-04-09 | 2024-04-09 | Node. Js RPC framework implementation method based on Fury serialization and soft communication protocol |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118138622A true CN118138622A (en) | 2024-06-04 |
Family
ID=91240526
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410422409.6A Pending CN118138622A (en) | 2024-04-09 | 2024-04-09 | Node. Js RPC framework implementation method based on Fury serialization and soft communication protocol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118138622A (en) |
-
2024
- 2024-04-09 CN CN202410422409.6A patent/CN118138622A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112114979B (en) | Remote procedure call method and device | |
| JP5052522B2 (en) | System and method for optimization utilizing history of web service communication | |
| US8375094B2 (en) | Creating a message readable by a plurality of heterogeneous recipients | |
| JP5242887B2 (en) | Flexible transfer of typed application data | |
| CN109902274B (en) | Method and system for converting json character string into thraft binary stream | |
| US8332539B2 (en) | Binary data serialization system using machine dependant data alignment | |
| CN109815025B (en) | Service model calling method, device and storage medium | |
| US20080215613A1 (en) | Method and System For Transferring Objects Between Programming Platforms Computer Program Product Thereof | |
| JP2006209745A (en) | Method and system for binary serialization of document | |
| JP2011512730A (en) | Message encoding and decoding using template expression parameters | |
| US9715451B2 (en) | System and method for caching service results in a distributed caching system in a transactional processing environment | |
| US7404186B2 (en) | Signature serialization | |
| JP2005174120A (en) | Web service connection processing method, system, and program | |
| JP2000047964A (en) | Data transfer method on network, its device, its system, and recording medium having recorded data transfer program on network thereon | |
| CN106681734A (en) | Response method and device for function interface call | |
| JP2004046810A (en) | Dispatch method for application step in client/server environment and service portal system | |
| CN113839902A (en) | Communication protocol adaptation method, device, computer equipment and storage medium | |
| CN118138622A (en) | Node. Js RPC framework implementation method based on Fury serialization and soft communication protocol | |
| CN111338685B (en) | Common component configuration method, device, equipment and storage medium | |
| CN113495796A (en) | Service calling method and related equipment | |
| CN114500685B (en) | Third party communication library bridging method and system adapting to illusion engine application | |
| CN111314375B (en) | Medical insurance data adaptation engine, medical insurance data adaptation method, electronic equipment and storage medium | |
| CN115167869A (en) | Method, electronic device and medium for serialization and deserialization of Java object | |
| CN113849449A (en) | Communication system and information interaction method, device and medium | |
| CN113992644A (en) | Internet of things gateway system based on non-service technology and data processing method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |