CN117834701A

CN117834701A - Remote procedure call device, distributed system and electronic equipment

Info

Publication number: CN117834701A
Application number: CN202311864606.5A
Authority: CN
Inventors: 叶文广; 乔旺龙
Original assignee: Tianyun Rongchuang Data Science & Technology Beijing Co ltd
Current assignee: Tianyun Rongchuang Data Science & Technology Beijing Co ltd
Priority date: 2023-12-29
Filing date: 2023-12-29
Publication date: 2024-04-05

Abstract

The embodiment of the application discloses a remote procedure call device, a distributed system and electronic equipment, which are applied to the technical field of remote communication and can solve the problem of how to integrate all RPC protocols into one frame to realize inter-service communication in the distributed system. The remote procedure call device includes: the universal interface is used for providing a unified calling interface for each Remote Procedure Call (RPC) protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; the dynamic proxy module is used for dynamically selecting a target RPC protocol to realize the calling of the remote service; the data processing module is used for processing the data through a serialization technology and an anti-serialization technology; wherein, the RPC protocol includes: gRPC protocol, netty protocol, and Hadoop RPC protocol.

Description

Remote procedure call device, distributed system and electronic equipment

Technical Field

The embodiment of the application relates to the technical field of remote communication, in particular to a remote procedure call device, a distributed system and electronic equipment.

Background

In modern distributed systems, the remote procedure call protocol (Remote Procedure Call Protocol, RPC) framework plays a vital role, allowing efficient communication between different services. In building a large-scale distributed system, different services and applications may also employ different RPC protocols to enable communication. gRPC, netty and Hadoop RPC are three popular RPC protocols, each having its unique advantages, each adapted to different scenarios and requirements. At present, these RPC protocols are not compatible, so how to integrate all RPC protocols into one framework to implement inter-service communication in a distributed system is a problem that needs to be solved.

Disclosure of Invention

In order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present application provide a remote procedure call device, a distributed system, and an electronic device, which are used for solving the problem of how to integrate all RPC protocols into one framework to implement inter-service communication in the distributed system.

In order to achieve the above object, the technical solution provided in the embodiments of the present application is as follows:

in a first aspect, an embodiment of the present application provides a remote procedure call device, including: the universal interface is used for providing a unified call interface for each Remote Procedure Call (RPC) protocol;

a protocol adapter for providing a specific protocol call for each RPC protocol;

the dynamic proxy module is used for dynamically selecting a target RPC protocol to realize the calling of the remote service;

the data processing module is used for processing the data through a serialization technology and an inverse serialization technology;

wherein the RPC protocol comprises: gRPC protocol, netty protocol, and Hadoop RPC protocol.

As an optional implementation manner, in a first aspect of the embodiment of the present application, the protocol adaptor specifically includes:

gRPC adapter, is used for handling the data communication of crossing languages based on HTTP/2 protocol;

a Netty adapter for handling asynchronous event driven data communications based on TCP/UDP;

the Hadoop adapter is used for processing data communication in the Hadoop ecological system.

In a first aspect of the embodiments of the present application, as an optional implementation manner, the dynamic proxy module is specifically configured to receive a service call request sent by a client, and select the target RPC protocol according to the service call request.

As an optional implementation manner, in a first aspect of the embodiment of the present application, the data processing module is specifically configured to determine, according to the target RPC protocol, a serialization and anti-serialization format corresponding to the target RPC protocol, so as to perform an encoding process and a decoding process on data.

As an optional implementation manner, in the first aspect of the embodiment of the present application, the remote procedure call device further includes:

and the network communication unit is used for realizing TCP communication, UDP communication and file transmission based on a network communication framework.

And the security unit is used for realizing data encryption and identity verification.

and the error processing and monitoring unit is used for providing an error processing mechanism and a monitoring and logging function.

and the service discovery and load balancing unit is used for integrating the service discovery and load balancing mechanism so as to allocate corresponding service instances for the requests of the clients.

In a second aspect, embodiments of the present application provide a distributed system, the distributed system comprising: the remote procedure call device in the first aspect of the embodiment of the application.

In a third aspect, embodiments of the present application provide an electronic device configured with a distributed system in the second aspect of embodiments of the present application.

Compared with the prior art, the embodiment of the application has the following beneficial effects:

the embodiment of the application provides a remote procedure call device, a distributed system and electronic equipment, wherein the remote procedure call device comprises: the universal interface is used for providing a unified calling interface for each Remote Procedure Call (RPC) protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; the dynamic proxy module is used for dynamically selecting a target RPC protocol to realize the calling of the remote service; the data processing module is used for processing the data through a serialization technology and an anti-serialization technology; wherein, the RPC protocol includes: gRPC protocol, netty protocol, and Hadoop RPC protocol. The gRPC protocol, the Netty protocol and the Hadoop RPC protocol can be integrated in one frame through the device, multiple programming languages and platforms are supported, so that communication among different systems is more flexible and seamless, the overall communication efficiency is improved by utilizing HTTP/2 characteristics of gRPC and the high-performance network processing capacity of Netty, in addition, due to the provision of a uniform interface and an adapter, maintenance and future expansion work of the system are simplified, and resource consumption is reduced and response speed and throughput of the system are improved in an asynchronous and event-driven mode.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a remote procedure call device according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a universal interface of a remote procedure call device according to an embodiment of the present application;

FIG. 3 is a second schematic structural diagram of a remote procedure call device according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a dynamic proxy of a remote procedure call device according to an embodiment of the present application;

FIG. 5 is a third schematic diagram of a remote procedure call device according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a distributed system according to an embodiment of the present application;

Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to more clearly understand the foregoing objects, features and advantages of the present application, a technical solution of an embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present application, and it should be noted that, without conflict, the embodiment of the present application and features in the embodiment may be combined with each other, and it is apparent that the described embodiment is a part of the embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The terms "first" and "second" and the like in the description and in the claims, are used for distinguishing between different objects and not for describing a particular sequential order of objects.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In modern distributed systems, remote Procedure Call (RPC) frameworks play a vital role, allowing efficient communication between different services. In building a large-scale distributed system, different services and applications may employ different RPC protocols to enable communication. gRPC, netty and Hadoop RPC are three widely used RPC protocols, each having its unique advantages, each adapted to different scenarios and requirements. gRPC is a high performance, cross-language RPC framework developed by Google that supports multiple programming languages based on the HTTP/2 protocol, making communication between services more efficient and reliable. Netty is an asynchronous event driven network application framework that provides support for TCP, UDP, and file transfer, served as an underlying communication technology by many high performance RPC frameworks. The Hadoop RPC is designed specifically for Hadoop ecosystems, which allow efficient communication between Hadoop components.

The technical content involves several key aspects in designing a framework that supports both gRPC, netty and Hadoop RPC protocols. First, gRPC is a modern open source high performance RPC framework that can operate in any environment, efficiently connect services within and across data centers, and provide pluggable support for load balancing, call chain tracking, health checking, and identity verification. Netty is a high-performance, asynchronous event-driven NIO framework that provides support for TCP, UDP and file transfer, as an asynchronous NIO framework, all IO operations of Netty are asynchronous and non-blocking, and users can conveniently and actively acquire or acquire IO operation results through a Future-Listener mechanism.

In order to solve some or all of the above technical problems, an embodiment of the present application provides a remote procedure call device, a distributed system, and an electronic device, where the remote procedure call device includes: the universal interface is used for providing a unified calling interface for each Remote Procedure Call (RPC) protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; the dynamic proxy module is used for dynamically selecting a target RPC protocol to realize the calling of the remote service; the data processing module is used for processing the data through a serialization technology and an anti-serialization technology; wherein, the RPC protocol includes: gRPC protocol, netty protocol, and Hadoop RPC protocol. The gRPC protocol, the Netty protocol and the Hadoop RPC protocol can be integrated in one frame through the device, multiple programming languages and platforms are supported, so that communication among different systems is more flexible and seamless, the overall communication efficiency is improved by utilizing HTTP/2 characteristics of gRPC and the high-performance network processing capacity of Netty, in addition, due to the provision of a uniform interface and an adapter, maintenance and future expansion work of the system are simplified, and resource consumption is reduced and response speed and throughput of the system are improved in an asynchronous and event-driven mode.

As shown in fig. 1, an embodiment of the present application provides a remote procedure call device, which may specifically include:

a generic interface 11 for providing a unified call interface for each remote procedure call RPC protocol.

A protocol adapter 12 for providing a specific protocol call for each RPC protocol.

And the dynamic proxy module 13 is used for dynamically selecting a target RPC protocol to realize the call of the remote service.

A data processing module 14 for processing the data by a serialization technique and a deserialization technique.

Wherein, the RPC protocol includes: gRPC protocol, netty protocol and Hadoop RPC protocol; the target RPC protocol is any one of gRPC protocol, netty protocol and Hadoop RPC protocol.

In some embodiments, the remote procedure call device needs to define a generic RPC interface that will provide a unified call for all three protocols. This interface needs to be abstract enough to be able to adapt to the characteristics of the different protocols. The RPC interface is a generic interface 11, which can abstract details of different RPC protocols, so that details of different protocols are transparent to the end user, meaning that the user can seamlessly switch protocols without affecting the use of the service.

In the RPC protocol framework, a generic interface is a common concept. The generic interface defines a set of generic methods that can be implemented by different services. By using a generic interface, a client can invoke methods of different service providers without concern for specific service implementation details.

The generic interface typically defines a standard set of method signatures, such as the type of input parameters, the type of return values, etc. The service provider needs to implement these methods and sequence and communicate in a agreed format. The client then interacts with different service providers by invoking these methods.

The universal interface has the advantage that maintainability and expandability of the system can be improved. By defining a standard set of method signatures, different service providers can be easily replaced without modifying the client code. In addition, the universal interface can conveniently add new methods to meet new business requirements.

By using the universal interface, the abstraction of the service can be realized, so that the client and the server are decoupled more, and the flexibility and the expandability of the system are improved.

As shown in FIG. 2, the universal interface may implement asynchronous requests, synchronous requests, and exception handling.

In some embodiments, the universal interface is one of the key components for implementing asynchronous requests that allow a client to initiate a request without having to wait for the server's response, but to obtain the response at a later point in time, which may improve the concurrency performance and response speed of the system. The universal interface is an interface specification shared by the client and the server for defining a callable remote procedure. The general interface may include a method for asynchronous request and corresponding parameters, the client sends the asynchronous request to the server by calling the method defined in the general interface, without waiting for the response of the server, the server may put the asynchronous request into a queue after receiving the asynchronous request, the server may process the request in the queue by using a multithreading or asynchronous processing manner, the server may return the response to the client by means of a callback function or event notification after processing the asynchronous request, and the client may acquire the response when needed.

Specifically, the asynchronous request may be implemented by a completable eFuse < clienteply > sendRequestAsync (ClientRequest request) function, completable eFuse < clienteply > being the return type of the function, completable eFuse being a class in Java for representing the result of an asynchronous calculation that may be performed in another thread. Clienteply is the type of result of this asynchronous calculation. sendRequestAsync is the name of the function. ClientRequest request is a list of parameters for a function that receives a parameter of the type ClientRequest named request.

In some embodiments, the generic interface is also one of the key components for implementing the synchronization request. The synchronization request requires that the client must wait for the response of the server after initiating the request to continue to perform subsequent operations. Similar to asynchronous requests, a generic interface is an interface specification shared by the client and server for defining callable remote procedures. The generic interface may contain the method and corresponding parameters for the synchronization request. The client sends a synchronization request to the server by calling a method defined in the universal interface and blocks the response waiting for the server. After receiving the synchronization request, the server processes the synchronization request and returns a response to the client. The server should complete processing and return a response as soon as possible to reduce the latency of the client. And the server returns the processing result to the client through the universal interface. And after receiving the response, the client continues to execute the subsequent operation.

It should be noted that, when implementing the synchronization request, in order to avoid performance problems caused by long waiting time of the client, a reasonable timeout should be set, and after the timeout is exceeded, the client may choose to retry or discard the request. Both the server and client should perform exception handling to ensure proper handling, such as logging, returning error codes, etc., when an exception occurs. When multiple service instances provide the same service, a load balancing algorithm should be used to distribute requests to different service instances to improve the scalability and reliability of the system. When data is transmitted between a client and a server, serialization and deserialization operations are required. The selection of a proper serialization scheme can improve the efficiency and compatibility of data transmission.

Specifically, the synchronization request may be implemented by a ClientReply sendRequest (ClientRequest request) function, clienteply being the return type of the function, indicating that the function will return a clienteply object, sendRequest being the name of the function, clientRequest request being a list of parameters of the function, the function receiving a parameter of the type ClientRequest name request.

In some embodiments, exception handling of the generic interface is an important element to ensure system stability and reliability. The exception handling involves both the client and the server, and aims to properly handle when an exception occurs, avoiding system crashes or data loss. In a generic interface, a set of exception classes may be defined to represent possible error conditions, which should inherit from the generic exception base class for unified handling. When the server side encounters an abnormal condition during the request processing, a corresponding abnormal class should be thrown out, and detailed information about errors, such as error codes, error messages, stack tracking and the like, can be contained in the abnormal class. The client should capture the exception returned from the server when invoking the generic interface. The client can judge whether an error occurs by checking the return value or the abnormal object, and perform corresponding processing according to specific conditions. Both the client and the server should implement appropriate exception handling logic. At the client, logging, notifying the user, or other recovery operations may be performed after the anomaly is captured. At the server side, logs, rollback transactions, or other cleaning operations may be logged. In addition to throwing the exception, the server may also communicate error information to the client by returning an error code and an error message. The client can perform corresponding error processing according to the error code and the message.

In some embodiments, the remote procedure call device may implement the configuration of an adapter for each RPC protocol that is responsible for converting calls of the generic interface into calls of a specific protocol.

In some embodiments, as shown in fig. 3, protocol adapter 12 specifically includes:

gRPC adapter 121 for handling cross-language data communication based on HTTP/2 protocol.

The Netty adapter 122 is used for processing asynchronous event driven data communication based on TCP/UDP.

Hadoop adapter 123 is used to handle data communications in the Hadoop ecosystem.

It should be noted that, since the RPC protocol may generally include the above three protocols, namely, the gppc protocol, the Netty protocol, and the Hadoop RPC protocol, the protocol adapter may also be configured for the three RPC protocols respectively, that is, the protocol adapter includes: gRPC adapter corresponding to gRPC protocol, netty adapter corresponding to Netty protocol, and Hadoop adapter corresponding to Hadoop RPC protocol.

In some embodiments, the protocol adapter is responsible for converting calls of the generic interface into calls of a specific protocol.

Specifically, the gRPC adapter is used for converting the call of the universal interface into the call of the gRPC protocol, the Netty adapter is used for converting the call of the universal interface into the call of the Netty protocol, and the Hadoop adapter is used for converting the call of the universal interface into the call of the Hadoop RPC protocol.

In some embodiments, the dynamic proxy module is specifically configured to receive a service call request sent by the client, and select a target RPC protocol according to the service call request.

The target RPC protocol is any one of gRPC protocol, netty protocol and Hadoop RPC protocol.

It should be noted that the dynamic proxy module may be used to implement a dynamic proxy mechanism, allowing the client to call a remote service like a local service. The dynamic proxy module may dynamically select the appropriate RPC protocol based on the configuration or the requested content.

Further, since the protocol adapter has configured a corresponding protocol adapter for each RPC protocol, the dynamic proxy module will also select the protocol adapter corresponding to the target RPC protocol after selecting the appropriate target RPC protocol.

In some embodiments, in the RPC protocol framework, dynamic proxy is a common technique for implementing communication between a client and a server, and as shown in fig. 4, the main function of the dynamic proxy is to provide a middle layer, forward a request from the client to the server, and return a response from the server to the client.

The implementation of dynamic agents typically involves the following steps:

Interface definition: first, an interface is defined that is followed by both the server and the client. These interfaces define the remote procedures that the client can call, as well as the parameters and return values required for these procedures.

Agent generation: at the client, the dynamic proxy framework automatically generates proxy classes based on the interface definitions. This proxy class contains methods of communicating with the server that forward the request to the server when called.

Request forwarding: when the client calls the method of the proxy class, the proxy class packages the request into a format specified by a protocol and sends the request to the server through the network. This process typically involves serialization and network communication.

And (3) response returns: after receiving the request, the server executes corresponding operation, packages the result into a format specified by the protocol, and returns the result to the client. The proxy class of the client receives this response and parses it into a usable form for return to the client.

Through the dynamic proxy, the client can call the remote process like a local method, and the communication details of the bottom layer do not need to be concerned, so that the realization of remote call is greatly simplified, and the decoupling of the client and the server is easier.

In some embodiments, the data processing module is specifically configured to determine, according to the target RPC protocol, a serialization and anti-serialization format corresponding to the target RPC protocol, so as to perform encoding processing and decoding processing on the data.

In some embodiments, serialization and deserialization are important techniques for converting data into a format that can be transmitted over a network, or converting received data back into the original format.

Serialization is the process of converting a data structure or object into a binary string. In the RPC protocol framework, serialization is a process of converting state information of an object into a form that can be stored or transmitted, such as converting state information of an object into byte stream or XML format. This process typically involves extracting the field values of the objects and organizing them in a format to enable them to be sent over a network or stored on disk.

Deserialization is the opposite process, i.e., the process of converting a binary string back into a data structure or object. In the RPC protocol framework, inverse serialization is the re-conversion of received byte streams or XML formatted data into the form of objects. This process requires reorganizing the received data into a structured form for use by the application, in accordance with the original format specification.

The purpose of serialization and deserialization is to enable cross-platform and cross-language sharing of data. By converting data into a unified format, data exchange and communication between different applications and systems can be facilitated. Meanwhile, as the data can be encrypted and checked in the transmission process, serialization and deserialization are also beneficial to improving the safety and reliability of the data.

The various RPC Protocol frameworks can support various serialization formats, such as Protocol Buffers, JSON, XML, etc., so as to select an appropriate Protocol for data transmission and storage according to actual requirements.

In some embodiments, as shown in fig. 5, the remote procedure call device may further include: a network communication unit 15 for implementing TCP communication, UDP communication, and file transfer based on a network communication framework.

It should be noted that the network communication unit may utilize Netty as the underlying network communication framework because it provides support for TCP, UDP, and file transfer, and is asynchronous event driven, which helps to improve overall performance and scalability.

In network communications, the RPC protocol framework mainly involves the communication between a client and a server. The core principle of the RPC protocol framework is that a client and a server carry out serialized network communication according to a agreed communication protocol. In network communications, a client and a server need to establish a reliable transmission link to ensure reliable transmission of data. Common transport layer protocols include TCP and UDP. TCP is a connection oriented protocol that ensures orderly and reliable transmission of data, but may sacrifice some transmission efficiency. The UDP is a connectionless protocol, and has high transmission efficiency, but cannot guarantee reliable transmission of data. Therefore, when selecting the transport layer protocol, a trade-off needs to be made according to the actual requirements.

In network communications, the choice of network IO model also needs to be considered. Common network IO models include blocking IOs, non-blocking IOs, IO multiplexing, asynchronous IOs, and the like. Different IO models have different characteristics and application scenes, and need to be selected according to actual requirements. For example, for applications that require handling a large number of concurrent connections, the use of IO multiplexing or asynchronous IO models may be selected to increase the concurrent processing capacity of the system.

In addition, security and reliability of data also need to be considered in network communications. To ensure the security of the data, the data may be encrypted using an encryption algorithm to prevent theft or tampering of the data. To ensure the reliability of the data, a verification mechanism may be employed to verify the data to ensure the integrity and consistency of the data.

In summary, network communication in the RPC protocol framework involves a number of aspects, including the choice of transport layer protocol, the choice of network IO model, and security and reliability considerations for data. In practical application, selection and configuration are required according to specific requirements so as to achieve the optimal network communication effect.

In some embodiments, as shown in fig. 5, the remote procedure call device may further include: a security unit 16 for implementing data encryption and authentication.

It should be noted that the security element may implement security mechanisms including SSL/TLS encryption, authentication and authorization to ensure the security of RPC communications.

In some embodiments, security in the RPC protocol framework involves aspects such as data privacy, authentication, authorization, and transmission security.

Data privacy: the sensitive data is ensured not to be illegally acquired in the RPC communication process. Data privacy is protected by encryption, hashing, and the like.

And (3) identity authentication: by verifying the identities of the client and the server, it is ensured that only legitimate clients can access the service. Common authentication methods include the use of a user name and password, digital signatures, or the use of a security token, etc.

Authorization: the access control is performed on the client to ensure that it can only access the services it is authorized to use. This may be accomplished through the use of Access Control Lists (ACLs) or role-based access control (RBACs) mechanisms, or the like.

Transmission safety: and the communication safety between the client and the server is ensured. By using encryption techniques such as SSL/TLS, data in the communication process is protected from tampering or theft.

In addition, some other security issues need to be considered, such as preventing denial of service attacks (DoS attacks), preventing malicious code injection, etc.

To improve the security of the RPC framework, a series of security measures may be taken, such as using strong cryptographic policies, periodically updating and reinforcing security components, implementing security audits and monitoring, etc. Meanwhile, the latest security threat and loopholes are required to be continuously paid attention to, and the RPC framework is updated and upgraded in time so as to ensure the security of the system.

In some embodiments, as shown in fig. 5, the remote procedure call device may further include: an error handling and monitoring unit 17 for providing error handling mechanisms, and monitoring and logging functions.

It should be noted that the error handling and monitoring unit may provide error handling mechanisms, as well as monitoring and logging functions, to facilitate problem diagnosis and performance optimization.

Error handling and monitoring are important components in the RPC protocol framework that help improve system reliability and maintainability.

Error handling: the RPC protocol framework should provide an appropriate error handling mechanism to be able to handle and recover correctly when an error occurs. Common error handling approaches include: exception handling, retry, fault tolerance, transaction management, and the like. Exception handling refers to avoiding program crashes or data inconsistencies by capturing and handling exceptions. Retry refers to retrying to invoke a remote procedure when an error occurs to reduce the effects of network jitter or temporary errors. Fault tolerance refers to improving the availability and reliability of a system by designing redundant or backup systems. Transaction management refers to ensuring consistency and integrity of data by ensuring atomicity, consistency, isolation, and persistence of a series of operations.

And (3) monitoring: in order to discover and solve problems in the system in time, the RPC protocol framework should provide a monitoring function. Monitoring can help administrators and developers to learn about the system's operating conditions, performance bottlenecks, and potential errors. Common monitoring metrics include: request success rate, response time, throughput, error rate, resource utilization, etc. The RPC protocol framework should support real-time monitoring and historical data querying so that administrators can quickly locate and resolve problems. In addition, the monitoring data may also be used to analyze and optimize the performance of the system to improve the stability and reliability of the system.

In summary, error handling and monitoring in the RPC protocol framework is critical to ensuring reliability and stability of the system. By proper error processing mechanism and monitoring function, the errors and problems in the system can be effectively reduced, and the usability and performance of the system can be improved.

In some embodiments, as shown in fig. 5, the remote procedure call device may further include: a service discovery and load balancing unit 18 for integrating service discovery and load balancing mechanisms such that corresponding service instances are allocated for requests of clients.

It should be noted that the service discovery and load balancing unit may integrate service discovery and load balancing mechanisms to ensure that requests can be distributed to appropriate service instances, which may involve integration with existing service registries.

In the RPC protocol framework, service discovery and load balancing are two key components that help to improve system availability and performance.

Service discovery is a mechanism that allows clients to dynamically find and connect to available service instances at runtime. In a micro-service architecture, service instances may start and stop dynamically, so the client needs a way to discover currently available service instances.

Service discovery is typically implemented through a service registry that maintains network location information for currently available services. When a service instance starts up, it registers its own network location information with the service registry. The client obtains network location information of the available services by querying the service registry and establishes a connection with the service instance.

In addition, load balancing is a technique for distributing requests of clients over multiple service instances to achieve efficient utilization of resources and to improve performance of the system. In the RPC protocol framework, load balancing can help distribute requests, avoid overloading individual service instances, and ensure that all service instances are reasonably utilized.

Common load balancing strategies include Round Robin (Round Robin), random (Random), least connected (Least Connections), etc. These policies may be selected according to different scenarios and requirements. For example, a polling policy distributes requests to each service instance in sequence, while a random policy randomly selects one service instance to process the request. The minimum connection policy sends the request to the service instance with the minimum number of current connections.

To implement load balancing, the RPC protocol framework typically introduces a load balancer component at the client or server that is responsible for selecting an appropriate service instance to handle the client's request according to a configured load balancing policy.

The combination of service discovery and load balancing can further improve the expandability and usability of the system, through the service discovery, a client can dynamically perceive available service instances and distribute requests to the instances through the load balancing, so that the balanced utilization of resources and the high-performance operation of the system are realized.

The embodiment of the application provides a remote procedure call device, which comprises: the universal interface is used for providing a unified calling interface for each Remote Procedure Call (RPC) protocol; a protocol adapter for providing a specific protocol call for each RPC protocol; the dynamic proxy module is used for dynamically selecting a target RPC protocol to realize the calling of the remote service; the data processing module is used for processing the data through a serialization technology and an anti-serialization technology; wherein, the RPC protocol includes: gRPC protocol, netty protocol, and Hadoop RPC protocol. The gRPC protocol, the Netty protocol and the Hadoop RPC protocol can be integrated in one frame through the device, multiple programming languages and platforms are supported, so that communication among different systems is more flexible and seamless, the overall communication efficiency is improved by utilizing HTTP/2 characteristics of gRPC and the high-performance network processing capacity of Netty, in addition, due to the provision of a uniform interface and an adapter, maintenance and future expansion work of the system are simplified, and resource consumption is reduced and response speed and throughput of the system are improved in an asynchronous and event-driven mode.

As shown in fig. 6, embodiments of the present application provide a distributed system, which may include: the remote procedure call device shown in fig. 1, or the remote procedure call device shown in fig. 3, or the remote procedure call device shown in fig. 5.

As shown in fig. 7, an embodiment of the present application further provides an electronic device, where the electronic device may include: the distributed system shown in fig. 6.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In this application, the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In the present application, the memory may include volatile memory, random Access Memory (RAM), and/or nonvolatile memory in a computer readable medium, such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

In this application, it will be understood by those skilled in the art that all or part of the steps in the various methods of the above embodiments may be implemented by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, including permanent and non-permanent, removable and non-removable storage media. Storage media may embody any method or technology for storage of information, which may be computer readable instructions, data structures, program modules, or other data. Examples of storage media for a computer include, but are not limited to, phase change Memory (Parallel Random Access Memory, PRAM), static random access Memory (Static Random Access Memory, SRAM), dynamic random access Memory (Dynamic Random Access Memory, DRAM), programmable Read Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read Only Memory (Erasable Programmable Read Only Memory, EPROM), other types of random access Memory (Random Access Memory, RAM), read-Only Memory (ROM), one-time programmable Read Only Memory (OTPROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash Memory or other Memory technology, read Only Memory (Compact Disc Read-Only Memory, CD-ROM), digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art will also appreciate that the embodiments described in the specification are all alternative embodiments and that the acts and modules referred to are not necessarily required in the present application. The above embodiments are not necessarily independent embodiments, and the separation into the embodiments is merely used to highlight different technical features in different embodiments, and those skilled in the art should appreciate that the above embodiments may be combined arbitrarily.

In various embodiments of the present application, it should be understood that the size of the sequence numbers of the above processes does not mean that the execution sequence of the processes is necessarily sequential, and the execution sequence of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer-accessible memory. Based on such understanding, the technical solution of the present application, or a part contributing to the prior art or all or part of the technical solution, may be embodied in the form of a software product stored in a memory, including several requests for a computer device (which may be a personal computer, a server or a network device, etc., in particular may be a processor in the computer device) to perform part or all of the steps of the above-mentioned method of the various embodiments of the present application.

The foregoing is merely a specific embodiment of the application to enable one skilled in the art to understand or practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote procedure call device, the remote procedure call device comprising:

the universal interface is used for providing a unified call interface for each Remote Procedure Call (RPC) protocol;

2. The remote procedure call device of claim 1, wherein the protocol adapter specifically comprises:

3. The remote procedure call device of claim 1, wherein the remote procedure call device,

the dynamic proxy module is specifically configured to receive a service call request sent by a client, and select the target RPC protocol according to the service call request.

4. The remote procedure call device of claim 1, wherein the remote procedure call device,

the data processing module is specifically configured to determine, according to the target RPC protocol, a serialization format and an anti-serialization format corresponding to the target RPC protocol, so as to perform encoding processing and decoding processing on data.

5. The remote procedure call device of claim 1, wherein the remote procedure call device further comprises:

6. The remote procedure call device of claim 1, wherein the remote procedure call device further comprises:

7. The remote procedure call device of claim 1, wherein the remote procedure call device further comprises:

8. The remote procedure call device of claim 1, wherein the remote procedure call device further comprises:

9. A distributed system, the distributed system comprising: a remote procedure call device as claimed in any one of claims 1 to 8.

10. An electronic device, characterized in that the electronic device is configured with the distributed system according to claim 9.