CN107968786A - Distributed RPC systems and data processing method and processing device - Google Patents
Distributed RPC systems and data processing method and processing device Download PDFInfo
- Publication number
- CN107968786A CN107968786A CN201711266297.6A CN201711266297A CN107968786A CN 107968786 A CN107968786 A CN 107968786A CN 201711266297 A CN201711266297 A CN 201711266297A CN 107968786 A CN107968786 A CN 107968786A
- Authority
- CN
- China
- Prior art keywords
- request
- server
- identification information
- solicited message
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/133—Protocols for remote procedure calls [RPC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Abstract
The present invention discloses a kind of distribution RPC systems and data processing method and processing device.The system includes：Client, server end and resource management platform；Server end includes at least two servers, and resource management platform is deployed with real-time Computational frame；Client is used to send client identification information and solicited message to destination server at random；Destination server is used to set request ID for solicited message, and server identification information, solicited message and request ID are sent to resource management platform；Ask ID corresponding with client identification information and solicited message difference；Resource management platform is used to pass through real-time Computational frame response request message, meet with a response result, by response results and request ID send to the corresponding destination server of server identification information so that destination server by response results send to ask the corresponding clients of ID.Technical solution provided by the invention, can simplify process flow, improve efficiency.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of distribution RPC systems and data processing method and dress
Put.
Background technology
RPC (Remote Procedure Call Protocol, remote procedure call protocol) service is a kind of common
C/S (Client/Server, client/server) service architecture, is widely used in production environment.As client accesses
The increase of amount, server end can use distributed structure/architecture, and the request of parallel processing client is this to improve treatment effeciency
Referred to as distribution RPC frameworks.Referring to Fig. 1, Fig. 1 is the structure chart of the system using distribution RPC frameworks in the prior art, is such as schemed
Shown in 1, RPC client randomly choose a RPC Server to handle request.
And the system of current distributed RPC frameworks is used, it is necessary to (locate exclusively for the system deployment of distributed RPC frameworks
Manage device and memory etc.) resource and planning up and down wire management, it is comparatively laborious, it is less efficient.
The content of the invention
In view of this, the present invention provides a kind of distribution RPC systems and data processing method and processing device, place can be simplified
Flow is managed, improves efficiency.
To achieve the above object, the present invention provides following technical solution：
A kind of distributed remote invocation of procedure agreement RPC systems, including：
Client, the server end being connected with the client, the resource management being connected with the server end are put down
Platform；
Wherein, the server end includes at least two servers, and the resource management platform is deployed with real-time calculation block
Frame；
The client is used for, and client identification information and solicited message are sent to destination server, the mesh at random
Mark server is any server that the server end includes；
The destination server is used for, and request ID is set for the solicited message, by server identification information, the request
Information and the request ID are sent to the resource management platform；The request ID and the client identification information and it is described please
Ask information difference corresponding；
The resource management platform is used for, and responds the solicited message by the Computational frame in real time, meet with a response knot
Fruit, by the response results and the request ID send to the corresponding destination server of the server identification information, with
Make it is described with the corresponding destination server of the server identification information by the response results send to the request ID
Corresponding client.
Further, the Computational frame in real time is：
Apache Flink or Apache Storm.
Further, the resource management platform is：
Apache Hadoop YARN or Apache Mesos.
Further, the Computational frame in real time includes：
The built-in operator of degree of parallelism is calculated in multiple support operators between operator.
Further, the built-in operator supports according to target parameter Key aggregated datas.
A kind of data processing method, applied to the distributed remote invocation of procedure agreement RPC systems described in any of the above-described
In destination server, the described method includes：
Receive client identification information and solicited message that the client is sent；
Request ID is set for the solicited message, server identification information, the solicited message and the request ID are sent out
Send to the resource management platform, believe so that the resource management platform responds the request by the Computational frame in real time
Breath, meets with a response as a result, and determining the destination server by the server identification information；The request ID and the visitor
Family end identification information and solicited message difference are corresponding；
The response results and the request ID that the resource management platform is sent are received, the response results are sent
To with the corresponding clients of the request ID.
A kind of data processing equipment, applied to the distributed remote invocation of procedure agreement RPC systems described in any of the above-described
In destination server, described device includes：
Receiving module, the client identification information and solicited message sent for receiving the client；
Uploading module, sets request ID, by server identification information, the solicited message and described for the solicited message
Request ID is sent to the resource management platform, so that the resource management platform passes through described in the response of Computational frame in real time
Solicited message, meets with a response as a result, and determining the destination server by the server identification information；The request ID with
The client identification information and solicited message difference are corresponding；
Feedback module, for receiving the response results and the request ID that the resource management platform is sent, by institute
State response results send to the corresponding clients of the request ID.
It can be seen via above technical scheme that compared with prior art, the present invention provides a kind of distribution RPC systems and
Data processing method and device.Distribution RPC systems provided by the invention, including, client, is connected with the client
Server end, the resource management platform being connected with the server end, wherein, the server end includes at least two services
Device, the resource management platform are deployed with real-time Computational frame, and the client is used for, and client identification information and request are believed
Breath is random to send destination server, and the destination server is any server that the server end includes, the target clothes
Business device is used for, and request ID is set for the solicited message, and server identification information, solicited message and request ID are sent to described
Resource management platform, the request ID is corresponding with the client identification information and solicited message difference, the resource
Management platform is used for, and the solicited message is responded by the Computational frame in real time, meets with a response as a result, by the response results
With the request ID send to the corresponding destination server of the server identification information so that the destination service
Device by the response results send to the corresponding clients of the request ID.That is, server customer in response end is sent out
Real-time Computational frame has been given in the operation for the solicited message sent, and Computational frame is deployed in existing resource management platform in real time
On.Therefore, using technical solution provided by the invention, it is not necessary to as in the prior art exclusively for the system of distributed RPC frameworks
Resource and planning wire management up and down are disposed, but resource is provided by existing resource management platform, is provided by resource management platform
Unified management, can facilitate wire management up and down.Therefore, using technical solution provided by the invention, process flow can be simplified, carried
High efficiency.
In addition, technical solution provided in an embodiment of the present invention, by by the real-time meter being deployed on resource management platform
Frame is calculated, the parallel work-flow between operator in operator can be realized, due to technical solution provided by the invention, to solicited message
Processing and calculating operation perform (RPCServer is only responsible for forwarding) all in real-time Computational frame, and utilize real-time Computational frame
The parallel processing between operator in operator can be realized, so as to improve treatment effeciency, in addition, Computational frame is supported to press in real time
Key aggregated datas, the data for possessing identical Key distribute to same operator execution computing, can reduce and access external system
Number, so as to improve treatment effeciency.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is attached drawing needed in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is the structure chart of the system using distribution RPC frameworks in the prior art；
Fig. 2 is a kind of structure chart of distribution RPC systems provided in an embodiment of the present invention；
Fig. 3 is that a kind of operator of Computational frame in real time provided in an embodiment of the present invention performs logic chart；
Fig. 4 is a kind of flow chart of data processing method provided in an embodiment of the present invention；
Fig. 5 is a kind of structure chart of data processing equipment provided in an embodiment of the present invention.
Embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without making creative work
Embodiment, belongs to the scope of protection of the invention.
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, it is below in conjunction with the accompanying drawings and specific real
Mode is applied to be described in further detail the prior art and the present invention.
First, the prior art is introduced：
RPC (Remote Procedure Call Protocol, remote procedure call protocol) service is a kind of common
C/S (Client/Server, client/server) service architecture, is widely used in production environment.As client accesses
The increase of amount, server end can use distributed structure/architecture, and the request of parallel processing client is this to improve treatment effeciency
Referred to as distribution RPC frameworks.Referring to Fig. 1, Fig. 1 is the structure chart of the system using distribution RPC frameworks in the prior art, is such as schemed
Shown in 1, RPC client randomly choose a RPC Server to handle request.
But this distribution RPC frameworks have as a drawback that：
Need exclusively for spool above and below system deployment (the processor and memory etc.) resource of distributed RPC frameworks and planning
Reason, it is comparatively laborious, it is less efficient；
Do not interacted directly between RPC Server, all information exchanges are needed by external storage, cause to handle
It is inefficient；
Various processes can not be parallel in the program internal services of RPC Server ends, causes treatment effeciency relatively low.
For this reason, this application provides a kind of distribution RPC systems and data processing method and processing device, to solve above-mentioned ask
Topic.Specifically refer to following embodiments.
Embodiment
Referring to Fig. 2, Fig. 2 is a kind of distributed remote invocation of procedure agreement RPC systems provided in an embodiment of the present invention
Structure chart.As shown in Fig. 2, the system includes：
Client 11, the server end 2 being connected with the client 11, the resource being connected with the server end 2
Management platform 3；
It is understood that the server end 2 can connect multiple client 11.
Optionally, the resource management platform 3 can be the resource management platform of current mainstream, such as ApacheHadoop
YARN or Apache Mesos etc., the present invention do not limit this.
Wherein, the server end 2 includes at least two servers 21, and the resource management platform 3 is deployed with real-time meter
Calculate frame；
Optionally, the server end 2 includes multiple servers 21.
Optionally, the Computational frame in real time can be the real-time Computational frame of current mainstream, such as ApacheFlink or
Apache Storm etc., the present invention do not limit this.
The client 11 is used for, and client identification information and solicited message are sent to destination server at random, described
Destination server is any server 21 of the server end 2；
The destination server is used for, and request ID is set for the solicited message, by server identification information, solicited message
Sent with request ID to the resource management platform 3；
Specifically, the request ID is corresponding with the client identification information and solicited message difference.Namely
Say, by the request ID, can recognize that it is which bar solicited message which client is sent.Optionally, it is described to ask
Ask in ID, comprising the identification information to client, and the identification information to solicited message.
Specifically, the server identification information is the identification information of the destination server.
The resource management platform 3 is used for, and responds the solicited message by the Computational frame in real time, meet with a response knot
Fruit, by the response results and the request ID send to the corresponding destination service of the server identification information
Device so that the destination server by the response results send to the corresponding clients 11 of the request ID.
Specifically, the resource management platform 3 responds the solicited message by the Computational frame in real time, meet with a response
As a result, then by the response results and the request ID send to the corresponding target of the server identification information
Server, destination server again by the response results send to the corresponding clients 11 of the request ID, so as to fulfill
To the feedback of the solicited message of client 11.
Distribution RPC systems provided in an embodiment of the present invention, including, client, the service being connected with the client
Device end, the resource management platform being connected with the server end, wherein, the server end includes at least two servers,
The resource management platform is deployed with real-time Computational frame, and the client is used for, by client identification information and solicited message
It is random to send destination server, any server that the destination server includes for the server end, the destination service
Device is used for, and request ID is set for the solicited message, and server identification information, solicited message and request ID are sent to the money
Source capsule platform, the request ID is corresponding with the client identification information and solicited message difference, the resource pipe
Platform is used for, and the solicited message is responded by the Computational frame in real time, meet with a response as a result, by the response results and
The request ID send to the corresponding destination server of the server identification information so that the destination server
By the response results send to the corresponding clients of the request ID.That is, server customer in response end is sent
The operation of solicited message given real-time Computational frame, and Computational frame is deployed on existing resource management platform in real time,
Therefore, using technical solution provided in an embodiment of the present invention, it is not necessary to as being exclusively for distributed RPC frameworks in the prior art
Wire management above and below system deployment resource and planning, but resource is provided by existing resource management platform, carried by resource management platform
For unified management, wire management up and down can be facilitated.Therefore, using technical solution provided in an embodiment of the present invention, place can be simplified
Flow is managed, improves efficiency.
In addition, the real-time Computational frame provided in an embodiment of the present invention being deployed on existing resource management platform, including：
The built-in operator of degree of parallelism is calculated in multiple support operators between operator.
That is, Computational frame supports the calculating degree of parallelism in operator in real time, the calculating degree of parallelism between operator is also supported.
Specifically, Computational frame can include such as Source, Sink, Map, Filter, Reduce, GroupBy, Window in real time,
The operators such as Repartition, user can very easily realize various operators, and set each operator degree of parallelism of itself,
And it is also distributed perform between operator, you can to perform parallel.Optionally, Source is responsible for pulling user from data source
Request data；Sink is responsible for exporting result of calculation；Map is used for doing and mapping (a kind of that a kind of data dress is changed to another data
Operator, specific conversion logic realized by user)；Filter is used for filtering；Reduce is used for doing polymerizeing, and GroupBy is used
To be grouped to data；Window by data temporally or quantity polymerization；Repartition does data again to be split.
Specifically, referring to Fig. 3, the operator that Fig. 3 is a kind of Computational frame in real time provided in an embodiment of the present invention is performed and patrolled
Collect figure.As shown in figure 3, Source is responsible for drawing data from RPC Server, it is then sent to Map and changes, be then forwarded to
Reduce, which is done, to be polymerize, and is last transmitted to Sink, and result of calculation is sent to corresponding RPC Server by Sink.Institute is calculated in real time
Some operators all perform parallel, as shown in figure 3, it is 3 that the degree of parallelism of Source operators, which is 2, Map operators degree of parallelism,
Reduce degree of parallelisms are that the degree of parallelism of 2, Sink operators is 2.All operators of real-time Computational frame can be deployed to resource pipe
In platform, the application and release of the resource such as CPU core and memory are by resource management platform management.
It should be noted that the present invention does not limit the type for being deployed in operator on resource management platform, user can be with
Which according to the business demand of itself select that operator specifically needed.In addition, technical solution provided by the invention does not also limit
Specific service request type, based on different service request types, the corresponding operator in Computational frame in real time of user's selection
Corresponding programming is carried out, it is corresponding with service request to realize.As long as that is, apply distributed remote provided by the invention
Invocation of procedure agreement RPC systems, then without departure from protection scope of the present invention.
Specifically, calculating degree of parallelism of the operator in operator between operator built in Computational frame support in real time, so as to
Realize parallel computation, be conducive to improve calculating speed, lift treatment effeciency.
In addition, the real-time Computational frame provided in an embodiment of the present invention being deployed on existing resource management platform, in it
Put operator and support according to target parameter Key aggregated datas.
Specifically, supporting according to target parameter Key aggregated datas, refer to, support to gather data according to same target parameter Key
Close, calculated, the access times to external memory can be reduced, so as to improve efficiency.
Specifically, Key is usually the value included in solicited message, or generated according to solicited message.Than if any one
Class data, structure are<User name, price>, can be using user name as key, such as two data therein；<Jackie, 30>,<
Jakcie,20>, because key is Jackie, same Reduce operators will be addressed to, that is, realizes and presses Key aggregated datas.
It will illustrate below and the technical solution of the embodiment of the present invention be described in detail.
Such as the example of an each current overall consumption of user of statistics, RPC Client send data<User name, current consumption
The amount of money>The data of form give RPC Server, RPC Server to return to the total spending amount of user so far：
RPC Client send form<User ID, spending amount>Request of data give RPC Server；RPC
The data of reception are converted into by Server<IP, request Id, User ID, spending amount>, wait and (be deployed in resource management to put down
On platform) in real time Computational frame Source operator pulling datas；Source operators are pulled after data using User ID as Key
It is sent to downstream Map operators, real-time Computational frame ensures that the identical data of Key can be sent to identical Map, in this example i.e. phase
Data with User ID can be sent to identical Map, it is assumed that the data that User ID is 001 have all issued Map1, and Map1 is received
Consumption total value after 001 new spending amount with previously stored 001 is added, as 001 total amount consumed at present, random
One of Sink is sent to, for example data are returned to corresponding RPC Server, RPC by Sink1, Sink1 according to IP
Server returns data to corresponding RPC Client again, and whole request processing is completed.
As described above, all operators in real-time Computational frame, including Source, Sink support parallel, and can be with
Any combination；Whole calculating process can be operated on existing resource management platform；Real-time Computational frame support by key into
Row polymerization, so possessing the data of identical Key can all flock together, and such a operator can preserve part Key, such as this example
In Map operation, a Map be responsible for count certain customers spending amount, need not so access outside system, still may be used
To calculate the current whole spending amount of a user, so as to improve efficiency.
That is, technical solution provided in an embodiment of the present invention, by by the reality being deployed on resource management platform
When Computational frame, can realize the parallel work-flow between operator in operator, due to technical solution provided by the invention, request is believed
The processing and calculating operation of breath perform (RPCServer is only responsible for forwarding) all in real-time Computational frame, and utilize real-time calculate
Frame can realize the parallel processing between operator in operator, so as to improve treatment effeciency, in addition, Computational frame branch in real time
Hold and by Key aggregated datas, possess the data of identical Key and distribute to same operator and perform computing, the exterior system of access can be reduced
The number of system, so as to improve treatment effeciency.
In order to more comprehensively illustrate technical solution provided in an embodiment of the present invention, the present invention also provides a kind of data processing
Method.
Referring to Fig. 4, Fig. 4 is a kind of flow chart of data processing method provided in an embodiment of the present invention.The present invention is implemented
The data processing method that example provides, the distributed remote invocation of procedure agreement RPC systems provided applied to the embodiments of the present invention
Destination server in system, as shown in figure 4, this method includes：
Step S401, receives client identification information and solicited message that the client is sent；
Step S402, request ID is set for the solicited message, by server identification information, solicited message and request ID hairs
Send to the resource management platform, believe so that the resource management platform responds the request by the Computational frame in real time
Breath, meets with a response as a result, and determining the destination server by the server identification information；
Specifically, the request ID is corresponding with the client identification information and solicited message difference.
Step S403, receives the response results and the request ID that the resource management platform is sent, by the sound
Should result send to the corresponding clients of the request ID.
Data processing method provided in an embodiment of the present invention, the operation for the solicited message that server customer in response end is sent are handed over
Real-time Computational frame has been given, and Computational frame is deployed on existing resource management platform in real time, therefore, is provided using the present invention
Data processing method, it is not necessary to as in the prior art exclusively for distributed RPC frameworks system deployment resource and planning up and down
Wire management, but by existing resource management platform provide resource, provide unified management by resource management platform, can conveniently on
Lower wire management.Therefore, using technical solution provided by the invention, process flow can be simplified, improve efficiency.
In addition, data processing method provided in an embodiment of the present invention, by by the reality being deployed on resource management platform
When Computational frame, the parallel work-flow between operator in operator can be realized, due to data processing method provided by the invention, to please
Ask the processing of information and calculate operation and (server is only responsible for forwarding) is performed all in real-time Computational frame, and utilize and calculate in real time
Frame can realize the parallel processing between operator in operator, so as to improve treatment effeciency, in addition, Computational frame branch in real time
Hold and by Key aggregated datas, possess the data of identical Key and distribute to same operator and perform computing, the exterior system of access can be reduced
The number of system, so as to improve treatment effeciency.
In order to illustrate technical solution provided by the invention more fully hereinafter, at data provided in an embodiment of the present invention
Reason method, the present invention disclose a kind of data processing equipment.
Referring to Fig. 5, Fig. 5 is a kind of structure chart of data processing equipment provided in an embodiment of the present invention.The present invention is implemented
The data processing equipment that example provides, the distributed remote invocation of procedure agreement RPC systems provided applied to the embodiments of the present invention
Destination server in system, as shown in figure 5, the device includes：
Receiving module 501, the client identification information and solicited message sent for receiving the client；
Uploading module 502, sets request ID, by server identification information, solicited message and request for the solicited message
ID is sent to the resource management platform, so that the resource management platform responds the request by the Computational frame in real time
Information, meets with a response as a result, and determining the destination server by the server identification information；
Specifically, the request ID is corresponding with the client identification information and solicited message difference；
Feedback module 503, will for receiving the response results and the request ID that the resource management platform is sent
The response results send to the corresponding clients of the request ID.
Data processing equipment provided in an embodiment of the present invention, the operation for the solicited message that server customer in response end is sent are handed over
Real-time Computational frame has been given, and Computational frame is deployed on existing resource management platform in real time, therefore, is provided using the present invention
Data processing equipment, it is not necessary to as in the prior art exclusively for distributed RPC frameworks system deployment resource and planning up and down
Wire management, but by existing resource management platform provide resource, provide unified management by resource management platform, can conveniently on
Lower wire management.Therefore, using technical solution provided by the invention, process flow can be simplified, improve efficiency.
In addition, data processing equipment provided in an embodiment of the present invention, by by the reality being deployed on resource management platform
When Computational frame, the parallel work-flow between operator in operator can be realized, due to data processing equipment provided by the invention, to please
Ask the processing of information and calculate operation and (server is only responsible for forwarding) is performed all in real-time Computational frame, and utilize and calculate in real time
Frame can realize the parallel processing between operator in operator, so as to improve treatment effeciency, in addition, Computational frame branch in real time
Hold and by Key aggregated datas, possess the data of identical Key and distribute to same operator and perform computing, the exterior system of access can be reduced
The number of system, so as to improve treatment effeciency.
It can be seen via above technical scheme that compared with prior art, the present invention provides a kind of distribution RPC systems and
Data processing method and device.Distribution RPC systems provided by the invention, including, client, is connected with the client
Server end, the resource management platform being connected with the server end, wherein, the server end includes at least two services
Device, the resource management platform are deployed with real-time Computational frame, and the client is used for, and client identification information and request are believed
Breath is random to send destination server, and the destination server is any server that the server end includes, the target clothes
Business device is used for, and request ID is set for the solicited message, and server identification information, solicited message and request ID are sent to described
Resource management platform, the request ID is corresponding with the client identification information and solicited message difference, the resource
Management platform is used for, and the solicited message is responded by the Computational frame in real time, meets with a response as a result, by the response results
With the request ID send to the corresponding destination server of the server identification information so that the destination service
Device by the response results send to the corresponding clients of the request ID.That is, server customer in response end is sent out
Real-time Computational frame has been given in the operation for the solicited message sent, and Computational frame is deployed in existing resource management platform in real time
On, therefore, using technical solution provided by the invention, it is not necessary to as in the prior art exclusively for the system of distributed RPC frameworks
Resource and planning wire management up and down are disposed, but resource is provided by existing resource management platform, is provided by resource management platform
Unified management, can facilitate wire management up and down.Therefore, using technical solution provided by the invention, process flow can be simplified, carried
High efficiency.
In addition, technical solution provided in an embodiment of the present invention, by by the real-time meter being deployed on resource management platform
Frame is calculated, the parallel work-flow between operator in operator can be realized, due to technical solution provided by the invention, to solicited message
Processing and calculating operation perform (RPCServer is only responsible for forwarding) all in real-time Computational frame, and utilize real-time Computational frame
The parallel processing between operator in operator can be realized, so as to improve treatment effeciency, in addition, Computational frame is supported to press in real time
Key aggregated datas, the data for possessing identical Key distribute to same operator execution computing, can reduce and access external system
Number, so as to improve treatment effeciency.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or order.Moreover, term " comprising ", "comprising" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or equipment including a series of elements not only include that
A little key elements, but also including other elements that are not explicitly listed, or further include for this process, method, article or
The intrinsic key element of equipment.In the absence of more restrictions, the key element limited by sentence "including a ...", is not arranged
Except also there are other identical element in the process, method, article or apparatus that includes the element.
Each embodiment is described by the way of progressive in this specification, what each embodiment stressed be and other
The difference of embodiment, between each embodiment identical similar portion mutually referring to.For method disclosed in embodiment
For device, due to its with embodiment disclosed in system it is corresponding, so description is fairly simple, related part is referring to system
Part illustrates.
Can directly it be held with reference to the step of method or algorithm that the embodiments described herein describes with hardware, processor
Capable software module, or the two combination are implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Any other shape well known in reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register or technical field
In the storage medium of formula.
The foregoing description of the disclosed embodiments, enables professional and technical personnel in the field to realize or use the present invention.
A variety of modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and the principles and novel features disclosed herein phase one
The most wide scope caused.
Claims (7)
- A kind of 1. distributed remote invocation of procedure agreement RPC systems, it is characterised in that including：Client, the server end being connected with the client, the resource management platform being connected with the server end；Wherein, the server end includes at least two servers, and the resource management platform is deployed with real-time Computational frame；The client is used for, and client identification information and solicited message are sent to destination server, the target clothes at random Business device is any server that the server end includes；The destination server is used for, and request ID is set for the solicited message, by server identification information, the solicited message Sent with the request ID to the resource management platform；The request ID believes with the client identification information and the request Breath is corresponding respectively；The resource management platform is used for, and responds the solicited message by the Computational frame in real time, meets with a response as a result, will The response results and the request ID send to the corresponding destination server of the server identification information so that described The response results are sent to corresponding with the request ID with the server identification information corresponding destination server Client.
- 2. distributed remote invocation of procedure agreement RPC systems according to claim 1, it is characterised in that the meter in real time Calculating frame is：Apache Flink or Apache Storm.
- 3. distributed remote invocation of procedure agreement RPC systems according to claim 1, it is characterised in that the resource pipe Platform is：Apache Hadoop YARN or Apache Mesos.
- 4. according to claim 1-3 any one of them distributed remote invocation of procedure agreement RPC systems, it is characterised in that institute Stating real-time Computational frame includes：The built-in operator of degree of parallelism is calculated in multiple support operators between operator.
- 5. distributed remote invocation of procedure agreement RPC systems according to claim 4, it is characterised in that the built-in calculation Son supports according to target parameter Key aggregated datas.
- 6. a kind of data processing method, it is characterised in that applied to Claims 1 to 5 any one of them distributed remote mistake Destination server in journey invocation protocol RPC systems, the described method includes：Receive client identification information and solicited message that the client is sent；For the solicited message set request ID, by server identification information, the solicited message and the request ID send to The resource management platform, so that the resource management platform responds the solicited message by the Computational frame in real time, obtains The destination server is determined to response results, and by the server identification information；The request ID and the client Identification information and solicited message difference are corresponding；Receive the response results and the request ID that the resource management platform is sent, by the response results send to The corresponding clients of the request ID.
- 7. a kind of data processing equipment, it is characterised in that applied to Claims 1 to 5 any one of them distributed remote mistake Destination server in journey invocation protocol RPC systems, described device include：Receiving module, the client identification information and solicited message sent for receiving the client；Uploading module, sets request ID, by server identification information, the solicited message and the request for the solicited message ID is sent to the resource management platform, so that the resource management platform responds the request by the Computational frame in real time Information, meets with a response as a result, and determining the destination server by the server identification information；The request ID with it is described Client identification information and solicited message difference are corresponding；Feedback module, for receiving the response results and the request ID that the resource management platform is sent, by the sound Should result send to the corresponding clients of the request ID.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711266297.6A CN107968786A (en) | 2017-12-05 | 2017-12-05 | Distributed RPC systems and data processing method and processing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711266297.6A CN107968786A (en) | 2017-12-05 | 2017-12-05 | Distributed RPC systems and data processing method and processing device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107968786A true CN107968786A (en) | 2018-04-27 |
Family
ID=61999512
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711266297.6A Pending CN107968786A (en) | 2017-12-05 | 2017-12-05 | Distributed RPC systems and data processing method and processing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107968786A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101997823A (en) * | 2009-08-17 | 2011-03-30 | 联想(北京)有限公司 | Distributed file system and data access method thereof |
| CN102857565A (en) * | 2012-09-03 | 2013-01-02 | 重庆邮电大学 | Intelligent clothes trying-on system based on cloud computing |
| CN103473121A (en) * | 2013-08-20 | 2013-12-25 | 西安电子科技大学 | Mass image parallel processing method based on cloud computing platform |
| CN105450618A (en) * | 2014-09-26 | 2016-03-30 | Tcl集团股份有限公司 | Operation method and operation system of big data process through API (Application Programming Interface) server |
| CN106713226A (en) * | 2015-11-12 | 2017-05-24 | 卓望数码技术（深圳）有限公司 | Remote procedure call processing method used for distributed system and remote procedure call processing system thereof |
-
2017
- 2017-12-05 CN CN201711266297.6A patent/CN107968786A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101997823A (en) * | 2009-08-17 | 2011-03-30 | 联想(北京)有限公司 | Distributed file system and data access method thereof |
| CN102857565A (en) * | 2012-09-03 | 2013-01-02 | 重庆邮电大学 | Intelligent clothes trying-on system based on cloud computing |
| CN103473121A (en) * | 2013-08-20 | 2013-12-25 | 西安电子科技大学 | Mass image parallel processing method based on cloud computing platform |
| CN105450618A (en) * | 2014-09-26 | 2016-03-30 | Tcl集团股份有限公司 | Operation method and operation system of big data process through API (Application Programming Interface) server |
| CN106713226A (en) * | 2015-11-12 | 2017-05-24 | 卓望数码技术（深圳）有限公司 | Remote procedure call processing method used for distributed system and remote procedure call processing system thereof |
Non-Patent Citations (1)
| Title |
|---|
| 伍红 等: "Ninf：全局计算通信库", 《计算机应用研究》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107528870B (en) | A kind of collecting method and its equipment | |
| CN108847989A (en) | Log processing method, business service system and electronic equipment based on micro services framework | |
| JP6450759B2 (en) | Limitations of customer-oriented networks in distributed systems | |
| US9384227B1 (en) | Database system providing skew metrics across a key space | |
| CN105103506B (en) | For the method and system for the non-homogeneous bandwidth request allocation bandwidth in system for cloud computing | |
| CN109471727A (en) | A kind of task processing method, apparatus and system | |
| CN103997725B (en) | System and method for the mobile data network in abstract and layout network environment | |
| US10963363B2 (en) | Correlation based adaptive system monitoring | |
| CN108182140A (en) | Determine the performance capability with monitoring computer resource service | |
| CN110310034A (en) | A kind of service orchestration applied to SaaS, business flow processing method and apparatus | |
| CN107465708A (en) | A kind of CDN bandwidth scheduling systems and method | |
| CN103997523B (en) | Smart city operation system and its implementation based on cloud service | |
| CN103733198A (en) | Stream application performance monitoring metrics | |
| CN106462461A (en) | System, device and methods for billing user for their consumption of mobile broadband services and virtualized cloud resources | |
| CN106412009A (en) | Interface calling method and device | |
| CN105959125A (en) | Network data traffic billing system and billing method | |
| CN105408881A (en) | Bandwidth metering in large-scale networks | |
| CN109478147A (en) | Adaptive resource management in distributed computing system | |
| CN108984542A (en) | Distribution type data collection method and system | |
| CN109951320A (en) | A kind of expansible multi layer monitoing frame and its monitoring method of facing cloud platform | |
| US7562410B2 (en) | System and method for managing a computer network | |
| CN106464584A (en) | Providing router information according to a programmatic interface | |
| CN109074287A (en) | Infrastructure resources state | |
| CN110166507A (en) | More resource regulating methods and device | |
| US10119835B2 (en) | Multi-tenant customer portal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180427 |
|
| RJ01 | Rejection of invention patent application after publication |