CN109587046B - Data processing method and gateway system - Google Patents
Data processing method and gateway system Download PDFInfo
- Publication number
- CN109587046B CN109587046B CN201811397758.8A CN201811397758A CN109587046B CN 109587046 B CN109587046 B CN 109587046B CN 201811397758 A CN201811397758 A CN 201811397758A CN 109587046 B CN109587046 B CN 109587046B
- Authority
- CN
- China
- Prior art keywords
- request
- end service
- load balancer
- restful
- service program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
- H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a data processing method and a gateway system, which are applied to the gateway system, wherein the gateway system comprises: the system comprises a GraphQL parser, a data loader, a batch data loader, a RESTful request template, a load balancer and a back-end service program. The method can provide services for the client by using GraphQL, has the functions of flow segmentation, isolation and monitoring, automatically discovers the back-end service in the micro-service cluster through service discovery, and communicates with the back-end service in the manners of RESTful, RPC and the like. While the advantages of the GraphQL are utilized, the complexity of the GraphQL is hidden, and the development experience and the stability of the system are improved.
Description
Technical Field
The invention relates to the field of mobile internet, in particular to a data processing method and a gateway system.
Background
With the popularization of the internet and various mobile terminals, the rapid development of web pages and mobile applications using the C/S architecture, and the architecture of the server side with the sharply increased number of users is also more and more complex. RESTful specifications are widely adopted, and although the goal of RESTful specifications is easy maintenance and expansion, they are difficult to practice in practice and pose some problems.
The graph QL is used as a replacement scheme of RESTful, and can well solve the problems of version compatibility, design complexity, low efficiency when multiple resources are requested simultaneously and the like of RESTful. However, due to the nature of GraphQL, it is difficult for a traditional infrastructure (e.g., nginx) to perform traffic slicing, isolation, monitoring, etc. on its requests. Since these points are usually necessary for the micro service architecture, the micro service architecture is difficult to be directly implemented based on GraphQL.
Disclosure of Invention
In view of the above, an object of the embodiments of the present invention is to provide a data processing method and a gateway system, so as to alleviate the above problems.
In a first aspect, an embodiment of the present invention provides a data processing method, which is applied to a gateway system, where the gateway system includes: the system comprises a GraphQL analyzer, a data loader, a batch data loader, a RESTful request template, a load balancer and a back-end service program; the method comprises the following steps: in response to a user's request, converting the request into a RESTful request through the RESTful request template; resolving each field included in the RESTful request through the GraphQL parser; loading the various fields by the data loader; merging the same fields in the loaded fields through the batch data loader, and then loading the data included in each field to obtain the loaded data; and checking the state of the back-end service program through the load balancer, and determining a back-end service program to execute the loaded RESTful request.
With reference to an embodiment of the first aspect, converting the request into a RESTful request includes: converting the request to a RESTful request based on a pre-saved domain-specific language.
With reference to an implementation manner of the first aspect, checking, by the load balancer, a state of the backend service and determining a backend service to execute the RESTful request after being loaded, includes: and checking the state of the back-end service program through the load balancer, and determining a back-end service program in a normal state to execute the loaded RESTful request.
With reference to an embodiment of the first aspect, the method further includes: and current limiting and fusing measures of the load balancer ensure that other requests are not influenced under the condition that one back-end service is abnormal.
With reference to an embodiment of the first aspect, the method further includes: and detecting the back-end service program in the running state through a discovery mechanism, and sending the running state of the back-end service program to the load balancer.
In a second aspect, an embodiment of the present invention provides a gateway system, where the gateway system includes: the system comprises a GraphQL analyzer, a data loader, a batch data loader, a RESTful request template, a load balancer and a back-end service program; the RESTful request template is used for responding to a request of a user and converting the request into a RESTful request; the GraphQL parser parses each field included in the RESTful request; the data loader is used for loading each field; the batch data loader is used for merging the same fields in the loaded fields and then loading the data included in each field to obtain the loaded data; and the load balancer is used for checking the state of the back-end service program and determining a back-end service program to execute the loaded RESTful request.
Compared with the prior art, the data processing method and the gateway system provided by the embodiments of the invention have the beneficial effects that: the method can provide services for the client by using the GraphQL, has the functions of flow segmentation, isolation and monitoring, automatically discovers the back-end service in the micro-service cluster through service discovery, and communicates with the back-end service in the manners of RESTful, RPC and the like. While the advantages of the GraphQL are utilized, the complexity of the GraphQL is hidden, and the development experience and the stability of the system are improved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic diagram of a gateway system according to an embodiment of the present invention;
fig. 2 is a block diagram of an operating gateway system according to an embodiment of the present invention;
FIG. 3 is a flowchart of a data processing method according to an embodiment of the present invention;
fig. 4 is a second flowchart of a data processing method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Fig. 1 is a schematic diagram of a gateway system 100 according to an embodiment of the present invention. The gateway system 100 may include: the system comprises a GraphQL parser, a data loader, a batch data loader, a RESTful request template, a load balancer, a back-end service program and the like.
As shown in fig. 2, a block diagram of an electronic device 200 operating the gateway system 100 is shown. The electronic device 200 includes: memory 110, memory controller 120, processor 130, peripheral interface 140, input output unit 150, audio unit 160, display unit 170.
The memory 110, the memory controller 120, the processor 130, the peripheral interface 140, the input/output unit 150, the audio unit 160, and the display unit 170 are electrically connected to each other directly or indirectly, so as to implement data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.
The processor 130 is used to execute the executable modules stored in the memory 110.
The Memory 110 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 110 is used for storing a program, and the processor 130 executes the program after receiving an execution instruction, and the method defined by the flow disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 130, or implemented by the processor 130.
Referring to fig. 3, fig. 3 is a flowchart of a data processing method according to an embodiment of the present invention, where the method is applied to the gateway system 100. The flow shown in fig. 3 will be described in detail below, and the method includes:
step S110: in response to a user's request, converting the request into a RESTful request via the RESTful request template.
Wherein the request may be converted to a RESTful request based on a pre-saved domain-specific language. The request template is grouped by the back-end service to which it belongs.
Step S120: resolving, by the GraphQL parser, the fields included in the RESTful request.
And the GraphQL parser parses the received request which accords with the GraphQL language specification, and analyzes fields and parameters of the RESTful request.
Step S130: and loading the fields through the data loader.
The data loader is responsible for the loading of individual fields, but does not immediately load data but instead delegates the loading to the bulk data loader.
Step S140: and merging the same fields in the loaded fields through the batch data loader, and then loading the data included in each field to obtain the loaded data.
And merging the requests of the same field of the data loader to reduce the communication times with the back end and improve the loading efficiency.
Step S150: and checking the state of the back-end service program through the load balancer, and determining a back-end service program to execute the loaded RESTful request.
The health of the back-end is checked and a back-end must be selected to execute the request. At the same time, current limiting and fusing measures ensure that individual backend services do not affect other requests in the event of an anomaly or cause more stress on the backend.
Optionally, referring to fig. 4, the method further includes:
step S160: and current limiting and fusing measures of the load balancer ensure that other requests are not influenced under the condition that one back-end service is abnormal.
And detecting the back-end service program in the running state through a discovery mechanism, and sending the running state of the back-end service program to the load balancer.
The data processing method provided by the embodiment of the invention can provide services for the client by using GraphQL, has the functions of flow segmentation, isolation and monitoring, automatically discovers the back-end service in the micro-service cluster through service discovery, and communicates with the back-end service in the manners of RESTful, RPC and the like. While the advantages of the GraphQL are utilized, the complexity of the GraphQL is hidden, and the development experience and the stability of the system are improved.
Referring to fig. 1, an embodiment of the present invention further provides a gateway system 100, which corresponds to the data processing method in the foregoing method embodiment, and includes: the system comprises a GraphQL parser, a data loader, a batch data loader, a RESTful request template, a load balancer and a back-end service program.
The RESTful request template is used for responding to a request of a user and converting the request into a RESTful request;
the GraphQL parser parses each field included in the RESTful request;
the data loader is used for loading each field;
the batch data loader is used for merging the same fields in the loaded fields and then loading the data included in each field to obtain the loaded data;
and the load balancer is used for checking the state of the back-end service program and determining a back-end service program to execute the loaded RESTful request.
Optionally, the load balancer is configured to convert the request into a RESTful request based on a domain-specific language stored in advance.
Optionally, the load balancer is configured to check a state of the backend service program through the load balancer, and determine that a backend service program in a normal state executes the loaded RESTful request.
Optionally, the load balancer is further configured to ensure that other requests are not affected by a current limiting and fusing measure of the load balancer when one of the backend services is abnormal.
Optionally, the system further includes: and the service discovery mechanism module is used for detecting the back-end service program in the running state and sending the running state of the back-end service program to the load balancer.
In this embodiment, please refer to the contents described in the embodiments shown in fig. 1 to fig. 4 for the process of implementing each function of each functional module of the gateway system, which is not described herein again.
In summary, the data processing method and the gateway system provided in the embodiments of the present invention provide services for the client by using the GraphQL, and have the functions of traffic segmentation, isolation, and monitoring, automatically discover the backend services in the micro-service cluster through service discovery, and communicate with the backend services through RESTful, RPC, and other manners. While the advantages of the GraphQL are utilized, the complexity of the GraphQL is hidden, and the development experience and the stability of the system are improved.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. The system embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A data processing method, applied to a gateway system, the gateway system comprising: the system comprises a GraphQL analyzer, a data loader, a batch data loader, a RESTful request template, a load balancer and a back-end service program; the method comprises the following steps:
responding to a request of a user, and analyzing each field included in the request through the GraphQL analyzer;
converting the respective fields into corresponding RESTful requests through the RESTful request template;
loading, by the data loader, fields contained by the corresponding RESTful request;
merging the same fields in the loaded fields through the batch data loader, and then loading the data included in each field to obtain the loaded data;
and checking the state of the back-end service program through the load balancer, and determining a back-end service program to execute the loaded RESTful request.
2. The method of claim 1, wherein converting the request into a RESTful request comprises:
converting the request to a RESTful request based on a pre-saved domain-specific language.
3. The method of claim 1, wherein checking the state of the back-end service by the load balancer and determining a back-end service to execute the loaded RESTful request comprises:
and checking the state of the back-end service program through the load balancer, and determining a back-end service program in a normal state to execute the loaded RESTful request.
4. The method of claim 1, further comprising:
and current limiting and fusing measures of the load balancer ensure that other requests are not influenced under the condition that one back-end service is abnormal.
5. The method according to any one of claims 1-4, further comprising:
and detecting the back-end service program in the running state through a discovery mechanism, and sending the running state of the back-end service program to the load balancer.
6. A gateway system, characterized in that the gateway system comprises: the system comprises a GraphQL analyzer, a data loader, a batch data loader, a RESTful request template, a load balancer and a back-end service program;
the RESTful request template is used for responding to a request of a user and converting the request into a RESTful request;
the GraphQL parser parses each field included in the RESTful request;
the data loader is used for loading each field;
the batch data loader is used for merging the same fields in the loaded fields and then loading the data included in each field to obtain the loaded data;
and the load balancer is used for checking the state of the back-end service program and determining a back-end service program to execute the loaded RESTful request.
7. The system of claim 6, wherein the load balancer is configured to translate the request into a RESTful request based on a pre-saved domain-specific language.
8. The system of claim 6, wherein the load balancer is configured to check the state of the back-end service via the load balancer, and determine a back-end service in a normal state to execute the loaded RESTful request.
9. The system of claim 6, wherein the load balancer is further configured to ensure that other requests are not affected in the event of an anomaly in one of the backend services through current limiting and fusing measures of the load balancer.
10. The system according to any one of claims 6-9, further comprising: and the service discovery mechanism module is used for detecting the back-end service program in the running state and sending the running state of the back-end service program to the load balancer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811397758.8A CN109587046B (en) | 2018-11-22 | 2018-11-22 | Data processing method and gateway system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811397758.8A CN109587046B (en) | 2018-11-22 | 2018-11-22 | Data processing method and gateway system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109587046A CN109587046A (en) | 2019-04-05 |
| CN109587046B true CN109587046B (en) | 2021-05-14 |
Family
ID=65923742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811397758.8A Active CN109587046B (en) | 2018-11-22 | 2018-11-22 | Data processing method and gateway system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109587046B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107817996A (en) * | 2017-10-13 | 2018-03-20 | 贵州白山云科技有限公司 | A kind of optimization method and system of GraphQL requests |
| CN107948135A (en) * | 2017-10-31 | 2018-04-20 | 贵州白山云科技有限公司 | A kind of data processing method and device for supporting a variety of API protocols |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3484135A1 (en) * | 2008-04-02 | 2019-05-15 | Twilio Inc. | System and method for processing telephony sessions |
| US9530116B2 (en) * | 2013-05-28 | 2016-12-27 | Verizon Patent And Licensing Inc. | Finite state machine-based call manager for web-based call interaction |
| CN104980525B (en) * | 2015-07-10 | 2018-09-14 | 华南理工大学 | A kind of universality mobile computing system based on state middleware |
| US9753744B1 (en) * | 2016-05-27 | 2017-09-05 | Intuit Inc. | Defining application programming interfaces (APIS) using object schemas |
| CN206077428U (en) * | 2016-06-15 | 2017-04-05 | 上海电器科学研究所（集团）有限公司 | A kind of energy source gateway for supporting Restful interfaces |
-
2018
- 2018-11-22 CN CN201811397758.8A patent/CN109587046B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107817996A (en) * | 2017-10-13 | 2018-03-20 | 贵州白山云科技有限公司 | A kind of optimization method and system of GraphQL requests |
| CN107948135A (en) * | 2017-10-31 | 2018-04-20 | 贵州白山云科技有限公司 | A kind of data processing method and device for supporting a variety of API protocols |
Non-Patent Citations (1)
| Title |
|---|
| 把REST包装成GraphQL;林一二;《知乎专栏》;20161009;第1、4-14页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109587046A (en) | 2019-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10135936B1 (en) | Systems and methods for web analytics testing and web development | |
| CN109543891B (en) | Method and apparatus for establishing capacity prediction model, and computer-readable storage medium | |
| CN108694120B (en) | Method and device for testing service component | |
| CN112380145A (en) | SDK generation method, device, equipment and readable storage medium | |
| CN110888816A (en) | Program testing method, program testing device, and storage medium | |
| CN109189692A (en) | A kind of webservice interface test method and device | |
| CN109558315B (en) | Method, device and equipment for determining test range | |
| CN112882948A (en) | Stability testing method, device and system for application and storage medium | |
| CN115221033A (en) | Interface protocol testing method and device, computer readable medium and electronic equipment | |
| CN109587046B (en) | Data processing method and gateway system | |
| CN115297029B (en) | Method, device, equipment and medium for testing resolution performance of industrial internet identification | |
| CN113094268B (en) | Test method, test device, test equipment and test medium | |
| CN111737129B (en) | Service control method, device, computer readable medium and electronic equipment | |
| CN110837469B (en) | Data testing method and device, testing server and storage medium | |
| CN112379967A (en) | Simulator detection method, device, equipment and medium | |
| CN111967225A (en) | Index coding method and device and readable storage medium | |
| CN113760698A (en) | Method and device for converting test case file data | |
| KR101584661B1 (en) | RTE-based big data analysis system and method | |
| CN110704042A (en) | Program code processing method and device, server and storage medium | |
| CN112214391A (en) | Internet of things platform interface testing method and system and computer storage medium | |
| CN112988560A (en) | Method and device for testing system robustness | |
| CN114968829B (en) | Full link pressure test method, electronic device and storage medium | |
| CN115544134A (en) | Data access method, device, computer equipment and storage medium | |
| CN114968693A (en) | Performance test method and device | |
| CN116932339A (en) | Service log processing method, device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210423 Address after: Unit 1701e, China energy storage building, 3099 Keyuan South Road, high tech community, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Applicant after: SHENZHEN DAYU WUXIAN TECHNOLOGY Co.,Ltd. Address before: Unit 2301-L, Bike Building, No. 9 Scientific Research Road, Yuehai Street, Nanshan District, Shenzhen City, Guangdong Province, 518000 Applicant before: SHENZHEN MOSHI TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |