CN111209126A - Data transmission method and device between microservices and electronic equipment - Google Patents
Data transmission method and device between microservices and electronic equipment Download PDFInfo
- Publication number
- CN111209126A CN111209126A CN202010008268.5A CN202010008268A CN111209126A CN 111209126 A CN111209126 A CN 111209126A CN 202010008268 A CN202010008268 A CN 202010008268A CN 111209126 A CN111209126 A CN 111209126A
- Authority
- CN
- China
- Prior art keywords
- data
- service
- micro
- message queue
- microservice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000005540 biological transmission Effects 0.000 title claims abstract description 27
- 238000005192 partition Methods 0.000 claims abstract description 57
- 238000012545 processing Methods 0.000 claims abstract description 28
- 230000004931 aggregating effect Effects 0.000 claims abstract description 5
- 238000012546 transfer Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 230000002776 aggregation Effects 0.000 claims description 5
- 238000004220 aggregation Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000010606 normalization Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 230000004069 differentiation Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The embodiment of the application provides a method and a device for data transmission between micro services and electronic equipment, and belongs to the technical field of communication; the method comprises the following steps: aggregating the service data of the plurality of first microservices; writing the service data of each first micro service into a partition corresponding to the first micro service in the message queue; and acquiring the service data required by the second micro service from the partition corresponding to the target micro service in the message queue according to the target micro service corresponding to the second micro service. According to the technical scheme provided by the embodiment of the application, the data of the first micro service is stored in the partition corresponding to the first micro service in the message queue, so that the second micro service can acquire the data of the first micro service without accessing the first micro service, the data transmission and processing efficiency is optimized through the message queue, the second micro service does not need to be adjusted even if the interface of the first micro service is changed, and the adjustment cost of the micro service is reduced.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for data transmission between microservices, an electronic device, and a non-transitory readable storage medium for a computer.
Background
With the increasing demand of single applications, project maintenance becomes more and more difficult, and more applications begin to adopt micro-service architecture. The microservice is that each service runs in its own process, and the communication between services adopts a lightweight communication mechanism, usually using HTTP (hypertext transfer protocol) resource API (Application Programming Interface). The advantage of the micro-service architecture is that it exactly compensates the disadvantages of monolithic applications, for example, a single micro-service only focuses on one specific service function, so it has clear service and less code amount. It is relatively simple to develop and maintain a single microservice; local modification is easy to deploy, a single application only needs to be modified, the whole application needs to be re-deployed, and the micro-service solves the problem.
However, the microservice architecture also faces many challenges, such as high interface adjustment cost, communication between microservices through an interface, and if an API of a certain microservice is modified, all microservices using the interface may need to be adjusted.
Disclosure of Invention
An object of the embodiments of the present invention is to provide a method for transmitting data between micro services, so as to reduce the interface adjustment cost.
The embodiment of the application provides a data transmission method between micro services, wherein the micro services comprise a first micro service and a second micro service; the method comprises the following steps:
aggregating the service data of the plurality of first microservices;
writing the service data of each first micro service into a partition corresponding to the first micro service in a message queue;
and acquiring the service data required by the second micro service from the partition corresponding to the target micro service in the message queue according to the target micro service corresponding to the second micro service.
In an embodiment, the service data includes data to be processed, and after writing the service data of each of the first micro services into the partition corresponding to the first micro service in the message queue, the method further includes:
enabling a computing engine to filter and standardize the data to be processed of the specified partition;
and rewriting the data after the filtering and standardization processing into the specified partition.
In an embodiment, after the obtaining the service data required by the second microservice from the partition corresponding to the target microservice in the message queue, the method further includes:
and storing an acquisition record of the acquisition data of the second microservice from the message queue at the second microservice local.
In an embodiment, after the obtaining the service data required by the second microservice from the partition corresponding to the target microservice in the message queue, the method further includes:
and storing a record of data obtained by each second microservice from the message queue in a log system of the message queue.
In an embodiment, after storing a record of data obtained from the message queue by each second microservice in a log system of the message queue, the method further comprises:
locally to the second microservice, when an acquisition record is lost or needs to be modified, extracting a record of the second microservice acquisition data from the logging system.
In an embodiment, the acquiring the service data required by the second microservice from the partition corresponding to the target microservice in the message queue includes:
and acquiring the simple messages required by the second micro-service from the partition according to the batch of the simple messages in the partition.
The embodiment of the application also provides a data transmission device between the micro services, wherein the micro services comprise a first micro service and a second micro service; the device comprises:
the data aggregation module is used for aggregating the service data of the plurality of first micro services;
the data transfer module is used for writing the service data of each first micro service into a partition corresponding to the first micro service in a message queue;
and the data acquisition module is used for acquiring the service data required by the second micro service from the partition corresponding to the target micro service in the message queue according to the target micro service corresponding to the second micro service.
In an embodiment, the service data includes data to be processed, and the apparatus further includes:
and the data processing module is used for enabling a computing engine to filter and standardize the data to be processed of the specified partition, and rewriting the filtered and standardized data into the specified partition.
Further, an embodiment of the present application further provides an electronic device, where the electronic device includes:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to execute the data transmission method between the micro-services.
Further, an embodiment of the present application also provides a computer non-transitory readable storage medium, where the storage medium stores a computer program, and the computer program is executable by a processor to perform the data transmission method between the above microservices.
According to the technical scheme provided by the embodiment of the application, the data of the first micro service is stored in the partition corresponding to the first micro service in the message queue, so that the second micro service can acquire the data of the first micro service without accessing the first micro service, the data transmission and processing efficiency is optimized through the message queue, the second micro service does not need to be adjusted even if the interface of the first micro service is changed, and the adjustment cost of the micro service is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic application scenario diagram of a data transmission method between microservices according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a method for transmitting data between microservices according to an embodiment of the present application;
fig. 3 is a schematic flowchart illustrating a method for data transmission between microservices according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a method for data transmission between microservices according to another embodiment of the present application;
FIG. 5 is a schematic diagram illustrating the classification of message synchronization between services provided by an embodiment of the present application;
fig. 6 is a block diagram of a data transmission apparatus between microservices according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
Like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Fig. 1 is a schematic application scenario diagram of a data transmission method between microservices according to an embodiment of the present application. As shown in fig. 1, the application scenario includes a server 10, and the server 10 may be one or more servers. Microservice may be considered a process run by the server 10 for performing a certain function. The server 10 may have a plurality of micro services, and implement different functions, and for distinguishing, it is assumed that a plurality of first micro services and a plurality of second micro services exist. The data transfer between the microservices may be considered as a data transfer between a first microservice and a second microservice.
In the past, the interface adjustment of the first micro service and the second micro service for data transmission with the interface of the first micro service all need to be modified. If the second micro service data communicated with the first micro service is more, the workload of modifying the second micro service is larger, and more time is consumed. When the number of the first micro-services is also large, the adjustment cost of the micro-services is undoubtedly raised exponentially.
In this embodiment, the server 10 may use the method provided in this embodiment to implement data transmission between the first microservice and the second microservice, and even if the interface of the first microservice is changed, the second microservice does not need to be adjusted.
In one embodiment, as shown in fig. 1, the microservices run by the server 10 may include an aggregation service 11, a plurality of business services 12, a standardization service 13, and a processing service 14. The convergence service 11 may converge the service data of a plurality of service services 12 and write the service data of each service 12 into the partition corresponding to the service 12 in the message queue 15. The normalization service 13 may perform dirty data filtering and normalization on the traffic data in the partition, and then write the traffic data back to the partition. The processing service 14 may be one or more, and each processing service 14 may obtain the service data from the partition corresponding to the service 12 in the message queue 15 according to the service 12 paired with the processing service 14. Therefore, the processing service 14 can obtain the required data without accessing the business service 12, and even if the interface of the business service 12 is changed, the operation of the processing service 14 is not influenced, so that the adjustment cost of the micro-service can be reduced.
The embodiment of the application also provides the electronic equipment. The electronic device may be the server 10 shown in fig. 1. The electronic device may include a processor; a memory for storing processor-executable instructions; the processor is configured to execute the data transmission method between the microservices provided by the embodiment of the application.
The Memory may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk.
The application also provides a computer non-transitory readable storage medium, which stores a computer program, and the computer program can be executed by a processor to complete the data transmission method between the microservices provided by the embodiment of the application.
Fig. 2 is a schematic flowchart of a method for transmitting data between microservices according to an embodiment of the present disclosure. As shown in fig. 2, the method may include the following steps.
In step 210, traffic data for a plurality of first microservices is aggregated.
For differentiation, the microservice that provides data may be referred to as a first microservice, and the microservice that consumes data may be referred to as a second microservice. By adopting the method provided by the embodiment of the application, the second micro service can acquire the data of the first micro service without accessing the first micro service.
In an embodiment, the first micro-service may be a business service in the embodiment shown in fig. 1. The second microservice may be a processing service in the embodiment shown in fig. 1. The service end can obtain the service data of the plurality of first micro services by operating the convergence service. When the interface of the first micro service is changed, only the aggregated service needs to be adjusted.
In step 220, the service data of each of the first microservices is written into the partition corresponding to the first microservices in the message queue.
Wherein the message queue may be a Kafka (a distributed messaging system) message queue. The message queue may implement multiple partitions, each partition being a topic, each microservice specifying a topic. Data that needs to consume a certain microservice can be obtained from the corresponding partition. Therefore, the service data of the first micro service can be written into the partition corresponding to the first micro service in the message queue.
All data may be stored by default for one week according to the Kafka message queue self-storage mechanism. According to different acquisition situations, some topics can be consumed only once, and some topics can be consumed by a plurality of micro-services for a plurality of times.
In step 230, according to the target microservice corresponding to the second microservice, the service data required by the second microservice is obtained from the partition corresponding to the target microservice in the message queue.
For example, the second microservice may be a BI (Business Intelligence) system, a model system, or an interface service. The target micro service is a data source which needs to be consumed by the second micro service, and can be configured in advance according to actual data requirements. The target microservice belongs to one or several of the plurality of first microservices.
The service end can extract service data from the partition corresponding to the target micro service in the message queue according to the partition corresponding to each first micro service. The service data may be considered as data required for the second micro-service.
According to the technical scheme provided by the embodiment of the application, the data of the first micro service is stored in the partition corresponding to the first micro service in the message queue, so that the second micro service can acquire the data of the first micro service without accessing the first micro service, the data transmission and processing efficiency is optimized through the message queue, the second micro service does not need to be adjusted even if the interface of the first micro service is changed, and the adjustment cost of the micro service is reduced.
In an embodiment, the service data may include data to be processed, and as shown in fig. 3, after step 220, the method provided in this embodiment may further include step 221 and step 222.
In step 221, the computing engine is enabled to filter and normalize the to-be-processed data of the designated partition.
For differentiation, it may be the third microservice of the server that performs filtering and normalization. The computing engine may be a Spark streaming (Spark streaming is an extension of a Spark core API, can implement high throughput, and has real-time stream data processing of a fault-tolerant mechanism) or other streaming computing engines such as a flink (a framework and a distributed processing engine), and the third micro-service may perform dirty data filtering and standardized processing on the data to be processed of the designated partition. The normalization process may be regarded as performing format conversion on the data to be processed according to a preset rule.
In step 222, the filtered and normalized data is rewritten to the designated partition.
The third microserver may rewrite the dirty data filtered and normalized data into the designated partition.
Fig. 4 is a flowchart illustrating a data transmission method between microservices according to another embodiment of the present application. As shown in fig. 4, service a may aggregate traffic data for a plurality of first microservices and store the data in a Kafka message queue. Service B can normalize the data and write back the corresponding topic of the Kafka message queue. Service C may take the normalized data at the topic for corresponding data processing.
Assuming that the service C is a BI system, the BI system can store part of data into relational databases such as mysql and the like to generate a quasi-real-time report; assuming that service C is a model system, the model system can store data into ES or a graph database for modeling; assuming that service C is an interface service, the interface service may store data in Hbase or relational data to continue providing data support to other back-end services.
Therefore, each micro-service only needs to be dedicated to the part processed by the micro-service, and the data processing system which is originally huge is decoupled, so that the problem that a plurality of micro-services are dedicated to the field of the micro-service is solved, the processing requirement of the micro-service can be better met, the cooperation among the services can be completed through the message queue such as kafka, and the flowing of a complete data link is really realized.
In one embodiment, the service data may be a simple message that does not require data processing. The step 230 may include: and acquiring the simple messages required by the second micro-service from the partition according to the batch of the simple messages in the partition.
For simple messages that do not require extensive processing, data can be obtained directly from batches, which can be defined in terms of topic _ date _ time _ batches, as shown in fig. 5. For example, the batch may be convert _20190624_23_2, i.e., the second batch at 24/2019 of the converged service.
For the data to be processed which needs to be processed, the data to be processed needs to be sent to the designated topic, the service which needs to process the data submits tasks such as spark and flink to process the data and then sends the processed data to the designated topic, and the service which needs to continue processing the data appoints the topic to acquire the data or continue processing the data.
For simple messages that do not require extensive processing, there are several cases where service direct consumption retrieval is local: a certain topic has only one service to consume and each message is consumed only once; a certain topic has only one service to consume and each piece of data may be consumed many times; a certain topic is consumed by multiple services and only once; a certain topic is consumed multiple times by multiple services.
For simple messages that do not require a large amount of processing, whether each message is consumed for multiple times or once, or consumed by multiple services at the same time, the offset (i.e., consumption record) of each message can be maintained locally on the service side that consumes data, so as to ensure that the multiple services do not cause confusion of the offset when the data is repeatedly consumed. That is, at the second microservice location, an acquisition record is stored of the second microservice acquiring data from the message queue.
In order to maintain the offset, not only the offset may be maintained at the consuming side, but also a record of the data obtained from the message queue by each second microservice may be stored in a log system of the message queue. In order to substantially ensure consistency of data consumption records, a record of the second microservice acquisition data may be retrieved from the logging system locally to the second microservice when an acquisition record is lost or needs to be modified. Therefore, repeated consumption of data or data not consumed is avoided, and perfect and stable operation of the offset is guaranteed.
Through the technical scheme provided by the embodiment of the application, the transmission efficiency of the data among the micro services is greatly improved, the number of times of landing the data is reduced, and the data processing efficiency is improved. The data storage and transmission pressure is greatly reduced no matter the data needs to be processed or the data can be directly consumed.
Fig. 6 is a block diagram of a data transmission apparatus between microservices according to an embodiment of the present application. As shown in fig. 6, the apparatus may include: a data aggregation module 610, a data relay module 620 and a data acquisition module 630.
And a data aggregation module 610, configured to aggregate the service data of the plurality of first microservices.
And a data transfer module 620, configured to write the service data of each first microservice into a partition corresponding to the first microservice in a message queue.
A data obtaining module 630, configured to obtain, according to a target micro service corresponding to the second micro service, service data required by the second micro service from a partition corresponding to the target micro service in the message queue.
In an embodiment, the service data may include data to be processed, and the apparatus may further include:
and the data processing module is used for enabling a computing engine to filter and standardize the data to be processed of the specified partition, and rewriting the filtered and standardized data into the specified partition.
In an embodiment, the apparatus may further include: and the record storage module is used for storing an acquisition record of the second microservice acquiring data from the message queue at the second microservice local.
In an embodiment, the apparatus may further include: and the log recording module is used for storing the record of the data acquired by each second microservice from the message queue in a log system of the message queue.
In an embodiment, the apparatus may further include: and the record correction module is used for extracting the record of the data acquired by the second micro service from the log system when the acquired record is lost or needs to be corrected locally at the second micro service.
In an embodiment, the service data may include a simple message, and the data obtaining module 630 includes: and the batch-based acquisition unit is used for acquiring the simple messages required by the second micro service from the partition according to the batch of the simple messages in the partition.
The implementation process of the functions and actions of each module in the device is specifically described in the implementation process of the corresponding step in the data transmission method between the microservices, and is not described herein again.
In the embodiments provided in the present application, the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Claims (10)
1. A method for data transmission between microservices, wherein the microservices comprise a first microservices and a second microservices; the method comprises the following steps:
aggregating the service data of the plurality of first microservices;
writing the service data of each first micro service into a partition corresponding to the first micro service in a message queue;
and acquiring the service data required by the second micro service from the partition corresponding to the target micro service in the message queue according to the target micro service corresponding to the second micro service.
2. The method of claim 1, wherein the service data comprises data to be processed, and after writing the service data of each of the first micro services into the partition corresponding to the first micro service in the message queue, the method further comprises:
enabling a computing engine to filter and standardize the data to be processed of the specified partition;
and rewriting the data after the filtering and standardization processing into the specified partition.
3. The method of claim 1, wherein after the obtaining the service data required by the second microservice from the partition corresponding to the target microservice in the message queue, the method further comprises:
and storing an acquisition record of the acquisition data of the second micro service from the message queue at the local of the second micro service.
4. The method of claim 1, wherein after the obtaining the service data required by the second microservice from the partition corresponding to the target microservice in the message queue, the method further comprises:
and storing a record of data obtained by each second microservice from the message queue in a log system of the message queue.
5. The method of claim 4, wherein after storing a record in a log system of the message queue that each second microservice obtained data from the message queue, the method further comprises:
locally to the second microservice, when an acquisition record is lost or needs to be modified, extracting a record of the second microservice acquisition data from the logging system.
6. The method of claim 1, wherein the service data comprises a simple message, and the obtaining the service data required by the second microservice from the partition corresponding to the target microservice in the message queue comprises:
and acquiring the simple messages required by the second micro-service from the partition according to the batch of the simple messages in the partition.
7. An apparatus for data transmission between microservices, wherein the microservices comprise a first microservice and a second microservice; the device comprises:
the data aggregation module is used for aggregating the service data of the plurality of first micro services;
the data transfer module is used for writing the service data of each first micro service into a partition corresponding to the first micro service in a message queue;
and the data acquisition module is used for acquiring the service data required by the second micro service from the partition corresponding to the target micro service in the message queue according to the target micro service corresponding to the second micro service.
8. The apparatus of claim 7, wherein the traffic data comprises data to be processed, the apparatus further comprising:
and the data processing module is used for enabling a computing engine to filter and standardize the data to be processed of the specified partition, and rewriting the filtered and standardized data into the specified partition.
9. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to perform the method of data transfer between microservices of any one of claims 1 to 6.
10. A computer non-transitory readable storage medium, wherein the storage medium stores a computer program executable by a processor to perform the method for data transfer between microservices of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010008268.5A CN111209126A (en) | 2020-01-03 | 2020-01-03 | Data transmission method and device between microservices and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010008268.5A CN111209126A (en) | 2020-01-03 | 2020-01-03 | Data transmission method and device between microservices and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111209126A true CN111209126A (en) | 2020-05-29 |
Family
ID=70785519
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010008268.5A Pending CN111209126A (en) | 2020-01-03 | 2020-01-03 | Data transmission method and device between microservices and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209126A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111639138A (en) * | 2020-06-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN112235205A (en) * | 2020-09-21 | 2021-01-15 | 珠海市卓轩科技有限公司 | Method, device and storage medium for transmitting and consuming MQ message |
| CN112698817A (en) * | 2021-01-05 | 2021-04-23 | 深圳力维智联技术有限公司 | Message access method, device, system and storage medium based on micro-service architecture |
| CN113407365A (en) * | 2021-07-08 | 2021-09-17 | 杭州玳数科技有限公司 | Method and system for plug-in management of dirty data of real-time task based on Flink |
| CN113626869A (en) * | 2021-08-17 | 2021-11-09 | 未鲲(上海)科技服务有限公司 | Data processing method, system, electronic device and storage medium |
| CN112698817B (en) * | 2021-01-05 | 2024-06-07 | 深圳力维智联技术有限公司 | Message access method, device and system based on micro-service architecture and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106533804A (en) * | 2016-12-22 | 2017-03-22 | 成都西加云杉科技有限公司 | Network operation support system |
| US20180113707A1 (en) * | 2016-10-21 | 2018-04-26 | Fujitsu Limited | Microservice-based data processing apparatus, method, and program |
| US10123165B1 (en) * | 2017-09-19 | 2018-11-06 | International Business Machines Corporation | Eliminating false positives of neighboring zones |
| WO2019008548A1 (en) * | 2017-07-07 | 2019-01-10 | Rombit Nv | Method, computer system and computer program product for managing personal data |
| US20190130004A1 (en) * | 2017-10-27 | 2019-05-02 | Streamsimple, Inc. | Streaming Microservices for Stream Processing Applications |
| CN109739929A (en) * | 2018-12-18 | 2019-05-10 | 中国人民财产保险股份有限公司 | Method of data synchronization, apparatus and system |
| US20190182309A1 (en) * | 2017-12-08 | 2019-06-13 | Salesforce.Com, Inc. | Apparatus and method for low-latency message request/response processing |
| WO2019135207A1 (en) * | 2018-01-08 | 2019-07-11 | Here Global B.V. | Distributed processing system and method for the provision of location based services |
| US20190220289A1 (en) * | 2018-01-18 | 2019-07-18 | Sap Se | Configuration for Application Using Microservices |
| US20190331799A1 (en) * | 2018-04-30 | 2019-10-31 | Verizon Patent And Licensing Inc. | Microservices architecture for cloud-scale global navigation satellite system |
-
2020
- 2020-01-03 CN CN202010008268.5A patent/CN111209126A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180113707A1 (en) * | 2016-10-21 | 2018-04-26 | Fujitsu Limited | Microservice-based data processing apparatus, method, and program |
| CN106533804A (en) * | 2016-12-22 | 2017-03-22 | 成都西加云杉科技有限公司 | Network operation support system |
| WO2019008548A1 (en) * | 2017-07-07 | 2019-01-10 | Rombit Nv | Method, computer system and computer program product for managing personal data |
| US10123165B1 (en) * | 2017-09-19 | 2018-11-06 | International Business Machines Corporation | Eliminating false positives of neighboring zones |
| US20190130004A1 (en) * | 2017-10-27 | 2019-05-02 | Streamsimple, Inc. | Streaming Microservices for Stream Processing Applications |
| US20190182309A1 (en) * | 2017-12-08 | 2019-06-13 | Salesforce.Com, Inc. | Apparatus and method for low-latency message request/response processing |
| WO2019135207A1 (en) * | 2018-01-08 | 2019-07-11 | Here Global B.V. | Distributed processing system and method for the provision of location based services |
| US20190220289A1 (en) * | 2018-01-18 | 2019-07-18 | Sap Se | Configuration for Application Using Microservices |
| US20190331799A1 (en) * | 2018-04-30 | 2019-10-31 | Verizon Patent And Licensing Inc. | Microservices architecture for cloud-scale global navigation satellite system |
| CN109739929A (en) * | 2018-12-18 | 2019-05-10 | 中国人民财产保险股份有限公司 | Method of data synchronization, apparatus and system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111639138A (en) * | 2020-06-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111639138B (en) * | 2020-06-03 | 2023-04-25 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN112235205A (en) * | 2020-09-21 | 2021-01-15 | 珠海市卓轩科技有限公司 | Method, device and storage medium for transmitting and consuming MQ message |
| CN112235205B (en) * | 2020-09-21 | 2022-07-01 | 珠海市卓轩科技有限公司 | Method, device and storage medium for transmitting and consuming MQ message |
| CN112698817A (en) * | 2021-01-05 | 2021-04-23 | 深圳力维智联技术有限公司 | Message access method, device, system and storage medium based on micro-service architecture |
| CN112698817B (en) * | 2021-01-05 | 2024-06-07 | 深圳力维智联技术有限公司 | Message access method, device and system based on micro-service architecture and storage medium |
| CN113407365A (en) * | 2021-07-08 | 2021-09-17 | 杭州玳数科技有限公司 | Method and system for plug-in management of dirty data of real-time task based on Flink |
| CN113626869A (en) * | 2021-08-17 | 2021-11-09 | 未鲲(上海)科技服务有限公司 | Data processing method, system, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111209126A (en) | Data transmission method and device between microservices and electronic equipment | |
| CN109493076B (en) | Kafka message unique consumption method, system, server and storage medium | |
| CN112507029B (en) | Data processing system and data real-time processing method | |
| US20150032725A1 (en) | Systems and methods for efficient data ingestion and query processing | |
| CN113010565B (en) | Server real-time data processing method and system based on server cluster | |
| CN109918349A (en) | Log processing method, device, storage medium and electronic device | |
| CN111125260A (en) | Data synchronization method and system based on SQL Server | |
| CN112559475B (en) | Data real-time capturing and transmitting method and system | |
| CN112000741A (en) | Intranet and extranet data exchange system, method, device, computer equipment and medium | |
| CN114430353B (en) | Modbus protocol-based data acquisition method, system, device and medium | |
| CN112019605A (en) | Data distribution method and system of data stream | |
| CN113391973B (en) | Internet of things cloud container log collection method and device | |
| CN105208004B (en) | A kind of data storage method based on OBD equipment | |
| US10225357B2 (en) | Compact data structures for push notifications | |
| CN113204598A (en) | Data synchronization method, system and storage medium | |
| CN114090529A (en) | Log management method, device, system and storage medium | |
| CN111143465A (en) | Method and device for realizing data center station and electronic equipment | |
| CN113342746A (en) | File management system, file management method, electronic device, and storage medium | |
| US20110173165A1 (en) | Management of performance data | |
| Hegde et al. | Low latency message brokers | |
| CN116506300A (en) | Website traffic data statistics method and system | |
| CN108430067A (en) | A kind of Internet service mass analysis method and system based on XDR | |
| CN115757642A (en) | Data synchronization method and device based on filing log file | |
| CN108228662A (en) | Disperse the gathering method and its system of daily record | |
| CN111274316B (en) | Method and device for executing multi-level data stream task, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200529 |
|
| WD01 | Invention patent application deemed withdrawn after publication |