CN110737653A - micro-service-based enterprise data processing system and method - Google Patents
micro-service-based enterprise data processing system and method Download PDFInfo
- Publication number
- CN110737653A CN110737653A CN201910987142.4A CN201910987142A CN110737653A CN 110737653 A CN110737653 A CN 110737653A CN 201910987142 A CN201910987142 A CN 201910987142A CN 110737653 A CN110737653 A CN 110737653A
- Authority
- CN
- China
- Prior art keywords
- data
- enterprise
- micro
- service
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000008569 process Effects 0.000 claims abstract description 22
- 238000007405 data analysis Methods 0.000 claims abstract description 14
- 238000005192 partition Methods 0.000 claims abstract description 7
- 230000000007 visual effect Effects 0.000 claims abstract description 4
- 230000010354 integration Effects 0.000 claims description 36
- 238000004364 calculation method Methods 0.000 claims description 27
- 230000002159 abnormal effect Effects 0.000 claims description 26
- 238000006243 chemical reaction Methods 0.000 claims description 21
- 238000003672 processing method Methods 0.000 claims description 21
- 238000004140 cleaning Methods 0.000 claims description 15
- 238000011161 development Methods 0.000 claims description 13
- 238000012800 visualization Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000013481 data capture Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000005538 encapsulation Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 9
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013075 data extraction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
Abstract
The invention belongs to the technical field of data processing, and discloses micro-service-based integrated enterprise data processing systems and methods, wherein a data source provides data support for the systems, a data pipeline converts data into a message set divided by topics through micro service components corresponding to the data source, the data pipeline is concurrently set by using the partition function of the message system to achieve parallel processing of the data, the micro service components acquire the data from the data pipeline and process the data, the processed data are stored in a database and a data warehouse, the enterprise data are graphically displayed, analyzed and exported through a visual operation panel, and a multi-dimensional data analysis result is obtained by connecting the service system of an enterprise through an SDK API.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to micro-service-based enterprise data processing systems and methods.
Background
Currently, the closest prior art:
in current BI field products, complex data analysis needs customized development or needs professional maintenance and use, and the functions set by the products are limited in the use process, so that the change of data requirements cannot be responded quickly.
The system comprises a plurality of data processing platforms on the market, wherein the data processing platforms are customized according to own specific conditions, if the business is changed greatly, the development is needed greatly, the data processing platforms can be better suitable for different industry data requirements, but generally adopts a scheme with higher relative cost and is over-developed, so enterprise data processing systems and methods which utilize a lightweight architecture and are not over-designed are urgently needed to meet enterprise requirements and solve the problems in the prior art.
In summary, the problems of the prior art are as follows:
(1) when complex data are analyzed, the prior art needs customized development or needs professional personnel to maintain and use, and the functions set by the product are limited in the use process, so that the change of the data requirements cannot be responded quickly.
(2) When dealing with management of a large number of data sources, the prior art is easily confused or cannot realize distributed deployment, and cannot deal with access of large-scale data.
(3) The existing data processing platform is customized according to own specific conditions. If the service changes greatly, the development needs to be greatly improved.
(4) The existing data processing platform capable of meeting different industry data requirements adopts a scheme with relatively high cost, and the problem of over development exists.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides micro-service-based enterprise data processing systems and methods.
The invention is realized in such a way that enterprise data processing methods based on of micro services are integrated, and the enterprise data processing method based on the micro services comprises the following steps:
, the data source provides data support for the system, according to the different characteristics of the data source, the corresponding micro service components determine how to distribute and receive the data provided by the data source, some are actively pulled, some are open access ports, and are actively called by the service side;
secondly, the data is connected with a publish-subscribe message module and converted into a message set divided by topics through a micro-service component corresponding to a data source; the data pipelines are concurrently set by utilizing the partition function of the message system, so that the parallel processing of data is realized; the data pipeline name corresponds to the Topic of the corresponding message system;
thirdly, the micro-service component acquires data from the data pipeline, and performs data integration, cleaning, flow calculation and conversion processing through the application in the operation container; storing the processed data into a database and a data warehouse;
fourthly, performing diagrammatized display, data analysis and data export on the enterprise data through a visual operation panel; and (4) connecting a business system of an enterprise through the SDK API to obtain a multidimensional data analysis result.
, the enterprise data processing method based on micro-service determines how the data pipeline distributes and receives the data provided by the data source by the corresponding micro-service components according to different characteristics of the data source, wherein the distribution mode comprises active pulling and open access ports, and the open access ports are actively called by a service party;
for the data pipeline, micro services for data access are publishers of data, other micro services can be publishers or subscribers or both, the combination of the micro services and the data pipeline is carried out based on the above rules, the message system is transparent to users, and the data pipeline can manage the corresponding relation with the message system.
Further , the data integration method of the micro-service based integrated enterprise data processing method includes:
step , splitting the data integration process into subprocesses, and performing operation encapsulation on the subprocesses to obtain sub-operations, wherein the sub-operations are collected as parent operations;
step two, performing abnormal data capture and abnormal data processing on the sub-jobs;
thirdly, performing transaction setting on the sub-operation, and performing transaction setting on the father operation; integrating the data to start running;
monitoring the execution state of the sub-operation, when the sub-operation is completely operated successfully, the parent operation is operated successfully and data is submitted, and the data integration process is completed; and when the sub-operation fails to operate, data is not submitted, and the data integration process is completed.
Step , the method for capturing and processing abnormal data of sub-operation in step two specifically includes:
1) setting the condition of the abnormal data;
2) capturing abnormal data meeting the conditions, and when the abnormal data is corrected to obtain conventional data, continuing data integration on the conventional data; when the abnormal data can not be corrected to obtain conventional data, storing the abnormal data into a file, skipping the abnormal data, and continuing data integration;
3) when other abnormal data not meeting the condition is captured, the data integration process is ended.
Further , the stream computation of the microservice-based -integrated enterprise data processing method includes:
step , inputting system original data, and initializing and setting variables;
step two, calling a stream calculation engine, and processing data through the stream calculation engine to obtain a stream calculation operation result;
and step three, analyzing the flow calculation operation result obtained in the step two, sending the analysis result to a server, and displaying the analysis result through a display.
Step , in the second step, the stream computation engine is called in a multi-thread parallel mode, or different stream computation engines are called in a serial mode;
selecting different modes to call a stream calculation engine according to different data processing requirements; for a scene with small data volume, selecting a serial mode; and for a scene with large data quantity, a multithreading parallel mode is selected to perform parallel processing on the data.
Another objective of the present invention is to provide micro-service based -centered enterprise data processing system based on the -centered enterprise data processing method based on micro-service, wherein the -centered enterprise data processing system based on micro-service comprises:
the data source provides data support for the system;
the data pipeline is connected with the publishing and subscribing message module and converts data into a message set divided by topics through the micro-service component corresponding to the data source; the data pipelines are concurrently set by utilizing the partition function of the message system, so that the parallel processing of data is realized; the data pipeline name corresponds to the Topic of the corresponding message system;
the micro-service component acquires data from the data pipeline and processes the data by running the application in the container;
the storage module stores the processed data into a database and a data warehouse;
the visualization engine is used for performing diagrammatized display, data analysis and data export on enterprise data through a visualization operation panel; and (4) connecting a business system of an enterprise through the SDK API to obtain a multidimensional data analysis result.
Step , the microservice component includes:
the data integration module is used for realizing data integration through a data integration functional component arranged in the system;
the data cleaning module provides expression rules through cleaning functional components built in the system, and performs filtering, conversion and structural definition on data based on the rules;
the stream calculation module realizes the stream calculation of data through a stream calculation functional component arranged in the system;
the data conversion module realizes the conversion of data through a data conversion functional component arranged in the system;
the script uploading module is used for uploading a processing script by a user aiming at data which is poor in quality, random in structure and incapable of being accurately expressed by an expression, and processing the data through the script;
the component development module is used for developing components according to the access specification aiming at users with development capability and registering the components through the management platform; after successful registration, the user can create or update the data processing process with the component.
Another objective of the invention is to provide information data processing terminals for implementing the -based enterprise data processing method based on microservices.
It is another object of the invention to provide computer-readable storage media including instructions which, when executed on a computer, cause the computer to perform the microservice-based -materialized enterprise data processing method.
In summary, the advantages and positive effects of the invention are:
the invention realizes integration of batch and real-time data processing based on micro-services, runs on a Kubernetes platform, automatically establishes a data pipeline after accessing data from a data source, better responds to rapid enterprise demand change through free combination of the pipeline, realizes heterogeneous integration of data and processing for improving data quality with high efficiency and low cost.
The integrated enterprise data processing system and method based on the micro-service, which are provided by the invention, only accord with the enterprise data processing service by deploying the containerized micro-service, have low cost, and can be easily accessed to a special scene in an expanding way according to the enterprise condition.
Drawings
FIG. 1 is a block diagram of an -based enterprise data processing system based on microservices, according to an embodiment of the present invention;
in the figure: 1. a data source; 2. a data pipe; 3. a publish-subscribe message module; 4. a micro-service component; 5. a storage module; 6. a visualization engine.
Fig. 2 is a flowchart of a method for processing -based enterprise data based on microservices according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of an -based enterprise data processing system based on microservices, as provided by an embodiment of the present invention.
Detailed Description
For purposes of making the objects, aspects and advantages of the present invention more apparent, the present invention will now be described in detail at with reference to the following examples.
To solve the problems of the prior art, the present invention provides microservice-based -integrated enterprise data processing systems and methods, which are described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the -based enterprise data processing system based on microservices according to the embodiment of the present invention includes a data source 1, a data pipeline 2, a publish-subscribe message module 3, a microservice component 4, a storage module 5, and a visualization engine 6.
And the data source 1 provides data support for the system.
The data pipeline 2 is connected with the publishing and subscribing message module 3 and converts data into a message set divided by topics through a micro-service component corresponding to a data source; the data pipelines are concurrently set by utilizing the partition function of the message system, so that the parallel processing of data is realized; the data pipe name corresponds to the Topic of the corresponding messaging system.
And the micro service component 4 acquires data from the data pipeline 2 and processes the data by running the application in the container.
And the storage module 5 is used for storing the processed data into a database and a data warehouse.
The visualization engine 6 is used for performing graphical display, data analysis and data export on enterprise data through a visualization operation panel; and (4) connecting a business system of an enterprise through the SDK API to obtain a multidimensional data analysis result.
And , the micro service component 4 comprises a data integration module 4-1, a data cleaning module 4-2, a stream calculation module 4-3, a data conversion module 4-4, a script uploading module 4-5 and a component development module 4-6.
And the data integration module 4-1 is used for realizing data integration through a data integration functional component built in the system.
And the data cleaning module 4-2 provides expression rules through cleaning functional components built in the system, and performs filtering, conversion and structured definition on data based on the rules.
And the stream computing module 4-3 is used for realizing stream computing of data through a stream computing functional component built in the system.
And the data conversion module 4-4 realizes the conversion of data through a data conversion functional component arranged in the system.
And the script uploading module 4-5 can upload a processing script aiming at the data which is poor in quality, random in structure and incapable of being accurately expressed by an expression, and the data is processed through the script.
The component development module 4-6 is used for developing components according to the access specification for users with development capability and registering the components through the management platform; after successful registration, the user can create or update the data processing process with the component.
And , according to different characteristics of the data source 1, determining how the data pipeline 2 distributes and receives the data provided by the data source 1 by the corresponding micro service component 4, wherein the distribution mode comprises active pulling and open access ports, and the open access ports are actively called by a service party.
, for the data pipe 2, micro services for data access are the publisher of data, and other micro services may be either the publisher or the subscriber or both (when the micro services obtain data from the pipe, if the data pipe is not written any more after the processing is finished, only subscribers are needed, and usually data is sent to the data warehouse as synchronization data, if the data is written into the pipe after the data processing is finished, the data is sent to the downstream micro services, and then the data is both the subscriber and the publisher, and usually data is washed and converted).
Further , the data integration component integrates the data by:
(1) splitting the data integration process into sub-processes, and performing operation packaging on the sub-processes to obtain sub-operations; the set of child jobs is a parent job.
(2) And performing exception data capture and exception data processing on the sub-operation.
(3) Performing transaction setting on a sub-job, and performing transaction setting on a parent job; and integrating the data to start running.
(4) Monitoring the execution state of the sub-operation, when the sub-operation is completely operated successfully, the parent operation is operated successfully and data is submitted, and the data integration process is completed; and when the sub-operation fails to operate, data is not submitted, and the data integration process is completed.
, the step (2) of performing exception data capture and exception data processing on the sub-job includes:
1) and setting the condition of the abnormal data.
2) Capturing abnormal data meeting the conditions, and when the abnormal data is corrected to obtain conventional data, continuing data integration on the conventional data; and when the abnormal data can not be corrected to obtain the conventional data, storing the abnormal data into a file, skipping the abnormal data, and continuing to perform data integration.
3) When other abnormal data not meeting the condition is captured, the data integration process is ended.
And , the flow calculation function component realizes the flow calculation of the data by the following steps:
at step , system raw data is input and variables are initially set.
And step two, calling a stream calculation engine, and processing the data through the stream calculation engine to obtain a stream calculation operation result.
And step three, analyzing the flow calculation operation result obtained in the step two, sending the analysis result to a server, and displaying the analysis result through a display.
And , in the second step, calling the stream computing engine in a multi-thread parallel mode, or calling different stream computing engines in a serial mode, selecting different modes to call the stream computing engines according to different data processing requirements, selecting a serial mode for a scene with small data volume, wherein the logic is simple, and selecting a multi-thread parallel mode for parallel processing of data for a scene with large data volume.
As shown in fig. 2, the method for processing -based enterprise data based on microservices according to the embodiment of the present invention includes the following steps:
s101: the data source provides data support for the system, and according to different characteristics of the data source, the corresponding micro service components determine how to distribute and receive data provided by the data source, some are actively pulled, and some are opened access ports and are actively called by a service party.
S102: the data is converted into a message set divided by topics through a micro-service component corresponding to a data source; and the data pipelines are concurrently set by utilizing the partition function of the message system, so that the parallel processing of the data is realized. The data pipe name corresponds to the Topic of the corresponding messaging system.
S103: the micro-service component acquires data from the data pipeline, and processes data integration, cleaning, flow calculation, conversion and the like are carried out through the application in the operation container; and storing the processed data into a database and a data warehouse.
S104: and performing diagrammatized display, data analysis and data export on the enterprise data through a visual operation panel. And (4) connecting a business system of an enterprise through the SDK API to obtain a multidimensional data analysis result.
The technical solution of the present invention is further described in step with reference to specific examples.
Example 1
As shown in FIG. 3, an illustrative diagram of a microservice-based -based enterprise data processing system is provided by an embodiment of the present invention.
According to different characteristics of the data source, the corresponding micro service components determine how to distribute and receive the data provided by the data source, some are actively pulled, and some are opened access ports and are actively called by a service party. The data pipeline is implemented based on a publish-subscribe message system (such as kafka), and data is converted into a message set divided by a topic through a micro-service component corresponding to a data source. The data pipe name corresponds to the Topic of the corresponding messaging system.
For the data pipeline, microservices for data access are definitely the publishers of data, and other microservices can be publishers or subscribers or both (when the microservices obtain data from the pipeline, after the data are processed, if the data pipeline is not written any more, only subscribers are usually as synchronous data to a data warehouse, if the data are written into the pipeline for use by the downstream microservices after the data are processed, the data are both subscribers and publishers, and usually as data cleaning, conversion and the like).
After the data flow passes through the pipeline, the data can be processed steps by application in a running container, such as cleaning, conversion, machine learning, flow calculation and the like, after the processing, the data can be stored on the ground, can flow back to the pipeline again, and can be processed steps, or can be directly displayed by a visualization engine.
The data extraction, cleaning, conversion and other operations are micro-service examples, common data access, data cleaning and conversion functions are built in the system for users to select, and the built-in cleaning function components provide expression rules for data filtering, conversion and structural definition based on the rules.
For users with development capability, the component can be developed according to the access specification, and the component is registered through the management platform. After successful registration, the set of new or updated data processing procedures can be used.
It should be noted that embodiments of the present invention may be implemented in hardware, software, or a combination of hardware and software, where the hardware components may be implemented using dedicated logic, and the software components may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated design hardware.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1, micro-service-based integrated enterprise data processing method, characterized in that, the micro-service-based integrated enterprise data processing method includes the following steps:
, the data source provides data support for the system, according to the different characteristics of the data source, the corresponding micro service components determine how to distribute and receive the data provided by the data source, some are actively pulled, some are open access ports, and are actively called by the service side;
secondly, the data is connected with a publish-subscribe message module and converted into a message set divided by topics through a micro-service component corresponding to a data source; the data pipelines are concurrently set by utilizing the partition function of the message system, so that the parallel processing of data is realized; the data pipeline name corresponds to the Topic of the corresponding message system;
thirdly, the micro-service component acquires data from the data pipeline, and performs data integration, cleaning, flow calculation and conversion processing through the application in the operation container; storing the processed data into a database and a data warehouse;
fourthly, performing diagrammatized display, data analysis and data export on the enterprise data through a visual operation panel; and (4) connecting a business system of an enterprise through the SDK API to obtain a multidimensional data analysis result.
2. The microservice-based -based enterprise data processing method according to claim 1, wherein the microservice-based -based enterprise data processing method determines how the data pipeline distributes and receives data provided by the data source according to different characteristics of the data source, wherein the distribution mode comprises active pulling and open access ports, and the open access ports are actively called by a business party;
for the data pipeline, micro services for data access are publishers of data, other micro services can be publishers or subscribers or both, the combination of the micro services and the data pipeline is carried out based on the above rules, the message system is transparent to users, and the data pipeline can manage the corresponding relation with the message system.
3. The microservice-based -based enterprise data processing method of claim 1, wherein the data integration method of the microservice-based -based enterprise data processing method comprises:
step , splitting the data integration process into subprocesses, and performing operation encapsulation on the subprocesses to obtain sub-operations, wherein the sub-operations are collected as parent operations;
step two, performing abnormal data capture and abnormal data processing on the sub-jobs;
thirdly, performing transaction setting on the sub-operation, and performing transaction setting on the father operation; integrating the data to start running;
monitoring the execution state of the sub-operation, when the sub-operation is completely operated successfully, the parent operation is operated successfully and data is submitted, and the data integration process is completed; and when the sub-operation fails to operate, data is not submitted, and the data integration process is completed.
4. The microservice-based -integrated enterprise data processing method of claim 3, wherein the method for performing exception data capture and exception data processing on sub-jobs in step two specifically comprises:
1) setting the condition of the abnormal data;
2) capturing abnormal data meeting the conditions, and when the abnormal data is corrected to obtain conventional data, continuing data integration on the conventional data; when the abnormal data can not be corrected to obtain conventional data, storing the abnormal data into a file, skipping the abnormal data, and continuing data integration;
3) when other abnormal data not meeting the condition is captured, the data integration process is ended.
5. The microservice-based -based enterprise data processing method of claim 1, wherein the stream computation of the microservice-based -based enterprise data processing method comprises:
step , inputting system original data, and initializing and setting variables;
step two, calling a stream calculation engine, and processing data through the stream calculation engine to obtain a stream calculation operation result;
and step three, analyzing the flow calculation operation result obtained in the step two, sending the analysis result to a server, and displaying the analysis result through a display.
6. The microservice-based -based enterprise data processing method of claim 5, wherein in step two, the stream computation engine is invoked in a multi-threaded parallel manner, or different stream computation engines are invoked in a serial manner;
selecting different modes to call a stream calculation engine according to different data processing requirements; for a scene with small data volume, selecting a serial mode; and for a scene with large data quantity, a multithreading parallel mode is selected to perform parallel processing on the data.
7, micro-service based enterprise data processing system based on the micro-service based enterprise data processing method of any of claims 1-6, wherein the micro-service based enterprise data processing system comprises:
the data source provides data support for the system;
the data pipeline is connected with the publishing and subscribing message module and converts data into a message set divided by topics through the micro-service component corresponding to the data source; the data pipelines are concurrently set by utilizing the partition function of the message system, so that the parallel processing of data is realized; the data pipeline name corresponds to the Topic of the corresponding message system;
the micro-service component acquires data from the data pipeline and processes the data by running the application in the container;
the storage module stores the processed data into a database and a data warehouse;
the visualization engine is used for performing diagrammatized display, data analysis and data export on enterprise data through a visualization operation panel; and (4) connecting a business system of an enterprise through the SDK API to obtain a multidimensional data analysis result.
8. The microservice-based -instantiated enterprise data processing system of claim 7, wherein the microservice component comprises:
the data integration module is used for realizing data integration through a data integration functional component arranged in the system;
the data cleaning module provides expression rules through cleaning functional components built in the system, and performs filtering, conversion and structural definition on data based on the rules;
the stream calculation module realizes the stream calculation of data through a stream calculation functional component arranged in the system;
the data conversion module realizes the conversion of data through a data conversion functional component arranged in the system;
the script uploading module is used for uploading a processing script by a user aiming at data which is poor in quality, random in structure and incapable of being accurately expressed by an expression, and processing the data through the script;
the component development module is used for developing components according to the access specification aiming at users with development capability and registering the components through the management platform; after successful registration, the user can create or update the data processing process with the component.
9, information data processing terminals for implementing the enterprise data processing method based on micro-services according to any of claims 1-6.
10, computer-readable storage medium comprising instructions that when executed on a computer cause the computer to perform the microservice-based -centered enterprise data processing method of any of claims 1-6 to .
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910987142.4A CN110737653B (en) | 2019-10-17 | 2019-10-17 | Integrated enterprise data processing system and method based on micro-service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910987142.4A CN110737653B (en) | 2019-10-17 | 2019-10-17 | Integrated enterprise data processing system and method based on micro-service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110737653A true CN110737653A (en) | 2020-01-31 |
| CN110737653B CN110737653B (en) | 2023-11-24 |
Family
ID=69269182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910987142.4A Active CN110737653B (en) | 2019-10-17 | 2019-10-17 | Integrated enterprise data processing system and method based on micro-service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110737653B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069021A (en) * | 2020-08-21 | 2020-12-11 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
| CN112817711A (en) * | 2021-01-22 | 2021-05-18 | 海南大学 | Data fusion system based on micro-service |
| CN113112807A (en) * | 2021-04-19 | 2021-07-13 | 重庆交通大学 | Expressway holographic sensing method integrating 4G probe and expressway electromechanical system |
| CN113268478A (en) * | 2021-06-24 | 2021-08-17 | 中国平安人寿保险股份有限公司 | Big data analysis method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106897277A (en) * | 2015-12-17 | 2017-06-27 | 成都飞机工业（集团）有限责任公司 | A kind of production and operation data visualization implementation method based on data mining |
| CN107193546A (en) * | 2017-04-11 | 2017-09-22 | 国网天津市电力公司信息通信公司 | A kind of micro services business application system |
| CN107959718A (en) * | 2017-11-17 | 2018-04-24 | 西北工业大学 | The micro services framework of enterprise-level application software under a kind of cloud computing environment |
| US20180159747A1 (en) * | 2016-12-05 | 2018-06-07 | General Electric Company | Automated feature deployment for active analytics microservices |
-
2019
- 2019-10-17 CN CN201910987142.4A patent/CN110737653B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106897277A (en) * | 2015-12-17 | 2017-06-27 | 成都飞机工业（集团）有限责任公司 | A kind of production and operation data visualization implementation method based on data mining |
| US20180159747A1 (en) * | 2016-12-05 | 2018-06-07 | General Electric Company | Automated feature deployment for active analytics microservices |
| CN107193546A (en) * | 2017-04-11 | 2017-09-22 | 国网天津市电力公司信息通信公司 | A kind of micro services business application system |
| CN107959718A (en) * | 2017-11-17 | 2018-04-24 | 西北工业大学 | The micro services framework of enterprise-level application software under a kind of cloud computing environment |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069021A (en) * | 2020-08-21 | 2020-12-11 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
| CN112069021B (en) * | 2020-08-21 | 2024-02-20 | 北京五八信息技术有限公司 | Flow data storage method and device, electronic equipment and storage medium |
| CN112817711A (en) * | 2021-01-22 | 2021-05-18 | 海南大学 | Data fusion system based on micro-service |
| CN113112807A (en) * | 2021-04-19 | 2021-07-13 | 重庆交通大学 | Expressway holographic sensing method integrating 4G probe and expressway electromechanical system |
| CN113268478A (en) * | 2021-06-24 | 2021-08-17 | 中国平安人寿保险股份有限公司 | Big data analysis method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110737653B (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110737653A (en) | micro-service-based enterprise data processing system and method | |
| US10389602B2 (en) | Automated feature deployment for active analytics microservices | |
| CN111339071B (en) | Method and device for processing multi-source heterogeneous data | |
| CN106055316B (en) | Supply chain financial engine system, system establishing method and server | |
| CN107526645B (en) | A kind of communication optimization method and system | |
| CN113632099A (en) | Distributed product defect analysis system, method and computer readable storage medium | |
| US20140237554A1 (en) | Unified platform for big data processing | |
| JP2022017588A (en) | Training method of deep-running framework, device, and storage medium | |
| CN110704465A (en) | Method, device and storage medium for processing service work order table | |
| CN112783614A (en) | Object processing method, device, equipment, storage medium and program product | |
| CN114265703A (en) | Cross-region computing power scheduling method, system and equipment for cloud server | |
| CN109819332B (en) | Method and device for improving performance of acquiring program data | |
| CN107679404A (en) | Method and apparatus for determining software systems potential risk | |
| CN114756301B (en) | Log processing method, device and system | |
| EP3343372A1 (en) | Distributed cache cleanup for analytic instance runs processing operating data from industrial assets | |
| CN109522089A (en) | Based on the distributed view of virtualized environment as recognition methods | |
| CN111538575B (en) | Resource scheduling system, method, device, equipment and medium | |
| CN114596046A (en) | Integrated platform based on unified digital model of business center station and data center station | |
| CN114490694A (en) | Business rule processing method and device, server and storage medium | |
| Tian | Application and analysis of artificial intelligence graphic element algorithm in digital media art design | |
| CN115767179A (en) | Video stream processing method, system, electronic device and storage medium | |
| CN113722341B (en) | Operation data processing method and related device | |
| CN116932147A (en) | Streaming job processing method and device, electronic equipment and medium | |
| CN117033027A (en) | Data processing method, device, electronic equipment and medium | |
| CN113515563A (en) | Data docking method, database, system and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |