CN117235160A - Index center - Google Patents

Abstract

The invention discloses an index center, and relates to the technical field of data processing. In the index center provided by the invention, an index management platform creates an index and stores the created index in a platform database so as to realize maintenance configuration of the platform database; the index driving engine monitors indexes stored in the monitoring platform database by applying and triggers an index data processing flow according to the maintenance configuration of the index management platform in the platform database; a data processing related component implanted with index data processing application initiates a data request to a target data source, and processes the acquired index data according to configuration to obtain index information; the index integrating platform integrates the index information according to the set conditions to generate report information, so that the solution requirements of operation and maintenance cost and data consistency can be reduced, and a one-stop solution is provided for a simple scene.

Description

Index center

Technical Field

The invention relates to the technical field of data processing, in particular to an index center.

Background

A data warehouse (DataWarehouse, DW or DWH) is a strategic collection that provides all types of data support for all levels of decision-making processes of an enterprise, which is a single data store. Created for analytical reporting and decision support purposes, provide guided business process improvement, monitoring time, cost, quality, and control for businesses that require business intelligence.

In the IT department of the internet or financial institution, there is a ground-based practice of data warehouse (referred to as a number of bins). The data is stored in several bins in separate classes. The application program customizes the development logic to obtain the several bin resources according to the self requirement.

Data analysis and decision-making need to rely on quantitative indicators, which are derived from different business scenarios, data sources and computational apertures, so a set of methods and tools are needed to normalize, define, develop and maintain the indicators. The index center is used as a data index management system, which can decouple and split the components of the index and define normalization in a logic table. On the basis, the follow-up assembly can be freely carried out according to a certain rule, and the function of self-defining indexes is realized. The indexes are stored and extracted in a metadata center, a data warehouse or a data lake and the like, so that the availability and response speed of the data can be improved, and the visual display and analysis of the data are supported.

The conventional concept of a digital warehouse is to perform data filtering and cleaning processing by an ETL tool, integrate data scattered everywhere, and store the data in a data warehouse, as shown in fig. 1.

The existing scheme improves the traditional ETL (extract-transform-load) digital bin concept, has the main aims of normalizing indexes and combining business attributes, can integrate company-level data into a system, and is convenient for non-professional staff to use, but the existing index center has the following problems:

1) The acquisition of the data source is mainly SQL, and other forms of data collection schemes are lacked;

2) The threshold of the ETL manufacturing process is higher, and a mature low-code scheme is absent;

3) The system performance is lower and can not withstand larger-scale access;

4) Only the dataset is saved, and functions of modeling and report presentation are absent.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides an index center.

In order to achieve the above object, the present invention provides the following solutions:

an index center, comprising:

the index management platform is used for creating indexes and storing the created indexes in the platform database to realize maintenance configuration of the platform database;

the index driving engine application is used for monitoring indexes stored in the platform database and triggering an index data processing flow according to the maintenance configuration of the index management platform in the platform database;

the data processing related component is implanted with index data processing application and is used for initiating a data request to a target data source, and processing the acquired index data according to configuration to obtain index information;

and the index integrating platform is used for integrating the index information according to the set conditions so as to generate report information.

Optionally, the index management platform creates an index using an index management page; editing information performed in the index creation process includes data source information, trigger information, and data processing flow information.

Optionally, in the process of monitoring the platform database data by the index driving engine application, when the trigger condition is reached, the index processing flow is automatically called, and meanwhile, local or third party calling is performed in an API mode.

Optionally, the index driving engine application uses an automatic task component xxl-job to trigger an index processing flow and select an execution mode in the process of the index processing flow; the execution modes comprise a dependent execution mode and an independent execution mode.

Optionally, the index data processing application includes:

the multi-source data collection application is used for acquiring source data from the outside according to the configured data source information; in the process of acquiring the source data, other index data are called as variables through configuration information stored by an index management platform, so that the dynamic requirement of the source data is realized;

the method comprises the steps that a low-code platform is applied, fixed processing logic exists in the data processing process, the data are processed by adopting the processing logic, and after the data are processed, the processed data are stored into Redis by using a Sink to wait for further processing; the fixed processing logic includes: the rows are divided into a plurality of rows by columns, columns by rows and single columns.

The multi-language data processing application is used for carrying out index processing on the data in a mode of writing codes in the data processing process;

and the data modeling application is used for realizing the structuring of the index.

Optionally, the functions of the multi-source data collection application integration include: reading a database, an API, a message queue and a file; wherein the database is acquired by using JDBC, and the API is processed by using a component corresponding to the API.

Alternatively, the multi-lingual data processing application uses graalvm as the JVM to support multiple programming languages running in the JVM; in a multi-language data processing application, using a Python interpreter to execute an alternative scheme of script processing, and storing an execution result into Redis; the execution result saved in Redis is utilized in writing the script.

Optionally, the structuring of the index is implemented as follows: processing the processed index result set by using an algorithm to obtain DDL data; when the method is executed for the first time, the data modeling application builds a table in the data warehouse so as to achieve the aim of deciding the data dimension in the data warehouse based on the configuration of the index management platform.

Optionally, the index integrating platform displays the generated report information in a customized mode according to the requirements of clients.

Optionally, the index integrating platform is further used for obtaining an index result by using the Redis data.

According to the specific embodiment provided by the invention, the invention discloses the following technical effects:

in the index center provided by the invention, an index management platform creates an index and stores the created index in a platform database so as to realize maintenance configuration of the platform database; the index driving engine monitors indexes stored in the monitoring platform database by applying and triggers an index data processing flow according to the maintenance configuration of the index management platform in the platform database; a data processing related component implanted with index data processing application initiates a data request to a target data source, and processes the acquired index data according to configuration to obtain index information; the index integrating platform integrates the index information according to the set conditions to generate report information, so that the solution requirements of operation and maintenance cost and data consistency can be reduced, and a one-stop solution is provided for a simple scene.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a prior art data warehouse;

FIG. 2 is a schematic diagram of an index center provided by the present invention;

FIG. 3 is a flowchart of the index processing provided by the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The invention aims to provide an index center with low operation and maintenance cost, which can reduce the solution requirement of data consistency and provide a one-stop solution for a simple scene.

Description of prior art terms:

the ETL tool is a process of loading data of the business system into a data warehouse after extraction, cleaning and conversion, and aims to integrate scattered, scattered and non-uniform data in enterprises together so as to provide analysis basis for decisions of the enterprises.

typescript is an open-source programming language developed by microsoft and is built by adding static type definitions on the basis of JavaScript. TypeScript is translated into JavaScript code by a TypeScript compiler or Babel, and can run on any browser, any operating system. typescript supports adding the header file of type information for the existing JavaScript library, and expands the support of popular libraries, such as jQuery, mongoDB, node. Js, D3.Js, and the like. The type definitions of these third party libraries are themselves open-sourced, and all developers can participate in the contribution.

Vue3 development framework: the open-source technology development framework is a front-end technology language which is popular in the market and active in communities.

Java is a programming language that is specifically designed for use in the Internet's distributed environment. Java has a "form and feel" similar to the C++ language, but is easier to use than the C++ language, and uses a "guided by objects" approach thoroughly in programming.

Structured query language (Structured QueryLanguage, SQL) is a special purpose programming language, and also a database query and programming language, for accessing data and querying, updating, and managing relational database systems.

Python provides an efficient, high-level data structure that can also be easily and efficiently programmed towards objects. Python grammar and dynamic type, as well as the nature of interpreted languages, make it a programming language for writing scripts and rapidly developing applications on most platforms.

Doris: apache Doris is a high-performance and real-time analysis type database based on MPP architecture, is well known in the aspect of extremely high speed and easy use, and can return a query result under massive data only by sub-second response time, so that not only can a high-concurrency point query scene be supported, but also a high-throughput complex analysis scene be supported. Based on the method, apache Doris can better meet the use scenes of report analysis, impromptu inquiry, unified data bin construction, data lake federal inquiry acceleration and the like, and a user can construct applications such as user behavior analysis, AB experiment platform, log retrieval analysis, user portrait analysis, order analysis and the like on the use scenes.

GraalVM is a high-performance JDK that is designed to speed up the execution of applications written in Java and other JVM languages, while also providing a runtime environment for JavaScript, python, and other popular languages. GraalVM provides two methods of running Java applications, using Graal just-in-time compiler (JIT) on the HotSpot JVM or as a native executable for advanced compilation (AOT). The multi-language functionality of GraalVM can mix multiple programming languages in a single application while eliminating foreign language call costs.

The Artmas is an online monitoring and diagnosis product, the state information of an application load, a memory, gc and a thread is checked in real time through a global view, and the service problem can be diagnosed under the condition that an application code is not modified, including checking in and out parameters and abnormality of method call, time consumption of monitoring method execution, class loading information and the like, so that the online problem checking efficiency is greatly improved.

Spring Cloud provides a developer with tools to quickly build some common patterns in a distributed system (e.g., configuration management, service discovery, circuit breakers, intelligent routing, micro-agents, control buses, disposable tokens, global locks, leader elections, distributed sessions, cluster states). Coordination of the distributed system adopts a template mode, and a Spring Cloud developer can quickly establish services and application programs for realizing the modes. They will perform well in any distributed environment, including developer's own laptop, data center, and Cloud foundation, etc. hosting platforms.

Quartz is a functionally rich open source job dispatcher and can be integrated into almost any Java application. From the smallest stand-alone application to the largest e-commerce system, quartz can be used to create simple or complex time schedules to execute tens, hundreds, or even tens of thousands of jobs. Its tasks are defined as jobs of standard Java components that can perform almost any operation you might program them. The Quartz scheduler includes many enterprise-level functions, such as supporting JTA transactions and clusters.

In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.

The index center provided by the invention accesses the three-party data source in the modes of database connection, three-party interface API and the like. Based on the extraction and processing of the series execution data of the flow engine, the query is provided to the outside in the form of index, and the process which originally needs complex modeling is simplified into the process of creating the index. Specifically, as shown in fig. 2, the index center provided by the present invention includes: an index management platform, an index driving engine application, a data processing related component and an index integration platform.

The index management platform is used for creating indexes and storing the created indexes in the platform database to realize maintenance configuration of the platform database; the index driving engine is used for monitoring indexes stored in the platform database, and triggering an index data processing flow according to the maintenance configuration of the index management platform in the platform database; the data processing related component is implanted with index data processing application and is used for initiating a data request to a target data source, and processing the acquired index data according to configuration to obtain index information; and the index integrating platform is used for integrating the index information according to the set conditions so as to generate report information.

Based on the above description, in the specific application process, firstly, the index management platform uses the index management page to create the index, and the information adopted in editing in the creation process comprises related information such as a data source, a trigger, a data processing flow and the like. The index management platform can store information into a platform database, monitor index data in the platform database according to an index driving engine application (index driving engine), automatically call an index processing flow when a trigger condition is reached in a detection process, and can also call a local or third party in an API (application program interface) mode. The index driving engine application mainly uses an automatic task component xxl-job to trigger an index data processing flow according to the configuration maintained in a platform database by an index management platform, and can select a dependent execution mode in the process of triggering the index manufacturing flow, namely, the execution mode can be executed according to a node ordering sequence, or the independent execution mode can be selected, and the setting mode is mainly used for realizing independent triggering of each node and having different triggering conditions and time intervals so as to adapt to unused service requirements.

The process of triggering the index data processing flow by using the automatic task component xxl-job is as follows: the node table is fused with the data of the xxl-job-info table in xxl-job. Wherein each record xxl-job-info represents an automated task. When an automatic task triggers, a related handler method is executed, and node class and related methods of nodes are integrated in the handler, for example: judging the acquisition condition of the starting result set, executing the acquisition of the result set, processing the node state, processing the data authority and the like, and judging and triggering according to preset logic.

The configuration information managed by the management platform comprises information related to nodes, such as data source IDs and types related to data set acquisition, entry parameters, next node IDs, event execution IDs and the like, and also comprises attributes related to task scheduling, scheduling start time, scheduling frequency and the like. The node and the class related to the scheduling are combined, so that the management platform is used for managing xxl-job tasks, and the xxl-job control node is used for triggering and running effects.

Further, a multi-source data collection application implanted in the data processing related component after the start is called, and a data request is initiated to a target data source so as to perform fixed processing on the data through the low-code platform application according to different configurations. The low-code platform application mainly performs data conversion in a preset data processing mode, and uses a Sink operator to output the processed data to a Redis.

In the actual application process, the functions of the multi-source data collection application are as follows: the source data is obtained from the outside according to the configured data source information, and the main integrated functions are a read database, an API, a message queue, a file and the like. The database is acquired using JDBC and other components such as APIs are also processed using their counterparts. In the process of acquiring the source data, other index data can be called as variables through configuration information stored by an index management platform, and the dynamic demand of the source data is realized, wherein the realization mode of the process is as follows:

and acquiring related parameter data from the nodes, wherein the Key is in a Key-Value structure, and a date is preset as a occupation in various configurations of data collection, such as SQL sentences, in the form of a occupation symbol in the collection application. In the process of executing collection, firstly, an API mode is used for calling indexes to obtain a result set of parameters, a tangent plane is used for replacing SQL sentences when SQL is executed, file names in file processing are replaced for placeholders, and the effect that a data source realizes dynamic collection according to external dynamic data is achieved.

In the practical application process, the low code platform is applied in the data processing process, and some fixed processing logic may exist, for example, row-to-column, column-to-row, single column is split into a plurality of rows and other preset data processing logic to process the low code. And the low-code platform mainly uses a link, takes the preset logics as operators, and finally uses Sink to store the processed data into Redis for further processing after the data is processed.

Meanwhile, a multi-language data processing application (multi-language data platform) can be used, data processing scripts can be written by using different programming languages according to programming habits of users, and the result is output to Redis as well. The form of the output result is unified into the form of a StringKey-JSONALE, wherein Key is the unique identification of a node or an index, and the JSON format is fixed into the form of an Array-Object. Where the Key of the Object is used as the column field of the index set, and Value is the Value of each row of the index set. Based on the above, there are two types of Redis Key, the first is a node identifier, which is used for storing data among result sets of each node in the index execution process, and can be processed by a data processing application, and the second is an index identifier, which is used for storing the final result of the index result set, and the data modeling should be invoked.

The subsequent processing logic is to invoke the data modeling application to model from the result set and store the data when the index result set flag is present.

In order to implement some complex data processing logic, a multilingual data processing application may perform indexing processing on data by writing code during data processing. In order to match the use habits of different users, the multi-language data processing application uses graalvm as a JVM, and the graalvm can support multiple programming languages to run in the JVM, including JavaScript, python and the like. There are also alternatives to using a Python interpreter to perform script processing in multi-lingual data processing applications if the customer support for Python third party libraries is relatively high. The execution result is stored in Redis, and the result stored in Redis can be utilized when writing the script.

The indexing means that a user converts a scattered data set into a data set conforming to a service viewing angle according to service attributes, a specific scheme is formulated by a technical viewing angle user, a platform is responsible for providing support for the user, and indexes realized by the technical viewing angle user are integrated for the user of the service viewing angle.

The processing result set procedure is determined by the code input by the user, and the variable transmission can be carried out through the method or the redis is directly called in the respective code for processing. The purpose of using graalvm is to store its parameters in JVM for Python language as well as directly as parameters of calling method. For example, if it is desired to process the result set of the last node with a Python code, the node ID of the last node may be directly used in the code to obtain the result set that i need to process.

Further, after the data is transferred, the data is subjected to data modeling application, the existing index data is analyzed by using an algorithm, a structured relational table is obtained, and the data is recorded into a data warehouse by taking the index as a unit. Among other things, the main purpose of data modeling applications is to achieve structuring of metrics. The process first uses the algorithm to process the processed index result set to finally obtain DDL data, and the application builds a table in the data warehouse according to the DDL data when the DDL data is executed for the first time. The purpose of this is to determine the dimensions of the data in the data warehouse, such as time, batch, etc., based on the configuration of the index management platform. This way of handling allows the index to be more easily traced back.

The algorithm adopted when the index result set is processed is mainly to find the greatest common divisor of the result set. Because the table structure of relational data requires all columns, the index results of each dimension are not necessarily full, so when the result set is fetched for the first time, all objectkeys of all columns in the result set are filtered out and processed for union. Then, part of ObjectValue is fetched, the data type and the approximate size are determined by using a regular expression matching mode, after the information of the ObjectValue and the information of the ObjectValue are fetched, the dimension column stored in the result set is fetched, and then the data structure can be started.

The process of building the table is divided into two parts, wherein the first part is that under the condition that the index table does not exist in a warehouse, according to the described flow, after the field name and the field type are obtained, a DDL statement can be generated. And then executed in the data warehouse. And if the second part is that the table exists in the warehouse, judging according to meta information of the table and the result set, taking the part of redundant meta of the result set to the corresponding field name and type information, and generating a DDL statement modified by the table.

And the index integrating platform integrates the data in the data warehouse according to the index, the data column, the dimension and other related information required by the user and generates a corresponding chart for the client at the service side. Specifically, after the persistence of the index is completed, the index information can be integrated in an index integration platform according to the conditions of the index, the field, the dimension and the like, so that report information is generated, and the report information is customized and displayed according to the requirements of clients.

Furthermore, the index integration platform can use Redis data besides structured index data to obtain some index results with higher real-time performance.

The purpose of the index center is to uniformly manage and maintain the data indexes related to the service so as to monitor, analyze and decide the service. The scheme of the index center at present comprises the functions of index management, index driving engine, multi-language index calculation, data modeling application, index integration platform and the like, and the operations of extracting, converting and storing the data are carried out, so that the data are unified from different source systems to a target system, data cleaning, processing and modeling are carried out, and support is provided for subsequent data analysis and decision making.

The specific contents of the index center provided above are described in detail below in units of modules:

A. an index management platform: the management function of the whole indexing flow is realized, and the relation type database is used for providing lasting guarantee for the whole function. For example, the functions of adding, deleting and modifying index data can be realized to drive the whole data processing system through the index configuration data stored in the database. Meanwhile, the index management platform can also realize the function of indexing all data flows, and by providing an API interface, the running results of other indexes or the execution flows of other indexes can be used in the data processing related components, so that the data of the whole system is comprehensively indexed, and the system can produce some more complex indexes. This application mainly provides support for each application in the flowchart to use other indexes, for example, in the process of collecting source data, the results of other indexes are wanted to be called as parameters.

As shown in fig. 3, in each step of the index processing, other indexes may participate. Aiming at the condition of the index data processing flow, the index management platform designs a data structure based on a linked list, and can store the configuration and result data of each node in the flow. And the index management platform performs transmission update in the data processing assembly based on logic processing functions such as logic judgment, data distribution and the like, and finally saves the data in a data warehouse.

Meanwhile, the index management platform also realizes some basic management functions, such as version management, index intermediate data record based on a platform database and a data warehouse, and the like, so that a user can conveniently check possible problems in the index generation process.

B. Index driven engine application: according to the configuration of the index management platform maintained in the platform database, an automatic task component xxl-job is used for triggering the index data processing flow, and a dependent execution mode (namely, execution according to a node ordering sequence) and independent execution (namely, each node is separately triggered and has different triggering conditions and time intervals) can be selected in the process of triggering the index manufacturing flow so as to adapt to different service requirements.

C. Multisource data collection application: the method has the functions of acquiring source data from the outside according to the configured data source information, and mainly integrating the functions of reading a database, an API, a message queue, a file and the like. The database is acquired by JDBC, and other components such as API are also processed by the corresponding components. In the process of acquiring the source data, other index data can be called as variables through the configuration information stored by the index management platform, so that the requirement of source data dynamic is met.

D. Low code platform application: during the data processing process, there may be some fixed processing logic, such as preset data processing logic of row-to-column, column-to-row, splitting a single column into multiple rows, etc., to perform low-code processing. The platform mainly uses a flink, takes the preset logics as each operator, and finally stores the result into a Redis by using a Sink after processing to wait for further processing.

E. Multilingual data processing applications: in order to realize some complex data processing logic, the data can be subjected to indexing processing in a mode of writing codes in the data processing process. In order to match the usage habits of different users, the multi-language data processing application uses graalvm as a JVM, which can support multiple programming languages to run in the JVM, including JavaScript, python, and the like. The main reasons for using this scheme are on the one hand that its support for JavaScript is better than the JVM built-in engine, on the other hand that integrating intermediate results into the JVM during computation is more helpful to use the temporary variables saved in the JVM when writing the script. There are also alternatives to using the Python interpreter to perform script processing in the application if customer support to the Python third party library is high. The execution result is stored in Redis, and the result stored in Redis can be utilized when the script is written.

F. Data modeling application: the main purpose of this application is to achieve structuring of the index. When the method is implemented, firstly, the processed index result set is processed by using an algorithm to finally obtain DDL data, and secondly, when the method is implemented, the application can build a table in a data warehouse according to the DDL data. The purpose of which is to determine the dimensions of the data in the data warehouse, such as time, batch, etc., based on the configuration of the index management platform. This way of handling allows the index to be more easily traced back.

Furthermore, there are two modes of index solidification, the first is to use a memory cache, such as Redis, to cache an index result set, so as to relieve the pressure of a data source end, and a user stores index data with high real-time requirements and also serves as a message queue for data communication in an index flow. The second is to use the MPP database Doris as a persistent storage scheme and preserve the history of the index by dimension.

G. Index integration platform: after the persistence of the index is completed, the index information can be integrated in the platform according to the conditions of the index, the field, the dimension and the like, so that report information is generated, and customized display is performed according to the report information of the client who needs.

Based on the above description, the present invention also has the following advantages over the prior art:

1) The invention provides a relatively perfect index management system and an index analysis system, which can meet the requirements of clients with different visual angles in the aspect of user use experience, and is convenient for the user IT department and business department to use indexes from different visual angles.

2) The system can be fully decoupled, the function module has strong expandability, and various requirements of different user site conditions can be met. For example, the multi-source data collection application and the multi-language data processing application are respectively integrated with a plurality of common processing modes, so that the characteristics of plug and play are maintained in terms of codes, and the cost for modifying special use scenes is relatively low.

3) The data storage structure of the present invention is relatively friendly. Different from the traditional large-width table mode of a plurality of bins, the invention adopts a multi-table storage structure divided according to indexes, so that each index data structure is relatively clear, and the transverse data of the table with different dimensionalities is also convenient to store.

4) The invention can realize the indexing of the data flow. The invention can index all the contents related to the data, for example, a certain date in an SQL sentence can be used as a parameter to be input.

5) The system has high availability. The distributed system scheme is used, each link is disassembled into different services, and the availability of each node is ensured by using load balancing.

6) The invention can construct the flow by using a front-end page dragging mode, and simultaneously, a plurality of default algorithms are used for reducing the threshold of the manual construction flow.

7) The process of receiving external data and outputting data of the flow has multiple alternative schemes, and the universality and the stability of the flow can be ensured.

8) The method and the device use the memory to reserve the flow to execute the intermediate data, thereby being convenient for quickly positioning the node with the problem in the process of the flow problem.

9) When the source data changes, the invention does not need to operate the view, the whole process is automatically identified by a program and the whole view is automatically and incrementally reloaded through a view layout engine and a view algorithm engine.

In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.

The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.

Claims (10)

1. An index center, comprising:

the index management platform is used for creating indexes and storing the created indexes in the platform database to realize maintenance configuration of the platform database;

the index driving engine application is used for monitoring indexes stored in the platform database and triggering an index data processing flow according to the maintenance configuration of the index management platform in the platform database;

the data processing related component is implanted with index data processing application and is used for initiating a data request to a target data source, and processing the acquired index data according to configuration to obtain index information;

and the index integrating platform is used for integrating the index information according to the set conditions so as to generate report information.

2. The metrics center of claim 1, characterized in that the metrics management platform creates metrics using a metrics management page; editing information performed in the index creation process includes data source information, trigger information, and data processing flow information.

3. The index center according to claim 1, wherein the index driving engine automatically calls an index processing flow when a trigger condition is reached in the process of monitoring the platform database data by the index driving engine application, and simultaneously makes a local or third party call by means of an API.

4. The metrics center of claim 3, characterized in that the metrics driver engine application uses an automatic task component xxl-job to trigger metrics process flows and select execution modes during metrics process flows; the execution modes comprise a dependent execution mode and an independent execution mode.

5. The metric center of claim 4, wherein the metric data processing application comprises:

the multi-source data collection application is used for acquiring source data from the outside according to the configured data source information; in the process of acquiring the source data, other index data are called as variables through configuration information stored by an index management platform, so that the dynamic requirement of the source data is realized;

the method comprises the steps that a low-code platform is applied, fixed processing logic exists in the data processing process, the data are processed by adopting the processing logic, and after the data are processed, the processed data are stored into Redis by using a Sink to wait for further processing; the fixed processing logic includes: the rows are divided into a plurality of rows by columns, columns by rows and single columns.

The multi-language data processing application is used for carrying out index processing on the data in a mode of writing codes in the data processing process;

and the data modeling application is used for realizing the structuring of the index.

6. The metric center of claim 5, wherein the multi-source data collection application-integrated functionality comprises: reading a database, an API, a message queue and a file; wherein the database is acquired by using JDBC, and the API is processed by using a component corresponding to the API.

7. The metric center of claim 5, wherein the multi-lingual data processing application uses graalvm as a JVM to support multiple programming languages running in the JVM; in a multi-language data processing application, using a Python interpreter to execute an alternative scheme of script processing, and storing an execution result into Redis; the execution result saved in Redis is utilized in writing the script.

8. The index center according to claim 5, wherein the structuring of the index is achieved by: processing the processed index result set by using an algorithm to obtain DDL data; when the method is executed for the first time, the data modeling application builds a table in the data warehouse so as to achieve the aim of deciding the data dimension in the data warehouse based on the configuration of the index management platform.

9. The index center of claim 1, wherein the index integration platform customizes the report information generated according to customer needs.

10. The metric center of claim 1, wherein the metric integration platform is further configured to obtain the metric results using the Redis data.