CN117610519B - Method and system for dynamically rendering report based on index model - Google Patents
Method and system for dynamically rendering report based on index model Download PDFInfo
- Publication number
- CN117610519B CN117610519B CN202410085436.9A CN202410085436A CN117610519B CN 117610519 B CN117610519 B CN 117610519B CN 202410085436 A CN202410085436 A CN 202410085436A CN 117610519 B CN117610519 B CN 117610519B
- Authority
- CN
- China
- Prior art keywords
- data
- dimension
- data model
- index
- physical table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000009877 rendering Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000013499 data model Methods 0.000 claims abstract description 222
- 238000012545 processing Methods 0.000 claims abstract description 51
- 238000013507 mapping Methods 0.000 claims description 21
- 238000004806 packaging method and process Methods 0.000 claims description 20
- 238000013515 script Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 16
- 238000012216 screening Methods 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 8
- 230000001360 synchronised effect Effects 0.000 claims description 6
- 238000011161 development Methods 0.000 abstract description 5
- 239000000126 substance Substances 0.000 abstract description 2
- 238000005259 measurement Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 238000003339 best practice Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 239000000047 product Substances 0.000 description 2
- 239000002994 raw material Substances 0.000 description 2
- 238000009517 secondary packaging Methods 0.000 description 2
- 239000011265 semifinished product Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
Abstract
The invention discloses a method and a system for dynamically rendering a report based on an index model, belongs to the technical field of business systems, and aims to solve the technical problem of how to solve the problem that the report cannot be dynamically adjusted and rendered during development and ensure data consistency and accuracy. The method comprises the following steps: processing the collected business data based on the application scene, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background; constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary substance, and integrating the dimension data model, the index data model and the full-dimension data model constructed in the background with a front-end interface; and loading a dimension data model and an index data model on the front-end interface based on the data request to form a business report, and rendering and dynamically displaying the business report.
Description
Technical Field
The invention relates to the technical field of business systems, in particular to a method and a system for dynamically rendering a report based on an index model.
Background
In the big data age, the generation and the display of the report form are important links of data processing and analysis, and at present, two traditional report form generation methods in the industry are adopted, one is a static report form which is developed in a personalized way, namely, the data and the format of the report form are fixed after being generated, and cannot be dynamically adjusted according to real-time data or specific requirements; and secondly, writing SQL scripts by report tool operators through the report tool by using the user-defined report tool, and rendering a report by the report tool according to the returned result to display report contents. At present, two reports are high in development and learning cost, cannot be mastered by users who do not understand the technology, and cannot meet the requirements of user data index standardization, data accuracy and consistency due to different SQL writing levels of different implementation personnel and non-uniform statistical caliber and granularity.
How to solve the problem that the report cannot be dynamically adjusted and rendered during report development and ensure the consistency and accuracy of data is a technical problem to be solved.
Disclosure of Invention
The technical task of the invention is to provide a method and a system for dynamically rendering a report based on an index model, so as to solve the technical problems that the report cannot be dynamically adjusted and rendered during development and ensure the consistency and accuracy of data.
The invention discloses a method for dynamically rendering a report based on an index model, which is applied to a system comprising a front end and a background, and comprises the following steps:
and (3) data acquisition: for an enterprise to be acquired, carrying out standardized processing on each business system data in the enterprise, and acquiring global business data in the enterprise through a unified data acquisition channel;
Data processing: processing the collected service data based on an application scene, dividing the service data into three layers of ODS, CDM and ADS, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background;
Modeling data: constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary object, carrying out primary key field association on the dimension data model and the index data model to form a data model mapping relation, constructing a full dimension data model in a wide table form, storing the association relation between the dimension data model and the index data model into a database, and integrating the dimension data model, the index data model and the full dimension data model constructed in the background with a front end interface;
front-end rendering: and loading a dimension data model and an index data model on the front-end interface based on the data request to form a business report, and rendering and dynamically displaying the business report.
Preferably, the data acquisition channel supports data acquisition from structured, semi-structured and unstructured data sources, supports batch, real-time, full-scale or incremental synchronous modes, and supports off-line data acquisition and real-time data acquisition modes.
Preferably, when a dimension data model is constructed based on metadata information in a dimension physical table, reading the dimension physical table metadata information in a database, identifying a field name, a field meaning and a field type of the dimension physical table, and defining the dimension data model by normalizing the field meaning and the field type of the physical table;
The dimension data model comprises a source physical table of a dimension field, a mapping relation between the dimension field and a physical table field, an identification weight of the dimension field associated source physical table, filtering conditions of the source physical table, a data dictionary name of the dimension field and service classification of the dimension data model;
When an index data model is constructed based on metadata information in summarized matters, reading summarized physical table metadata information in a database, identifying the field names, field meanings and field types of the summarized physical table, and defining the index data model by normalizing the field meanings and the field names of the physical table;
The index data model comprises a plurality of different data types, the data types comprise a numerical type, a text type and a date type, each index field of the index data model is provided with a group of index functions, the index functions comprise a sum, a total bar number, an average value and a variance, the index data model comprises source physical table weights of dimension fields, filtering conditions of source physical tables, excluding non-support dimensions of indexes and index statistical units.
Preferably, after integrating the dimension data model, the index data model and the full dimension data model with the front end interface, the front end interface and the background cooperate to execute the following steps:
Selecting different data items and screening conditions through a front-end interface, wherein the data items comprise data sources, dimensions and indexes, and the screening conditions comprise query conditions and sorting conditions;
acquiring the dimension, index, query condition and sorting condition selected by the front-end interface, and searching the actual physical table and field information according to the mapping relation of the data model;
According to the relation between the dimension data model and the index data model, splicing SQL scripts of different data to be queried according to indexes and query conditions;
all SQL scripts are executed concurrently through a dispatching thread pool, and a query result is obtained;
And splicing and packaging the query results, and returning the data to the front-end interface after packaging is finished.
Preferably, the front-end rendering comprises the steps of:
packaging page components used by the dimension data model and the index data model constructed in the background into a general application label;
Selecting a data model through front-end interface dragging, and constructing a business report through the front-end interface dragging dimension and index;
dynamically updating the calculation method and the data source of the index model according to the real-time data, and rendering the service report in real time;
and dynamically adjusting the layout and the look-up of the business report according to the user interaction.
In a second aspect, the present invention provides a system for dynamically rendering a report based on an index model, applied between a front-end interface and a background, for performing service report rendering by using the method for dynamically rendering a report based on an index model according to any one of the first aspects, where the system includes a data acquisition module, a data processing module, a data modeling module, and a front-end rendering module;
The data acquisition module is used for executing the following steps: for an enterprise to be acquired, carrying out standardized processing on each business system data in the enterprise, and acquiring global business data in the enterprise through a unified data acquisition channel;
The data processing module is used for executing the following steps: processing the collected service data based on an application scene, dividing the service data into three layers of ODS, CDM and ADS, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background;
the data modeling module is used for executing the following steps: constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary object, carrying out primary key field association on the dimension data model and the index data model to form a data model mapping relation, constructing a full dimension data model in a wide table form, storing the association relation between the dimension data model and the index data model into a database, and integrating the dimension data model, the index data model and the full dimension data model constructed in the background with a front end interface;
The front-end rendering module is configured to perform the following: and loading a dimension data model and an index data model on the front-end interface based on the data request to form a business report, and rendering and dynamically displaying the business report.
Preferably, the data acquisition channel supports data acquisition from structured, semi-structured and unstructured data sources, supports batch, real-time, full-scale or incremental synchronous modes, and supports off-line data acquisition and real-time data acquisition modes.
Preferably, when constructing the dimension data model based on metadata information in the dimension physical table, the data modeling module is configured to perform the following: reading dimension physical table metadata information in a database, identifying a field name, a field meaning and a field type of a dimension physical table, and defining a dimension data model through normalizing the field meaning and the field type of the physical table;
The dimension data model comprises a source physical table of a dimension field, a mapping relation between the dimension field and a physical table field, an identification weight of the dimension field associated source physical table, a filtering condition of the source physical table, a data dictionary name of the dimension field and service classification of the dimension data model;
When an index data model is constructed based on metadata information in the summary object, the data modeling module is configured to perform the following steps: reading summarized physical table metadata information in a database, identifying the names, field meanings and field types of summarized physical table fields, and defining an index data model through normalizing the field meanings and names of the physical tables;
The index data model comprises a plurality of different data types, wherein the data types comprise a numerical type, a text type and a date type, each index field of the index data model is provided with a group of index functions, each index function comprises a sum, a total number, an average value and a variance, the index data model comprises source physical table weights of dimension fields, filtering conditions of the source physical table, excluding index non-support dimensions and index statistical units.
Preferably, after integrating the dimension data model, the index data model and the full dimension data model with the front end interface, the front end interface and the background cooperate to execute the following steps:
Selecting different data items and screening conditions through a front-end interface, wherein the data items comprise data sources, dimensions and indexes, and the screening conditions comprise query conditions and sorting conditions;
acquiring the dimension, index, query condition and sorting condition selected by the front-end interface, and searching the actual physical table and field information according to the mapping relation of the data model;
According to the relation between the dimension data model and the index data model, splicing SQL scripts of different data to be queried according to indexes and query conditions;
all SQL scripts are executed concurrently through a dispatching thread pool, and a query result is obtained;
And splicing and packaging the query results, and returning the data to the front-end interface after packaging is finished.
Preferably, the front-end rendering module is configured to perform the following operations:
packaging page components used by the dimension data model and the index data model constructed in the background into a general application label;
Selecting a data model through front-end interface dragging, and constructing a business report through the front-end interface dragging dimension and index;
dynamically updating the calculation method and the data source of the index model according to the real-time data, and rendering the service report in real time;
and dynamically adjusting the layout and the look-up of the business report according to the user interaction.
The method and the system for dynamically rendering the report based on the index model have the following advantages: the user can dynamically generate the report through selecting different data items at the front end, screening, inquiring and other operations on the data, and the user and the implementation personnel without program development capability can dynamically generate the report through selecting dimensions and indexes, so that the data quality and the data consistency can be ensured while the requirements of quick corresponding business requirement change are met.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method for dynamically rendering a report based on an index model according to embodiment 1;
fig. 2 is a flow chart of front-end rendering in a method for dynamically rendering a report based on an index model according to embodiment 1.
Detailed Description
The invention will be further described with reference to the accompanying drawings and specific examples, so that those skilled in the art can better understand the invention and implement it, but the examples are not meant to limit the invention, and the technical features of the embodiments of the invention and the examples can be combined with each other without conflict.
The embodiment of the invention provides a method and a system for dynamically rendering a report based on an index model, which are used for solving the technical problems that the report cannot be dynamically adjusted and rendered during development and guaranteeing the consistency and accuracy of data.
Example 1:
the invention discloses a method for dynamically rendering a report based on an index model, which is applied to a system comprising a front end and a background, as shown in figure 1, and comprises four steps of data acquisition, data processing, digital modeling and front end rendering.
Step S100, data acquisition: for enterprises to be acquired, carrying out standardized processing on each business system data in the enterprises, and acquiring global business data in the enterprises through a unified data acquisition channel.
The data acquisition channel in this embodiment supports data acquisition from structured, semi-structured, and unstructured data sources, and supports multiple synchronization modes in batch, real-time, full-scale, or incremental, and supports acquisition modes for offline data acquisition and real-time data acquisition.
The implementation of the embodiment requires that a set of data processing suite is provided by a user side for carrying out operations such as standardization, mild summarization processing, personalized data processing and the like on the original data, and mainly used data computing resources comprise basic resources such as big data computing, high-performance analysis type databases and the like.
When the data is acquired, the unified data acquisition channel provided by the data processing suite is used for supporting multiple structured, semi-structured and unstructured data sources in a framework, supporting multiple synchronous modes of batch or real-time, full or incremental, and realizing acquisition of mass offline data and real-time data. Before data acquisition, unified standardization and normalization are carried out on data of each business system in an enterprise according to business plates and data fields, the data acquisition strictly follows data standards, the acquisition of internal and external data sources of the enterprise is completed, and the acquired data content contains global business data in the enterprise. The data elements of each business system in the big data computing library must be uniformly defined and standard field coded, so that each business department in the enterprise can read and understand the business dimension and measurement, and can correctly use the related dimension and measurement.
Step S200, data processing: processing the collected service data based on the application scene, dividing the service data into three layers of ODS, CDM and ADS, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background.
As a specific implementation of data processing, based on the data acquisition and the large data calculation library, based on different data application scenes, the data is processed into semi-finished products or finished products from raw materials in various modes, so as to support the data consumption requirements of different scenes.
Data processing the data processing is divided into three layers, ODS/CDM/ADS, according to industry best practices. Based on the data processing suite, a link relation based on a fact table, a dimension table and each other is established through a clear business relation, so that light processing summarization of the universality summarization data is realized, the light processing summarization table has universality, and the deep processing of personalized business by a user can be supported while the requirement of data modeling can be met.
Step S300, modeling data: the method comprises the steps of constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary object, carrying out primary key field association on the dimension data model and the index data model to form a data model mapping relation, constructing a wide-table full-dimension data model, storing the association relation between the dimension data model and the index data model into a database, and integrating the dimension data model, the index data model and the full-dimension data model constructed in the background with a front-end interface.
The data modeling of the embodiment relies on the data summary table and the dimension table after the data processing treatment, builds a data model for the summary data table and the dimension data for the data report, and specifically comprises four steps of building the dimension data model, building the index data model, building the model association relationship and data integration application.
Constructing a dimension data model: when a dimension data model is constructed based on metadata information in a dimension physical table, the dimension physical table metadata information in a database is read, the field name, the field meaning and the field type of the dimension physical table are identified, and the dimension data model is defined through normalized physical table field meaning and name.
The dimension data model comprises a source physical table of a dimension field, a mapping relation between the dimension field and a physical table field, an identification weight of the dimension field associated source physical table, filtering conditions of the source physical table, a data dictionary name of the dimension field and service classification of the dimension data model.
Constructing an index data model: when an index data model is constructed based on metadata information in summarized matters, reading summarized physical table metadata information in a database, identifying the field names, field meanings and field types of the summarized physical table, and defining the index data model by normalizing the field meanings and the field names of the physical table;
The index data model comprises a plurality of different data types, wherein the data types comprise a numerical type, a text type and a date type, each index field of the index data model is provided with a group of index functions, each index function comprises a sum, a total number, an average value and a variance, the index data model comprises source physical table weights of dimension fields, filtering conditions of the source physical table, excluding index non-support dimensions and index statistical units.
Constructing a model association relation: and carrying out primary key field association on the dimension data model and the index data model, constructing a full dimension data model in a large-width table form, and storing the association relation of the data model into a service database, so that the assembly processing of the data query SQL is convenient when the subsequent user requests data.
Data model integration application: after the background data model and the front-end application are integrated technically, a user can select different data items and screening conditions through components such as a selection frame, a drop-down list and the like on an interface, and according to the selected relevant dimension and index, a result is dynamically displayed on a visual interface, and a required report result is obtained by adjusting different parameters.
After integrating the dimension data model, the index data model and the full dimension data model with the front end interface, the front end interface and the background are matched for executing the following steps:
(1) Selecting different data items and screening conditions through a front-end interface, wherein the data items comprise data sources, dimensions and indexes, and the screening conditions comprise query conditions and sorting conditions;
(2) Acquiring the dimension, index, query condition and sorting condition selected by the front-end interface, and searching the actual physical table and field information according to the mapping relation of the data model;
(3) According to the relation between the dimension data model and the index data model, splicing SQL scripts of different data to be queried according to indexes and query conditions;
(4) All SQL scripts are executed concurrently through a dispatching thread pool, and a query result is obtained;
(5) And splicing and packaging the query results, and returning the data to the front-end interface after packaging is finished.
Step S400 front end rendering: and loading a dimension data model and an index data model on the front-end interface based on the data request to form a business report, and rendering and dynamically displaying the business report.
In this embodiment, the front-end rendering includes the following steps:
(1) Packaging page components used by the dimension data model and the index data model constructed in the background into a general application label;
(2) Selecting a data model through front-end interface dragging, and constructing a business report through the front-end interface dragging dimension and index;
(3) Dynamically updating the calculation method and the data source of the index model according to the real-time data, and rendering the service report in real time;
(4) And dynamically adjusting the layout and the look-up of the business report according to the user interaction.
As the implementation of front-end rendering, prospective popular frames such as node. Js and Vue, react, elementUI are introduced into a front-end development program, the frames are subjected to secondary packaging, page components used by a dimension and index model constructed in the background are packaged into a universal application label, a user can drag and select a data model in a main functional interface, and a service report is combined by dragging the dimension and the index. The front-end interface dynamically adjusts the data and the display effect of the report according to the real-time data and the indexes and the dimensions selected by the user operation. Specifically, the calculation method and the data source of the index model are dynamically updated according to the real-time data, and the report is rendered in real time. Meanwhile, the layout and style of the report form are dynamically adjusted according to user interaction, so that requirements in different scenes are met.
When the report is rendered as shown in fig. 2, the dimension and the index in the request are analyzed through the front end, the index calculation formula is identified, the actual physical table and the field information are searched according to the mapping relation, the actual physical table and the field information are specifically the mapping relation of the data model, the SQL scripts of different data to be queried are spliced according to the dimension data model and the index data model relation, all SQL scripts are executed in batches through the dispatching thread pool, SQL return results are collected after execution is finished, index calculation is carried out to generate total, calculation column processing is carried out based on the total, the final query result is obtained, the query result is spliced and packaged, and the data are returned to the front end interface after packaging is finished.
Example 2:
The invention discloses a system for dynamically rendering a report based on an index model, which is applied between a front-end interface and a background and is used for rendering a business report by the method disclosed in embodiment 1.
The data acquisition module is used for executing the following steps: for enterprises to be acquired, carrying out standardized processing on each business system data in the enterprises, and acquiring global business data in the enterprises through a unified data acquisition channel.
The data acquisition channel in this embodiment supports data acquisition from structured, semi-structured, and unstructured data sources, and supports multiple synchronization modes in batch, real-time, full-scale, or incremental, and supports acquisition modes for offline data acquisition and real-time data acquisition.
The implementation of the embodiment requires that a set of data processing suite is provided by a user side for carrying out operations such as standardization, mild summarization processing, personalized data processing and the like on the original data, and mainly used data computing resources comprise basic resources such as big data computing, high-performance analysis type databases and the like.
When the data is acquired, the unified data acquisition channel provided by the data processing suite is used for supporting multiple structured, semi-structured and unstructured data sources in a framework, supporting multiple synchronous modes of batch or real-time, full or incremental, and realizing acquisition of mass offline data and real-time data. Before data acquisition, unified standardization and normalization are carried out on data of each business system in an enterprise according to business plates and data fields, the data acquisition strictly follows data standards, the acquisition of internal and external data sources of the enterprise is completed, and the acquired data content contains global business data in the enterprise. The data elements of each business system in the big data computing library must be uniformly defined and standard field coded, so that each business department in the enterprise can read and understand the business dimension and measurement, and can correctly use the related dimension and measurement.
The data processing module is used for executing the following steps: processing the collected service data based on the application scene, dividing the service data into three layers of ODS, CDM and ADS, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background.
As a specific implementation of data processing, based on the data acquisition and the large data calculation library, based on different data application scenes, the data is processed into semi-finished products or finished products from raw materials in various modes, so as to support the data consumption requirements of different scenes.
Data processing the data processing is divided into three layers, ODS/CDM/ADS, according to industry best practices. Based on the data processing suite, a link relation based on a fact table, a dimension table and each other is established through a clear business relation, so that light processing summarization of the universality summarization data is realized, the light processing summarization table has universality, and the deep processing of personalized business by a user can be supported while the requirement of data modeling can be met.
The data modeling module is used for executing the following steps: the method comprises the steps of constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary object, carrying out primary key field association on the dimension data model and the index data model to form a data model mapping relation, constructing a wide-table full-dimension data model, storing the association relation between the dimension data model and the index data model into a database, and integrating the dimension data model, the index data model and the full-dimension data model constructed in the background with a front-end interface.
In the embodiment, the data modeling relies on the data summary table and the dimension table after the data processing treatment, and the summary data table and the dimension data are constructed into a data model for the data report. The data modeling module of the embodiment is used for executing four operations of constructing a dimension data model, constructing an index data model, constructing a model association relationship and applying data integration.
When constructing the dimension data model, the data modeling module is used for executing the following operations: when a dimension data model is constructed based on metadata information in a dimension physical table, the dimension physical table metadata information in a database is read, the field name, the field meaning and the field type of the dimension physical table are identified, and the dimension data model is defined through normalized physical table field meaning and name.
The dimension data model comprises a source physical table of a dimension field, a mapping relation between the dimension field and a physical table field, an identification weight of the dimension field associated source physical table, filtering conditions of the source physical table, a data dictionary name of the dimension field and service classification of the dimension data model.
When the index data model is constructed, the data modeling module is used for executing the following operations: when an index data model is constructed based on metadata information in the summarized substance, reading summarized physical table metadata information in a database, identifying the field name, the field meaning and the field type of the summarized physical table, and defining the index data model by normalizing the field meaning and the field name of the physical table.
The index data model comprises a plurality of different data types, wherein the data types comprise a numerical type, a text type and a date type, each index field of the index data model is provided with a group of index functions, each index function comprises a sum, a total number, an average value and a variance, the index data model comprises source physical table weights of dimension fields, filtering conditions of the source physical table, excluding index non-support dimensions and index statistical units.
When the association relation of the model is constructed, the data modeling module is used for executing the following operations: and carrying out primary key field association on the dimension data model and the index data model, constructing a full dimension data model in a large-width table form, and storing the association relation of the data model into a service database, so that the assembly processing of the data query SQL is convenient when the subsequent user requests data.
After the background data model and the front-end application are integrated technically through the data modeling module, a user can select different data items and screening conditions through components such as a selection frame, a drop-down list and the like on an interface, and according to the selected relevant dimension and index, a result is dynamically displayed on a visual interface, and a required report result is obtained through adjusting different parameters.
After integrating the dimension data model, the index data model and the full dimension data model with the front end interface, the front end interface and the background are matched for executing the following steps:
(1) Selecting different data items and screening conditions through a front-end interface, wherein the data items comprise data sources, dimensions and indexes, and the screening conditions comprise query conditions and sorting conditions;
(2) Acquiring the dimension, index, query condition and sorting condition selected by the front-end interface, and searching the actual physical table and field information according to the mapping relation of the data model;
(3) According to the relation between the dimension data model and the index data model, splicing SQL scripts of different data to be queried according to indexes and query conditions;
(4) All SQL scripts are executed concurrently through a dispatching thread pool, and a query result is obtained;
(5) And splicing and packaging the query results, and returning the data to the front-end interface after packaging is finished.
The front-end rendering module is configured to perform the following: and loading a dimension data model and an index data model on the front-end interface based on the data request to form a business report, and rendering and dynamically displaying the business report.
In this embodiment, the front-end rendering module is configured to perform the following operations:
(1) Packaging page components used by the dimension data model and the index data model constructed in the background into a general application label;
(2) Selecting a data model through front-end interface dragging, and constructing a business report through the front-end interface dragging dimension and index;
(3) Dynamically updating the calculation method and the data source of the index model according to the real-time data, and rendering the service report in real time;
(4) And dynamically adjusting the layout and the look-up of the business report according to the user interaction.
As the implementation of front-end rendering, prospective popular frames such as node. Js and Vue, react, elementUI are introduced into a front-end development program, the frames are subjected to secondary packaging, page components used by a dimension and index model constructed in the background are packaged into a universal application label, a user can drag and select a data model in a main functional interface, and a service report is combined by dragging the dimension and the index. The front-end interface dynamically adjusts the data and the display effect of the report according to the real-time data and the indexes and the dimensions selected by the user operation. Specifically, the calculation method and the data source of the index model are dynamically updated according to the real-time data, and the report is rendered in real time. Meanwhile, the layout and style of the report form are dynamically adjusted according to user interaction, so that requirements in different scenes are met.
When the report is rendered, the dimensionality and the index in the request are analyzed through the front end, an index calculation formula is identified, the actual physical table and field information are searched according to the mapping relation, the actual physical table and field information are specifically mapped to the data model, SQL scripts of different data to be queried are spliced according to the dimensionality data model and the index data model relation, all SQL scripts are executed in batches through a dispatching thread pool, SQL return results are collected after execution is finished, index calculation is carried out to generate total, calculation column processing is carried out based on the total, a final query result is obtained, the query result is spliced and packaged, and the data are returned to the front end interface after packaging is finished.
While the invention has been illustrated and described in detail in the drawings and in the preferred embodiments, the invention is not limited to the disclosed embodiments, but it will be apparent to those skilled in the art that many more embodiments of the invention can be made by combining the means of the various embodiments described above and still fall within the scope of the invention.
Claims (2)
1. A method for dynamically rendering a report based on an index model, which is applied to a system comprising a front end and a background, the method comprising the following steps:
and (3) data acquisition: for an enterprise to be acquired, carrying out standardized processing on each business system data in the enterprise, and acquiring global business data in the enterprise through a unified data acquisition channel;
Data processing: processing the collected service data based on an application scene, dividing the service data into three layers of ODS, CDM and ADS, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background;
Modeling data: constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary object, carrying out primary key field association on the dimension data model and the index data model to form a data model mapping relation, constructing a full dimension data model in a wide table form, storing the association relation between the dimension data model and the index data model into a database, and integrating the dimension data model, the index data model and the full dimension data model constructed in the background with a front end interface;
Front-end rendering: loading a dimension data model and an index data model on a front-end interface based on a data request to form a business report, and rendering and dynamically displaying the business report;
the data acquisition channel supports data acquisition from structured, semi-structured and unstructured data sources, supports a plurality of synchronous modes of batch, real-time, full-quantity or increment, and supports an offline data acquisition mode and a real-time data acquisition mode;
When a dimension data model is built based on metadata information in a dimension physical table, reading the dimension physical table metadata information in a database, identifying a field name, a field meaning and a field type of the dimension physical table, and defining the dimension data model by normalizing the field meaning and the field type of the physical table;
The dimension data model comprises a source physical table of a dimension field, a mapping relation between the dimension field and a physical table field, an identification weight of the dimension field associated source physical table, filtering conditions of the source physical table, a data dictionary name of the dimension field and service classification of the dimension data model;
When an index data model is constructed based on metadata information in summarized matters, reading summarized physical table metadata information in a database, identifying the field names, field meanings and field types of the summarized physical table, and defining the index data model by normalizing the field meanings and the field names of the physical table;
The index data model comprises a plurality of different data types, wherein the data types comprise a numerical type, a text type and a date type, each index field of the index data model is provided with a group of index functions, each index function comprises a sum, a total bar number, an average value and a variance, the index data model comprises source physical table weights of dimension fields, filtering conditions of source physical tables, excluding non-support dimension of indexes and index statistical units;
After integrating the dimension data model, the index data model and the full dimension data model with the front end interface, the front end interface and the background are matched for executing the following steps:
Selecting different data items and screening conditions through a front-end interface, wherein the data items comprise data sources, dimensions and indexes, and the screening conditions comprise query conditions and sorting conditions;
acquiring the dimension, index, query condition and sorting condition selected by the front-end interface, and searching the actual physical table and field information according to the mapping relation of the data model;
According to the relation between the dimension data model and the index data model, splicing SQL scripts of different data to be queried according to indexes and query conditions;
all SQL scripts are executed concurrently through a dispatching thread pool, and a query result is obtained;
splicing and packaging the query result, and returning the data to the front end interface after packaging is finished;
the front-end rendering comprises the following steps:
packaging page components used by the dimension data model and the index data model constructed in the background into a general application label;
Selecting a data model through front-end interface dragging, and constructing a business report through the front-end interface dragging dimension and index;
dynamically updating the calculation method and the data source of the index model according to the real-time data, and rendering the service report in real time;
And dynamically adjusting the layout and the style of the business report according to the user interaction.
2. A system for dynamically rendering a report based on an index model is characterized by being applied between a front-end interface and a background and used for rendering a business report by the method for dynamically rendering the report based on the index model according to claim 1, and comprises a data acquisition module, a data processing module, a data modeling module and a front-end rendering module;
The data acquisition module is used for executing the following steps: for an enterprise to be acquired, carrying out standardized processing on each business system data in the enterprise, and acquiring global business data in the enterprise through a unified data acquisition channel;
The data processing module is used for executing the following steps: processing the collected service data based on an application scene, dividing the service data into three layers of ODS, CDM and ADS, and establishing a summary physical table, a dimension physical table and an association relation between the summary physical table and the dimension physical table in the background;
the data modeling module is used for executing the following steps: constructing a dimension data model based on metadata information in a dimension physical table, constructing an index data model based on metadata information in a summary object, carrying out primary key field association on the dimension data model and the index data model to form a data model mapping relation, constructing a full dimension data model in a wide table form, storing the association relation between the dimension data model and the index data model into a database, and integrating the dimension data model, the index data model and the full dimension data model constructed in the background with a front end interface;
The front-end rendering module is configured to perform the following: loading a dimension data model and an index data model on a front-end interface based on a data request to form a business report, and rendering and dynamically displaying the business report;
the data acquisition channel supports data acquisition from structured, semi-structured and unstructured data sources, supports a plurality of synchronous modes of batch, real-time, full-quantity or increment, and supports an offline data acquisition mode and a real-time data acquisition mode;
When constructing a dimension data model based on metadata information in a dimension physical table, the data modeling module is used for executing the following steps: reading dimension physical table metadata information in a database, identifying a field name, a field meaning and a field type of a dimension physical table, and defining a dimension data model through normalizing the field meaning and the field type of the physical table;
The dimension data model comprises a source physical table of a dimension field, a mapping relation between the dimension field and a physical table field, an identification weight of the dimension field associated source physical table, a filtering condition of the source physical table, a data dictionary name of the dimension field and service classification of the dimension data model;
When an index data model is constructed based on metadata information in the summary object, the data modeling module is configured to perform the following steps: reading summarized physical table metadata information in a database, identifying the names, field meanings and field types of summarized physical table fields, and defining an index data model through normalizing the field meanings and names of the physical tables;
The index data model comprises a plurality of different data types, wherein the data types comprise a numerical type, a text type and a date type, each index field of the index data model is provided with a group of index functions, each index function comprises a sum, a total bar number, an average value and a variance, the index data model comprises source physical table weights of dimension fields, filtering conditions of the source physical table, excluding index unsupported dimension and index statistical units;
After integrating the dimension data model, the index data model and the full dimension data model with the front end interface, the front end interface and the background are matched for executing the following steps:
Selecting different data items and screening conditions through a front-end interface, wherein the data items comprise data sources, dimensions and indexes, and the screening conditions comprise query conditions and sorting conditions;
acquiring the dimension, index, query condition and sorting condition selected by the front-end interface, and searching the actual physical table and field information according to the mapping relation of the data model;
According to the relation between the dimension data model and the index data model, splicing SQL scripts of different data to be queried according to indexes and query conditions;
all SQL scripts are executed concurrently through a dispatching thread pool, and a query result is obtained;
splicing and packaging the query result, and returning the data to the front end interface after packaging is finished;
The front-end rendering module is used for executing the following operations:
packaging page components used by the dimension data model and the index data model constructed in the background into a general application label;
Selecting a data model through front-end interface dragging, and constructing a business report through the front-end interface dragging dimension and index;
dynamically updating the calculation method and the data source of the index model according to the real-time data, and rendering the service report in real time;
And dynamically adjusting the layout and the style of the business report according to the user interaction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410085436.9A CN117610519B (en) | 2024-01-22 | 2024-01-22 | Method and system for dynamically rendering report based on index model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410085436.9A CN117610519B (en) | 2024-01-22 | 2024-01-22 | Method and system for dynamically rendering report based on index model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117610519A CN117610519A (en) | 2024-02-27 |
| CN117610519B true CN117610519B (en) | 2024-05-03 |
Family
ID=89952038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410085436.9A Active CN117610519B (en) | 2024-01-22 | 2024-01-22 | Method and system for dynamically rendering report based on index model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117610519B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104636393A (en) * | 2013-11-13 | 2015-05-20 | 航天信息股份有限公司 | Self-adaptation report form establishing method based on user self-definition behavior analysis |
| CN106933544A (en) * | 2015-12-29 | 2017-07-07 | 航天信息股份有限公司 | Declare table generating method and system |
| CN109669949A (en) * | 2018-12-27 | 2019-04-23 | 广州云趣信息科技有限公司 | A kind of dynamic report generation method and system realization based on data model |
| CN109684616A (en) * | 2018-12-13 | 2019-04-26 | 山东浪潮通软信息科技有限公司 | Dynamic statement formula assembles the method and system made a report on |
| CN110276059A (en) * | 2019-06-24 | 2019-09-24 | 银联商务股份有限公司 | A kind for the treatment of method and apparatus of dynamic statement |
| CN113934820A (en) * | 2021-10-14 | 2022-01-14 | 广州广电运通金融电子股份有限公司 | Visual processing system, method, storage medium and terminal for unstructured data |
| CN115470195A (en) * | 2022-09-29 | 2022-12-13 | 信华信技术股份有限公司 | Index data automatic calculation method and device fusing dimension models |
| CN116306538A (en) * | 2023-02-23 | 2023-06-23 | 上海乾臻信息科技有限公司 | Dynamic report generation method, device, system and storage medium |
| CN116578627A (en) * | 2023-05-05 | 2023-08-11 | 山东浪潮数字商业科技有限公司 | Method and device for providing data support for multi-service platform, medium and equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10509794B2 (en) * | 2017-04-28 | 2019-12-17 | Splunk Inc. | Dynamically-generated files for visualization sharing |
-
2024
- 2024-01-22 CN CN202410085436.9A patent/CN117610519B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104636393A (en) * | 2013-11-13 | 2015-05-20 | 航天信息股份有限公司 | Self-adaptation report form establishing method based on user self-definition behavior analysis |
| CN106933544A (en) * | 2015-12-29 | 2017-07-07 | 航天信息股份有限公司 | Declare table generating method and system |
| CN109684616A (en) * | 2018-12-13 | 2019-04-26 | 山东浪潮通软信息科技有限公司 | Dynamic statement formula assembles the method and system made a report on |
| CN109669949A (en) * | 2018-12-27 | 2019-04-23 | 广州云趣信息科技有限公司 | A kind of dynamic report generation method and system realization based on data model |
| CN110276059A (en) * | 2019-06-24 | 2019-09-24 | 银联商务股份有限公司 | A kind for the treatment of method and apparatus of dynamic statement |
| CN113934820A (en) * | 2021-10-14 | 2022-01-14 | 广州广电运通金融电子股份有限公司 | Visual processing system, method, storage medium and terminal for unstructured data |
| CN115470195A (en) * | 2022-09-29 | 2022-12-13 | 信华信技术股份有限公司 | Index data automatic calculation method and device fusing dimension models |
| CN116306538A (en) * | 2023-02-23 | 2023-06-23 | 上海乾臻信息科技有限公司 | Dynamic report generation method, device, system and storage medium |
| CN116578627A (en) * | 2023-05-05 | 2023-08-11 | 山东浪潮数字商业科技有限公司 | Method and device for providing data support for multi-service platform, medium and equipment |
Non-Patent Citations (1)
| Title |
|---|
| 保险绩效系统动态报表系统设计与研究;姚佳琦;;内蒙古科技与经济;20170430(第08期);第75-76页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117610519A (en) | 2024-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11776084B2 (en) | Patent mapping | |
| US11386085B2 (en) | Deriving metrics from queries | |
| Thürer et al. | On the meaning of ‘waste’: review and definition | |
| CN110955717A (en) | Visual dynamic display method and system based on big data | |
| AU735010B3 (en) | Business intelligence system | |
| US20060218160A1 (en) | Change control management of XML documents | |
| US20130166563A1 (en) | Integration of Text Analysis and Search Functionality | |
| US20060215832A1 (en) | Data access service queries | |
| US20150186776A1 (en) | Contextual data analysis using domain information | |
| KR101505858B1 (en) | A templet-based online composing system for analyzing reports or views of big data by providing past templets of database tables and reference fields | |
| US11698918B2 (en) | System and method for content-based data visualization using a universal knowledge graph | |
| DE102012221251A1 (en) | Semantic and contextual search of knowledge stores | |
| CN112100200A (en) | Method for automatically generating SQL (structured query language) statements based on dimension model | |
| US20060026174A1 (en) | Patent mapping | |
| US8260772B2 (en) | Apparatus and method for displaying documents relevant to the content of a website | |
| US10146881B2 (en) | Scalable processing of heterogeneous user-generated content | |
| US8615733B2 (en) | Building a component to display documents relevant to the content of a website | |
| EP1774432A4 (en) | Patent mapping | |
| US10789261B1 (en) | Visual distributed data framework for analysis and visualization of datasets | |
| US20070282804A1 (en) | Apparatus and method for extracting database information from a report | |
| CN117610519B (en) | Method and system for dynamically rendering report based on index model | |
| CN116010439A (en) | Visual Chinese SQL system and query construction method | |
| US20230044287A1 (en) | Semantics based data and metadata mapping | |
| CN114116773A (en) | Structured Query Language (SQL) text auditing method and device | |
| Spielberg et al. | The FachRef-Assistant: Personalised, subject specific, and transparent stock management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |