CN108805597B - Model construction method and device and data report generation method and device - Google Patents
Model construction method and device and data report generation method and device Download PDFInfo
- Publication number
- CN108805597B CN108805597B CN201710312337.XA CN201710312337A CN108805597B CN 108805597 B CN108805597 B CN 108805597B CN 201710312337 A CN201710312337 A CN 201710312337A CN 108805597 B CN108805597 B CN 108805597B
- Authority
- CN
- China
- Prior art keywords
- index
- construction
- basic
- indexes
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
Landscapes
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Engineering & Computer Science (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Data Mining & Analysis (AREA)
- Economics (AREA)
- Marketing (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a model construction method and a device thereof, and a data report generation method and a device thereof, wherein the model construction method comprises the following steps: executing the construction tasks of all basic indexes in parallel to generate a plurality of basic index tables; respectively carrying out time sequence analysis on the construction tasks of the summary indexes to determine the fastest construction path of each summary index; respectively executing the construction tasks of the summary indexes according to the fastest construction paths to generate a plurality of summary index tables; and constructing a data model according to the basic index tables and the summary index tables. The construction task of the basic indexes is executed based on the source data table depended by the basic indexes, and the construction task of the summary indexes is executed based on the basic indexes and/or the summary index table depended by the summary indexes. The method and the device fully utilize the resource advantages of the cluster, greatly shorten the calculation time and ensure the timeliness of the output result.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a model construction method and device and a data report generation method and device.
Background
In the construction of data marts, with the development of business systems, the data volume is increased, the index number is increased, and the dependency relationship of data tables is complicated.
On the one hand, the above problems may lead to a late time for generating the final fact table, and the results cannot be generated in time for analysis and summarization every day, so that the timeliness of the data cannot be utilized. For example, in a sales morning meeting at 9 o 'clock per day, analysis and summarization using data from the previous day are required, and if the results cannot be produced at 9 o' clock in time, the timeliness of the data from the previous day cannot be used.
On the other hand, due to the complex business situation, the indexes need to be adjusted frequently according to the actions of competitors, and the problem also causes the overall backtracking of the downstream of the adjusted indexes depending on the indexes during each adjustment, so that the backtracking task amount is very large.
Disclosure of Invention
In view of the above-mentioned drawbacks and deficiencies in the prior art, it is desirable to provide a model building method and apparatus, a data report generating method and apparatus, which shorten data processing time to ensure timeliness of output results; and it is desirable to further provide a model construction method and apparatus, a data report generation method and apparatus for reducing the number of backtracking tasks after index adjustment.
In a first aspect, the present invention provides a model building method, comprising:
executing the construction tasks of all basic indexes in parallel to generate a plurality of basic index tables;
respectively carrying out time sequence analysis on the construction tasks of the summary indexes to determine the fastest construction path of each summary index;
respectively executing the construction tasks of the summary indexes according to the fastest construction paths to generate a plurality of summary index tables;
and constructing a data model according to the basic index tables and the summary index tables.
The construction task of the basic indexes is executed based on the source data table depended by the basic indexes, and the construction task of the summary indexes is executed based on the basic indexes and/or the summary index table depended by the summary indexes.
In a second aspect, the present invention further provides a data report generating method, including the above model building method, and generating a data report according to the data model.
In a third aspect, the present invention provides a model building apparatus, comprising:
the basic index building unit is used for executing the building tasks of all basic indexes in parallel and generating a plurality of basic index tables;
the time sequence analysis unit is configured to perform time sequence analysis on the construction tasks of the summary indexes respectively so as to determine the fastest construction path of each summary index;
the summary index construction unit is configured to execute construction tasks of summary indexes according to the fastest construction paths respectively and generate a plurality of summary index tables;
and the model building unit is configured for building a data model according to the basic index tables and the summary index tables.
The construction task of the basic indexes is executed based on the source data table depended by the basic indexes, and the construction task of the summary indexes is executed based on the basic indexes and/or the summary index table depended by the summary indexes.
In a fourth aspect, the present invention further provides a data report generating device, including the model building device, and a report generating unit, configured to generate a data report according to the data model.
In a fifth aspect, the present invention also provides an apparatus comprising one or more processors and a memory, wherein the memory contains instructions executable by the one or more processors to cause the one or more processors to perform a model building method or a data table generation method provided according to embodiments of the present invention.
In a sixth aspect, the present invention also provides a computer-readable storage medium storing a computer program for causing a computer to execute a model building method or a data table generating method provided according to embodiments of the present invention.
The model construction method and device, and the data report generation method and device provided by the embodiments of the invention adopt a parallel mechanism in the construction of the basic index, and determine the fastest construction path of the summary index by performing time sequence analysis on the construction task in the construction of the summary index, so that the resource advantages of the cluster are fully utilized, the calculation time is greatly shortened, and the timeliness of the output result is ensured;
the model construction method and device and the data report generation method and device provided by some embodiments of the invention further reduce the number of backtracking tasks after adjusting the source data table by aggregating all basic indexes constructed by relying on the same source data table in the same basic index table for output;
the model construction method and device and the data report generation method and device provided by some embodiments of the invention further perform the construction task of the complex logic index independently by performing the construction task of the simple logic index in a polymerization manner, thereby further shortening the calculation time;
the model construction method and device and the data report generation method and device provided by some embodiments of the invention further reduce the number of backtracking tasks after the indexes are adjusted by aggregating simple logic indexes of the same dimension into the same summary index table for output and independently generating the summary index table for output from complex logic indexes.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
fig. 1 is a flowchart of a model building method according to an embodiment of the present invention.
Fig. 2 is a flowchart of step S30 in a preferred embodiment of the method shown in fig. 1.
FIG. 3 is a flow diagram of a preferred embodiment of the method shown in FIG. 1.
Fig. 4 is a flowchart of a data table generating method according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of a model building apparatus according to an embodiment of the present invention.
Fig. 6 is a schematic structural view of a preferred embodiment of the apparatus shown in fig. 5.
Fig. 7 is a schematic structural view of a preferred embodiment of the apparatus shown in fig. 5.
Fig. 8 is a schematic structural diagram of a data table generating apparatus according to an embodiment of the present invention.
Fig. 9 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 is a flowchart of a model building method according to an embodiment of the present invention.
As shown in fig. 1, in this embodiment, the model building method provided by the present invention includes:
s10: executing the construction tasks of all basic indexes in parallel to generate a plurality of basic index tables;
s30: respectively carrying out time sequence analysis on the construction tasks of the summary indexes to determine the fastest construction path of each summary index;
s50: respectively executing the construction tasks of the summary indexes according to the fastest construction paths to generate a plurality of summary index tables;
s70: and constructing a data model according to the basic index tables and the summary index tables.
The construction task of the basic indexes is executed based on the source data table depended by the basic indexes, and the construction task of the summary indexes is executed based on the basic indexes and/or the summary index table depended by the summary indexes.
Specifically, in this embodiment, in step S10, one basic index table is generated according to the basic indexes built by relying on the same source data table, for example, basic indexes a, B, and C are built by relying on the source data table a, d, and e are built by relying on the source data table B, and f, g, h, and i are built by relying on the source data table C, then the first basic index table generated in step S10 includes the basic indexes a, B, and C, the second basic index table includes the basic indexes d and e, and the third basic index table includes the basic indexes f, g, h, and i. When any source data table needs to be adjusted, for example, the source data table a needs to be adjusted, only the first basic index table is affected, and the second basic index table and the third basic index table are not affected, so that the number of backtracking tasks after the source data table is adjusted is reduced.
In further embodiments, different basic index table generation strategies may also be configured according to actual needs, for example, a basic index table is separately generated for each basic index, or sub-table output is performed on each basic index constructed by depending on the same source data table according to a preset grouping strategy, and so on.
In this embodiment, the summary index includes a simple logic index and a complex logic index. The simple logic index is a summary index with simpler constructed operational logic, and the complex logic index is a summary index with more complex constructed operational logic. In this embodiment, the simple logical indicator is a summary indicator constructed by a sum operation (sum function), and the complex logical indicator is a summary indicator constructed by any other means except the sum operation.
In more embodiments, simple logic indexes and complex logic indexes distinguished according to different classification rules (for example, different operation types are set as the classification rules, or a certain operation magnitude is set as the classification rules, etc.) may be configured according to actual requirements; or according to actual needs, the simple logic index and the complex logic index are not distinguished, and all summary indexes are regarded as complex logic indexes.
In step S30, for the summary index where there are multiple build paths, the fastest build path is determined by performing time series analysis on the build tasks of each build path. Specifically, the time sequence analysis method may be configured to analyze historical time consumption and completion time of each construction task according to actual requirements, or run a simulation task in real time to obtain time consumption of the construction task and predict actual completion time, and the like.
Preferably, only the construction tasks of the complex logic indexes can be subjected to time sequence analysis to determine the fastest construction path; any one of the construction paths can be directly selected as the fastest construction path for the simple logic index.
In step S50, each summary index is constructed according to the fastest construction path determined in step S30, and several summary index tables are generated. Specifically, in this embodiment, the construction tasks of the simple logic indexes in the same dimension are executed in an aggregated manner, and the construction tasks of the complex logic indexes are executed separately. Wherein, the dimension is the angle for observing each basic index and each summary index. For example, for take-away products, the dimensions include a bouquet dimension, a store dimension, a city dimension, etc., and different dimensions may be configured for different products.
Preferably, a summary index table is generated according to each simple logic index constructed under the same dimensionality, and the summary index table is separately generated according to each complex logic index. When any basic index needs to be adjusted, only the summary index table generated by the aggregated simple logic index is affected (if no simple logic index depends on the basic index, the summary index table is not affected), and the summary index table generated by the complex logic index depending on the basic index does not affect the summary index tables generated according to other complex logic indexes, so that the backtracking task quantity after the source data table is adjusted is reduced.
In more embodiments, the aggregation execution strategy of the construction tasks of the simple logic indexes can be configured to aggregate and execute the construction tasks of the simple logic indexes that are independent of each other according to actual requirements, or the construction tasks of the simple logic indexes are executed independently, and the like; and configuring the generation strategy of the summary index table to generate the summary index table according to each simple logic index independently, and the like.
In step S70, the type of the data model is a constellation model. In further embodiments, the type of the data model can be configured into other data models commonly used in the art according to actual requirements.
In the embodiment, a parallel mechanism is adopted in the construction of the basic indexes, and the fastest construction path of the summary index is determined by carrying out time sequence analysis on the construction tasks in the construction of the summary index, so that the resource advantages of the cluster are fully utilized, the calculation time is greatly shortened, and the timeliness of the output result is guaranteed; the basic indexes built by depending on the same source data table are aggregated in the same basic index table for output, so that the number of backtracking tasks after the source data table is adjusted is reduced; and, through aggregating the construction tasks of executing the simple logic indexes, independently executing the construction tasks of the complex logic indexes, further shortening the calculation time; and the simple logic indexes of the same dimensionality are aggregated to be output in the same summary index table, and the complex logic indexes are independently generated to be output in the summary index table, so that the backtracking task quantity after the indexes are adjusted is reduced.
Fig. 2 is a flowchart of step S30 in a preferred embodiment of the method shown in fig. 1.
As shown in fig. 2, in a preferred embodiment, step S30 includes:
s31: judging whether the first summary index has at least two construction paths:
otherwise, step S33 is executed: determining the only construction path of the first summary index as the fastest construction path;
if yes, go to step S35: acquiring the earliest starting time, the latest ending time and the average spending time of each basic index and/or summary index on which the first summary index depends in each construction path, and the average spending time of the first summary index in each construction path; and the number of the first and second groups,
s37: and calculating the latest end time of the first summary index in each construction path to determine the fastest construction path for constructing the first summary index.
For example, for the summary index t, the summary index t can be directly constructed based on the summary index r, i.e. r-t; or can be constructed based on a summary index s constructed according to r, namely r-s-t, then:
in step S31, it is determined that there are two building paths for the summary index t, and the process advances to step S35.
In step S35, the earliest start time, the latest end time, and the average elapsed time of each summary index on which the summary index t depends in the two building paths, and the average elapsed time of the summary index t in the two building paths are obtained:
in path r-t, r has an earliest start time of 6:30, a latest end time of 7:00, and an average elapsed time of 15 minutes; t takes an average of 30 minutes;
in the path r-s-t, the earliest starting time of r is 6:30, the latest ending time is 7:00, and the average time spent is 15 minutes; s has an earliest starting time of 6:30+15 to 6:45, an average elapsed time of 10 minutes, and a latest ending time of 7:00+10 to 7: 10; the average time taken for t is 10 minutes.
In step S37, the latest end times of t in the two construction paths are calculated:
in path r-t, t has a latest end time of 7:00+30 to 7: 30; namely, in the path r-t, the construction task of t can be completed at the latest 7: 30;
in path r-s-t, t has a latest end time of 7:10+10 to 7: 20; that is, in the path r-s-t, the construction task of t can be completed at the latest 7: 20.
Comparing 7:30 with 7:20, the path r-s-t is the fastest construction path of the summary index t.
FIG. 3 is a flow diagram of a preferred embodiment of the method shown in FIG. 1.
As shown in fig. 3, in a preferred embodiment, step S70 is preceded by:
s60: and executing the construction task of each statistical index according to each basic index table and each summary index table to generate a plurality of statistical index tables.
Specifically, the statistical index can be constructed by calculating common statistical calculation means such as the same ratio, the ring ratio, and the ratio.
In step S70, the data model is constructed based on the statistical index tables generated in step S60.
In a preferred embodiment, in step S60, a statistical index table is separately generated according to each statistical index, so as to reduce the number of backtracking tasks for generating the statistical index table after adjusting the indexes.
Fig. 4 is a flowchart of a data table generating method according to an embodiment of the present invention.
As shown in fig. 4, in this embodiment, the data table generating method provided by the present invention includes the model building method shown in fig. 3, and:
s90: and generating a data report according to the data model constructed in the step S70.
In further embodiments, the data report generation method may further use the data model constructed by the model construction method provided in any of the above embodiments to generate a data report.
Fig. 5 is a schematic structural diagram of a model building apparatus according to an embodiment of the present invention. The apparatus shown in fig. 5 may correspondingly perform the method shown in fig. 1.
As shown in fig. 5, in the present embodiment, the model building apparatus provided by the present invention includes a base index building unit 10, a time sequence analyzing unit 30, a summary index building unit 50, and a model building unit 70.
The basic index constructing unit 10 is configured to execute the task of constructing each basic index in parallel, and generate a plurality of basic index tables.
The timing analysis unit 30 is configured to perform timing analysis on the construction tasks of the summary indexes respectively to determine a fastest construction path of each summary index.
The summary index construction unit 50 is configured to execute the construction task of each summary index according to each fastest construction path, and generate a plurality of summary index tables.
The model building unit 70 is configured to build a data model according to the base index tables and the summary index tables.
The construction task of the basic indexes is executed based on the source data table depended by the basic indexes, and the construction task of the summary indexes is executed based on the basic indexes and/or the summary index table depended by the summary indexes. For a specific model construction principle, reference is made to the method shown in fig. 1, and details are not repeated here.
In a preferred embodiment, the base index constructing unit 10 is further configured to generate a base index table according to each base index constructed by relying on the same source data table.
In a preferred embodiment, the summary index constructing unit 50 is further configured to aggregate and execute the construction tasks of the simple logic indexes in independent or same dimension; and independently executing the construction task of each complex logic index.
In a preferred embodiment, the summary index constructing unit 50 is further configured to generate a summary index table according to each simple logic index constructed in the same dimension; and independently generating a summary index table according to each complex logic index.
Fig. 6 is a schematic structural view of a preferred embodiment of the apparatus shown in fig. 5. The apparatus shown in fig. 6 may correspondingly perform the method shown in fig. 2.
As shown in fig. 6, in a preferred embodiment, the timing analysis unit 30 includes a path detection subunit 31, a data acquisition subunit 33, and a timing analysis subunit 35.
The path detection subunit 31 is configured to determine whether the first summary index has at least two constructed paths: if not, determining the only construction path of the first summary index as the fastest construction path; if yes, the fastest building path for the first summary index is determined by the data acquisition subunit 33 and the timing analysis subunit 35.
The data obtaining subunit 33 is configured to obtain the earliest start time, the latest end time, and the average elapsed time of each base index and/or summary index on which the first summary index depends in each build path, and the average elapsed time of the first summary index in each build path.
The timing analysis subunit 35 is configured to calculate a latest end time of the first summary indicator in each build path to determine a fastest build path for building the first summary indicator.
For a specific timing analysis principle, refer to the method shown in fig. 2, which is not described herein again.
Fig. 7 is a schematic structural view of a preferred embodiment of the apparatus shown in fig. 5. The apparatus shown in fig. 7 may correspondingly perform the method shown in fig. 3.
As shown in fig. 7, in a preferred embodiment, the model building apparatus provided in the present invention further includes: a statistical indicator construction unit 60.
The statistical index constructing unit 60 is configured to execute a task of constructing each statistical index according to each basic index table and each summary index table, and generate a plurality of statistical index tables.
Correspondingly, the data model construction performed by the model construction unit 70 is also based on each statistical index table generated by the statistical index construction unit 60.
In a preferred embodiment, the statistical indicator constructing unit 60 is further configured to separately generate a statistical indicator table according to each statistical indicator.
Fig. 8 is a schematic structural diagram of a data table generating apparatus according to an embodiment of the present invention.
As shown in fig. 8, in this embodiment, the data report generating apparatus provided by the present invention includes the model building apparatus shown in fig. 7, and a report generating unit 90.
The report generation unit 90 is configured to generate a data report according to the data model constructed by the model construction unit 70.
In further embodiments, the data table generating device may include the model building device provided in any of the above embodiments.
Fig. 9 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.
As shown in fig. 9, as another aspect, the present application also provides an apparatus 900 including one or more Central Processing Units (CPUs) 901 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. In the RAM903, various programs and data necessary for the operation of the apparatus 900 are also stored. The CPU901, ROM902, and RAM903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
The following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output section 907 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.
In particular, according to an embodiment of the present disclosure, the model construction method or the data table generation method described in any of the above embodiments may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing a model building method or a data table generation method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 909, and/or installed from the removable medium 911.
As yet another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus of the above-described embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the model building method or the data table generating method described in the present application.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, for example, each of the described units may be a software program provided in a computer or a mobile intelligent device, or may be a separately configured hardware device. Wherein the designation of a unit or module does not in some way constitute a limitation of the unit or module itself.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the present application. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (18)
1. A method of model construction, the method comprising:
executing the construction tasks of the basic indexes in parallel to generate a plurality of basic index tables, wherein one basic index table is generated according to each basic index constructed by depending on the same source data table;
respectively carrying out time sequence analysis on the construction tasks of the summary indexes to determine the fastest construction path of each summary index;
respectively executing the construction tasks of all summary indexes according to all the fastest construction paths to generate a plurality of summary index tables;
constructing a data model according to each basic index table and each summary index table;
the construction task of the basic index is executed based on the source data table depended by the basic index, and the construction task of the summary index is executed based on the basic index table depended by the summary index and/or the summary index table.
2. The model building method according to claim 1, wherein the performing a time series analysis on the building tasks of the summary indexes respectively to determine a fastest building path of the summary indexes comprises performing the following time series analysis on the building tasks of the summary indexes respectively:
judging whether the first summary index has at least two construction paths: if not, determining that the only construction path of the first summary index is the fastest construction path;
if so, acquiring the earliest starting time, the latest ending time and the average spending time of each basic index and/or summary index on which a first summary index depends in each construction path, and the average spending time of the first summary index in each construction path; and the number of the first and second groups,
and calculating the latest end time of the first summary index in each constructed path to determine the fastest constructed path for constructing the first summary index.
3. The model building method according to claim 1, wherein the summary index includes a simple logic index and a complex logic index;
aggregating and executing the construction tasks of the simple logic indexes under the condition of mutual independence or the same dimensionality; the dimension is an angle for observing each basic index and each summary index;
and independently executing the construction task of each complex logic index.
4. The model building method according to claim 3, wherein a summary index table is generated according to each simple logic index built under the same dimension;
and generating a summary index table according to each complex logic index.
5. The model building method according to any one of claims 1 to 4, wherein before building the data model from each of the base index tables and each of the summary index tables, further comprising:
executing the construction task of each statistical index according to each basic index table and each summary index table to generate a plurality of statistical index tables;
the construction of the data model is further based on each of the statistical indicator tables.
6. The model building method according to claim 5, wherein a statistical index table is separately generated from each of the statistical indexes.
7. The model building method according to any one of claims 1-4, characterized in that the data model is a constellation model.
8. A data report generation method, characterized by comprising the model construction method according to any one of claims 1 to 7, and:
and generating a data report according to the data model.
9. A model building apparatus, characterized in that the apparatus comprises:
the basic index building unit is configured for executing the building tasks of the basic indexes in parallel and generating a plurality of basic index tables, wherein the basic index building unit is further configured for generating one basic index table according to the basic indexes built by depending on the same source data table;
the time sequence analysis unit is configured to perform time sequence analysis on the construction tasks of the summary indexes respectively so as to determine the fastest construction path of each summary index;
the summary index construction unit is configured to execute construction tasks of summary indexes according to the fastest construction paths respectively and generate a plurality of summary index tables;
the model building unit is configured to build a data model according to each basic index table and each summary index table;
the construction task of the basic index is executed based on the source data table depended by the basic index, and the construction task of the summary index is executed based on the basic index table depended by the summary index and/or the summary index table.
10. The model construction apparatus according to claim 9, characterized in that the timing analysis unit includes:
a path detection subunit, configured to determine whether the first summary indicator has at least two constructed paths: if not, determining that the only construction path of the first summary index is the fastest construction path; if so, determining the fastest constructing path of the first summary index through a data acquisition subunit and a time sequence analysis subunit;
a data obtaining subunit, configured to obtain an earliest starting time, a latest ending time and an average spending time of each base index and/or summary index on which a first summary index depends in each building path, and an average spending time of the first summary index in each building path;
and the time sequence analysis subunit is configured to calculate the latest end time of the first summary index in each building path so as to determine the fastest building path for building the first summary index.
11. The model building apparatus according to claim 9, wherein the summary index includes a simple logic index and a complex logic index;
the summary index construction unit is further configured to aggregate and execute construction tasks of the simple logic indexes in independent or same dimensionality; and independently executing the construction task of each complex logic index;
and the dimension is an angle for observing each basic index and each summary index.
12. The model building apparatus according to claim 11, wherein the summary index building unit is further configured to generate a summary index table according to each simple logical index built under the same dimension; and generating a summary index table according to each complex logic index.
13. The model building apparatus according to any one of claims 9 to 12, further comprising:
the statistical index construction unit is configured to execute construction tasks of the statistical indexes according to the basic index tables and the summary index tables to generate a plurality of statistical index tables;
the model building unit is further configured to build a data model according to each of the basic index tables, each of the summary index tables, and each of the statistical index tables.
14. The model building apparatus according to claim 13, wherein the statistical index building unit is further configured to generate a statistical index table separately according to each of the statistical indexes.
15. The model building apparatus according to any one of claims 9-12, wherein the data model is a constellation model.
16. A data report generating apparatus, characterized by comprising a model building apparatus according to any one of claims 9-15, and:
and the report generation unit is configured for generating a data report according to the data model.
17. An electronic device, characterized in that the device comprises:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method recited in any of claims 1-8.
18. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710312337.XA CN108805597B (en) | 2017-05-05 | 2017-05-05 | Model construction method and device and data report generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710312337.XA CN108805597B (en) | 2017-05-05 | 2017-05-05 | Model construction method and device and data report generation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108805597A CN108805597A (en) | 2018-11-13 |
| CN108805597B true CN108805597B (en) | 2021-11-23 |
Family
ID=64054703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710312337.XA Active CN108805597B (en) | 2017-05-05 | 2017-05-05 | Model construction method and device and data report generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108805597B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109933617B (en) * | 2019-03-08 | 2021-05-25 | 恒生电子股份有限公司 | Data processing method, data processing device, related equipment and related medium |
| CN111383104A (en) * | 2020-04-02 | 2020-07-07 | 中国工商银行股份有限公司 | Method and device for acquiring fund investment supervision indexes |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207940B (en) * | 2010-03-31 | 2014-11-05 | 国际商业机器公司 | Method and system for checking data |
| US9529875B2 (en) * | 2014-01-13 | 2016-12-27 | International Business Machines Corporation | Transforming timeseries and non-relational data to relational for complex and analytical query processing |
| CN104899199B (en) * | 2014-03-04 | 2018-12-28 | 阿里巴巴集团控股有限公司 | A kind of data warehouse data processing method and system |
| CN103853838B (en) * | 2014-03-17 | 2017-09-12 | 中国联合网络通信集团有限公司 | A kind of data processing method and device |
| CN105718565B (en) * | 2016-01-20 | 2019-07-02 | 北京京东尚科信息技术有限公司 | The construction method and construction device of data warehouse model |
-
2017
- 2017-05-05 CN CN201710312337.XA patent/CN108805597B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108805597A (en) | 2018-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110221975A (en) | Create the method and device of interface use-case automatic test script | |
| CN104252481A (en) | Dynamic check method and device for consistency of main and salve databases | |
| CN104919445B (en) | Transformation generation system | |
| CN109815267A (en) | The branch mailbox optimization method and system, storage medium and terminal of feature in data modeling | |
| CN111512283B (en) | Radix estimation in a database | |
| CN102446135A (en) | Software quality detection method | |
| CN106803799A (en) | A kind of performance test methods and device | |
| CN108805597B (en) | Model construction method and device and data report generation method and device | |
| CN111415027A (en) | Method and device for constructing component prediction model | |
| CN114168565B (en) | Backtracking test method, device and system of business rule model and decision engine | |
| CN113742069A (en) | Capacity prediction method and device based on artificial intelligence and storage medium | |
| RU97550U1 (en) | DEVICE FOR MODELING COMMUNICATIONS OF NETWORK ELEMENTS BASED ON THE HIERARCHICAL COMPLEX OF MULTI-LEVEL REPRESENTATION OF CORPORATE FUNCTIONALLY ORIENTED NETWORKS | |
| Dabkowski et al. | Network science enabled cost estimation in support of MBSE | |
| CN107273293B (en) | Big data system performance test method and device and electronic equipment | |
| CN115455091A (en) | Data generation method and device, electronic equipment and storage medium | |
| CN115080607A (en) | Method, device, equipment and storage medium for optimizing structured query statement | |
| CN109669668B (en) | Method and device for realizing simulated transaction execution in system performance test | |
| CN110083624B (en) | Stream data processing method, stream data processing apparatus, and computer medium | |
| CN113934894A (en) | Data display method based on index tree and terminal equipment | |
| CN110046975B (en) | Accounting method and device, computing equipment and computer storage medium | |
| US20170161359A1 (en) | Pattern-driven data generator | |
| CN113377801A (en) | Data inspection method, data inspection device, electronic equipment and computer storage medium | |
| CN114036174B (en) | Data updating method, device, equipment and storage medium | |
| CN106708838A (en) | Flow data query method and apparatus | |
| Alexandrov et al. | Myriad: parallel data generation on shared-nothing architectures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |