CN109165238B - Data processing method and device for generating period index data - Google Patents
Data processing method and device for generating period index data Download PDFInfo
- Publication number
- CN109165238B CN109165238B CN201810646630.4A CN201810646630A CN109165238B CN 109165238 B CN109165238 B CN 109165238B CN 201810646630 A CN201810646630 A CN 201810646630A CN 109165238 B CN109165238 B CN 109165238B
- Authority
- CN
- China
- Prior art keywords
- index
- dimension
- period
- data
- combination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method and a data processing device for generating periodic index data, wherein the data processing method comprises the following steps: receiving an application request of the periodic index data; determining the dimension of the applied periodic index data and the index dimension relation of the index from a preset index dimension relation table, wherein the index dimension relation is used for representing the combination of the dimension and the index; determining a statistical time period and a current date of the applied period index data; the applied period indexes are subjected to priority sorting according to the statistical time period to generate a period index table; associating the dimension attribute of the applied period index data with a period index table to obtain the combination of the dimension, the dimension attribute and the index of the applied period index data; and acquiring corresponding periodic index data from a data warehouse according to the combination of the applied dimension and the index and the statistical time period, and writing the periodic index data into a periodic index table. The invention enables the indexes to be generated in a mode of counting out the periodic table, and is beneficial to the statistics and analysis of the periodic indexes.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data processing method and device for generating periodic index data.
Background
At present, big data analysis is more and more favored, and especially artificial intelligence is rising, and big data is its important support. For some internet companies, especially internet companies providing platforms, statistical analysis of data indexes of the internet companies can provide intelligence basis for strategic decisions of the companies.
The index data stream has the characteristics of high acquisition speed, large information amount, unstable data and the like, and when the traditional index generation method is used for data analysis, different index quantities obtained by a database from the analysis process of user data are used for carrying out deduplication and accumulation calculation on thousands of user data. However, this approach does not solve the periodic statistics problem well.
Disclosure of Invention
The invention provides a data processing method and a data processing device for generating period index data, and aims to solve the technical problem that the prior art cannot solve period statistics.
In one aspect of the present invention, a data processing method for generating cycle index data is provided, including: receiving an application request of periodic index data, wherein the application request carries dimensions, dimensional attributes and indexes of the applied periodic index data; determining the dimension of the applied periodic index data and the index dimension relation of the index from a preset index dimension relation table, wherein the index dimension relation is used for representing the combination of the dimension and the index; determining a statistical time period and a current date of the applied period index data; the applied period indexes are subjected to priority ranking according to the statistical time period to generate a period index table; associating the dimension attribute of the applied period index data with the period index table to obtain the combination of the dimension, the dimension attribute and the index of the applied period index data; and acquiring corresponding period index data from a data warehouse according to the combination of the applied dimension and the index and the statistical time period, and writing the corresponding period index data into the period index table.
Optionally, obtaining corresponding period index data from the data warehouse according to the statistical time period according to the combination of the applied dimension and the index includes: determining SQL sentences corresponding to the combination of the applied dimensions and indexes; inputting the current date to the SQL statement, and automatically generating a data summary time period according to the statistical time period and the current date; and executing the determined SQL statement, and acquiring cycle index data in the time period from the data warehouse.
Optionally, the statistical time period includes: acquiring corresponding cycle index data from a data warehouse according to the applied dimension and index combination and the statistical time cycle by day, week and month, wherein the cycle index data comprises the following steps: judging whether the statistical time period is a week or a month or not for the same SQL statement; when the statistical time period is week or month, the period index data of the statistical time period of week or month is firstly obtained, and then the period index data of the statistical time period of day is obtained.
Optionally, each combination of the dimension and the index corresponds to a previously written SQL statement.
Optionally, determining the SQL statement corresponding to the combination of the applied dimension and the index includes: searching an SQL template corresponding to the applied index from an index SQL configuration table, wherein the index SQL configuration table is configured with the index and SQL templates with different corresponding dimensions; and determining the SQL sentence corresponding to the combination of the applied dimensionality and the index from the searched SQL template according to the applied dimensionality.
Optionally, determining the dimension of the applied period index data and the index dimension relationship of the index from a preset index dimension relationship table includes: acquiring a dimension name and a dimension ID of the applied period index data from a dimension attribute table; acquiring an index name and an index ID of the applied periodic index data from an index information table; and searching the obtained dimension name and dimension ID and the index name and index ID in the index dimension relation table to determine the combination of the dimension and the index of the applied period index data.
Optionally, the dimension attribute table is configured with a corresponding relationship of a dimension ID, a dimension code, a dimension name, a dimension attribute ID, an attribute value, and an attribute name.
Optionally, the index information table is configured with a corresponding relationship of an index classification ID, an index classification code, an index classification name, an index ID, an index code, and an index name.
Optionally, associating the dimension attribute of the applied period index data with the period index table to obtain a combination of the dimension, the dimension attribute and the index of the applied period index data includes: determining the name of the dimension attribute of the applied period index data and the corresponding relation between the dimension attribute and the dimension from a dimension attribute table; and associating the dimension attributes with the period index table according to the corresponding relation between the dimension attributes and the dimensions.
In another aspect of the present invention, there is provided a data processing apparatus for generating cycle index data, including: the system comprises a receiving unit and a processing unit, wherein the receiving unit is used for receiving an application request of the period index data, and the application request carries dimensions, dimension attributes and indexes of the applied period index data; the system comprises a first determining unit, a second determining unit and a third determining unit, wherein the first determining unit is used for determining the dimension of applied periodic index data and the index dimension relation of indexes from a preset index dimension relation table, and the index dimension relation is used for representing the combination of the dimension and the indexes; a second determination unit for determining a statistical time period and a current date of the applied period index data; the generating unit is used for carrying out priority sequencing on the applied period indexes according to the statistical time period to generate a period index table; the correlation unit is used for correlating the dimension attribute of the applied period index data with the period index table to obtain the combination of the dimension, the dimension attribute and the index of the applied period index data; and the acquisition unit is used for acquiring corresponding periodic index data from a data warehouse according to the statistical time period according to the combination of the applied dimension and the index and writing the periodic index data into the periodic index table.
According to the embodiment of the invention, the period index table is formed by carrying out priority sorting according to the statistical time period, and the period index data is generated by writing the period index table after calculation according to the applied dimension index from the data warehouse index dimension relation table, so that the index is generated in a period table output manner, and the statistics and analysis of the period index are facilitated.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a specific example of a data processing method for generating cycle index data according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a specific example of determining a dimension and an index dimension relationship of an index according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a specific example of obtaining cycle index data according to an embodiment of the present invention;
FIG. 4 is a flow chart of a specific example of data flow in an embodiment of the present invention;
fig. 5 is a schematic block diagram of a specific example of the data processing apparatus for generating the cycle index data in the embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The technical features mentioned in the different embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.
The embodiment provides a data processing method for generating cycle index data, which is applied to a computer device, and as shown in fig. 1, the method includes:
step S101, an application request of the period index data is received, wherein the application request carries the dimension, the dimension attribute and the index of the applied period index data.
The application request can be initiated by a user (company leader, product manager, etc.), and the application request can be initiated by multiple persons or one person. For example, a leader of an internet company needs to check a period index of an internet platform maintained by the company to know an operation condition of the current platform, and at this time, the index to be checked and a corresponding dimension can be selected, and the application request is initiated through a terminal.
Optionally, in the embodiment of the present invention, an approval process may be set after the application request, and rules such as an approver and an approval index may be configured in advance. The specific approval process comprises the following steps: the application request is approved. Whether a specific approval requesting party has the authority to check the corresponding index period or not is judged, and if yes, approval is passed; otherwise, the patient is rejected.
Step S102, determining the dimension of the applied period index data and the index dimension relation of the index from a preset index dimension relation table, wherein the index dimension relation is used for representing the combination of the dimension and the index.
Step S103, determining the statistical time period and the current date of the applied period index data.
And step S104, performing priority sequencing on the applied period indexes according to the statistical time period to generate a period index table.
The embodiment of the invention can perform priority ranking according to the length of the statistical time period, for example, the shorter the period is, the higher the priority is. And generating a period index table after sorting, wherein the period index table is a form not containing period index data, and the period index table contains indexes to be counted, corresponding dimensionalities, time periods to be counted and priority.
Step S105, the dimension attribute of the applied period index data is associated with the period index table, and the combination of the dimension, the dimension attribute and the index of the applied period index data is obtained.
Dimension attributes are subdivisions made of dimensions, e.g., "City" contains attributes of Beijing, Shanghai, Shenzhen, Guangzhou, etc.; the 'platform' comprises attributes of an applet, an APP, a PC end and the like.
And step S106, acquiring corresponding period index data from a data warehouse according to the statistical time period according to the combination of the applied dimension and the index, and writing the corresponding period index data into the period index table.
And calculating the period index data from the index dimension relation table of the data warehouse according to the applied dimension index and the statistical time period, and writing the period index data into the period index table to complete the statistics of the period index. And outputting the period index table to the applicant for viewing. When statistical calculation is performed, the period index data corresponding to the combination of the applied dimension and the index can be acquired from the data warehouse one by one according to the priority order.
According to the embodiment of the invention, the period index table is formed by carrying out priority sequencing according to the statistical time period, and the period index table is written into the period index table after calculation according to the applied dimension index from the data warehouse index dimension relation table to generate period index data and generate period indexes, thereby being beneficial to statistics and analysis of the period indexes.
As an optional implementation manner of the embodiment of the present invention, as shown in fig. 2, step S102 in the foregoing embodiment specifically includes:
step S1021, obtaining the dimension name and the dimension ID of the applied period index data from the dimension attribute table;
step S1022, acquiring an index name and an index ID of the applied period index data from the index information table;
step S1023, the obtained dimension name and dimension ID and the index name and index ID are searched in the index dimension relationship table to determine the combination of the dimension and the index of the applied period index data.
The dimension attribute table is configured with corresponding relations of a dimension ID, a dimension code, a dimension name, a dimension attribute ID, an attribute value and an attribute name; the index information table is configured with a correspondence relationship of an index classification ID, an index classification code, an index classification name, an index ID, an index code, and an index name.
By way of example with the index data of the Tuba rabbit dress and repair net, the index information table is shown in Table 1:
TABLE 1
The dimension attribute table is shown in table 2:
TABLE 2
The index classification code, index code and dimension code described in table 1 and table 2 above are mainly used in SQL statements to represent corresponding classifications, indexes and dimensions.
An example of the combination of indices and dimensions determined from the two tables is shown in table 3:
| index name | Dimension name | Index ID | Dimension ID |
| GMV | Summary of the invention | 1001 | 0 |
| Income of assembly enterprise | Summary | 1002 | 0 |
| UV | Platform | 2003 | 2 |
| UV | Applet | 2003 | 4 |
| Number of active users | Name of APP | 2006 | 5 |
| Number of new clues | Summary of the invention | 3003 | 0 |
| Number of times a thread was initiated | Summary of the invention | 3002 | 0 |
| Number of sellable | Summary of the invention | 4003 | 0 |
| Rate of sale of newly added clues | Summary of the invention | 4005 | 0 |
| Assigning information numbers | Summary of the invention | 5001 | 0 |
| Number of times of wave rate | Summary of the invention | 5003 | 0 |
| Information number of deduction | Summary of the invention | 5005 | 0 |
| Contract subscription number | Summary of the invention | 5008 | 0 |
| Assembling GMV | Summary of the invention | 5009 | 0 |
| Supply chain GMV | Summary of the invention | 6001 | 0 |
| Supplementary material order number | Summary of the invention | 6002 | 0 |
| Amount of subsidiary order | Summary of the invention | 6003 | 0 |
| Staple number of main material | Summary of the invention | 6006 | 0 |
| Amount of master order | Summary of the invention | 6007 | 0 |
| Newly added complaint number | Summary of the invention | 7001 | 0 |
| Number of complaints in existence | Summary of the invention | 7002 | 0 |
| Number of complaints turned off | Summary of the invention | 7003 | 0 |
TABLE 3
As another optional implementation, the step S104 may include: determining the name of the dimension attribute of the applied period index data and the corresponding relation between the dimension attribute and the dimension from a dimension attribute table; and associating the dimension attributes with the period index table according to the corresponding relation between the dimension attributes and the dimensions.
In the embodiment of the invention, by establishing the incidence relation between the dimension attributes and the period index table, when period index data statistics is carried out, the index data of each dimension attribute under the corresponding dimension can be counted, so that more accurate index data of the dimension can be obtained.
Optionally, in the embodiment of the present invention, an index SQL configuration table may be configured in advance, where indexes and corresponding SQL templates with different dimensions are configured, a plurality of indexes may be written by one SQL, and one SQL can only belong to one index class, so that when performing period index statistics, a corresponding SQL statement is directly obtained, and corresponding period index data can be obtained by executing the statement. Specifically, as shown in fig. 3, the step S106 may include:
step S1061, determining an SQL statement corresponding to the combination of the applied dimension and the index;
step S1062, inputting the current date to the SQL statement, and automatically generating a data summary time period according to the statistical time period and the current date;
step S1063, executing the determined SQL statement, and acquiring cycle index data in the time period from the data warehouse.
The SQL sentences corresponding to the flow index combination of the small program dimension are as follows:
select'MPRO'as plat,sum(case when event＝'pv'then 1 else 0 end)as pv,count(distinct cookie)as uv from ods.ods_user_event_tracking where dt between${date_begin}and${date_curr}and from_unixtime(int(substr(cur_time,1,10)),'yyyyMMdd')between${date_begin}and${date_curr}and(app＝'houseDesign'or instr(app,'mpro')<>0)
specifically, when counting the period index data, at least two parameters need to be introduced, 1 is a counting time period, 2 is a current date, and 3 is a configuration SQL id (optional, used for selecting a corresponding SQL statement).
The script functions as follows: summary cycle indexes related to a flow field are generated in a data billboard mart (the indexes of the flow field comprise non-accumulative indexes such as au and uv, and a similar script needs to be designed as long as the non-accumulative indexes are added later).
use, panel _ flow _ ind [ 1: day, 2: week, 3: month ] < yyymmdd >
1: and counting the time period, and controlling whether to generate data of a daily schedule, a week schedule or a month schedule.
2: current date, input 20180419, match period 2: in week, the summarized data before the date of the week of 20180412 and 20180419 is generated, and the period 3 is matched: the year, the summarized data before the date of 20180319 and 20180419 are generated.
Further, the counting time period according to the embodiment of the present invention includes: acquiring corresponding period index data from the data warehouse according to the statistical time period according to the combination of the applied dimension and the index by day, week and month, wherein the acquiring comprises the following steps:
judging whether the statistical time period is a week or a month or not for the same SQL statement;
when the statistical time period is week or month, the period index data of the statistical time period of week or month is firstly obtained, and then the period index data of the statistical time period of day is obtained.
Specifically, if the script cycle is 1, only records whose cycle is 1 and valid are traversed.
If the script cycle is 2, then the same conf _ id takes the record with cycle 2 (week summary) as a priority, and takes the record with cycle 1 (without week summary record, week summary is always obtained by adding date summary record to data interval).
If the script cycle is 3, then the same conf _ id takes the record with cycle 3 (monthly summary) as a priority, and takes the record with cycle 1 as a priority.
Further preferably, the determining the SQL statement corresponding to the applied combination of the dimension and the index includes: searching an SQL template corresponding to the applied index from an index SQL configuration table; and determining the SQL sentence corresponding to the combination of the applied dimension and the index from the searched SQL template according to the applied dimension.
The data flow of the embodiment of the present invention is shown in fig. 4, wherein the hive layer to dim layer are the data output path of the current index. The warehouse layer to dm layer represents the number output according to a specific rule, such as group by sum and count (distinct). The flow direction in the warehouse layer represents that the warehouse data can be generated by summarizing the data to the mart aperture in the background.
An embodiment of the present invention further provides a data processing apparatus for generating cycle index data, where the apparatus may be configured to execute the data processing method provided in the embodiment of the present invention, and as shown in fig. 5, the apparatus includes: the device comprises a receiving unit 10, a first determining unit 20, a second determining unit 30, a generating unit 40, an associating unit 50 and an obtaining unit 60.
The receiving unit 10 is configured to receive an application request for periodic index data, where the application request carries dimensions, dimensional attributes, and indexes of the applied periodic index data;
the first determining unit 20 is configured to determine, from a preset index dimension relation table, a dimension of the applied period index data and an index dimension relation of the index, where the index dimension relation is used to represent a combination of the dimension and the index;
the second determining unit 30 is used for determining the statistical time period and the current date of the applied period index data;
the generating unit 40 is configured to perform priority ordering on the applied cycle indexes according to the statistical time period, and generate a cycle index table;
the correlation unit 50 is configured to correlate the dimension attribute of the applied period index data with the period index table to obtain a combination of the dimension, the dimension attribute, and the index of the applied period index data;
the obtaining unit 60 is configured to obtain corresponding period index data from the data warehouse according to the statistical time period according to the combination of the applied dimension and the index, and write the corresponding period index data into the period index table. For specific functions of the receiving unit 10, the first determining unit 20, the second determining unit 30, the generating unit 40, the associating unit 50, and the obtaining unit 60 in the embodiment of the present invention, reference may be made to steps S101 to S106 of the data processing method in the above embodiment of the present invention, and details are not repeated here.
According to the embodiment of the invention, the period index table is formed by carrying out priority sequencing according to the statistical time period, and the period index table is written into the period index table after calculation according to the applied dimension index from the data warehouse index dimension relation table to generate period index data and generate period indexes, thereby being beneficial to statistics and analysis of the period indexes.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should it be exhaustive of all embodiments. And obvious variations or modifications from the teachings herein remain within the scope of this application.
Claims (6)
1. A data processing method for generating cycle index data, comprising:
receiving an application request of periodic index data, wherein the application request carries dimensions, dimensional attributes and indexes of the applied periodic index data;
determining the dimension of the applied period index data and the index dimension relation of the index from a preset index dimension relation table, wherein the index dimension relation is used for representing the combination of the dimension and the index;
determining a statistical time period and a current date of the applied period index data;
the applied period indexes are subjected to priority ranking according to the statistical time period to generate a period index table;
associating the dimension attribute of the applied period index data with the period index table to obtain the combination of the dimension, the dimension attribute and the index of the applied period index data;
acquiring corresponding period index data from a data warehouse according to the statistical time period according to the combination of the applied dimension and the index, and writing the corresponding period index data into the period index table;
acquiring corresponding period index data from the data warehouse according to the statistical time period according to the combination of the applied dimension and the index comprises the following steps:
determining SQL sentences corresponding to the combination of the applied dimensions and indexes;
inputting the current date to the SQL statement, and automatically generating a data summary time period according to the statistical time period and the current date;
executing the determined SQL statement, and acquiring cycle index data in the time period from the data warehouse;
each combination of the dimension and the index corresponds to a pre-written SQL statement;
determining the SQL statement corresponding to the combination of the applied dimension and the index comprises:
searching an SQL template corresponding to the applied index from an index SQL configuration table, wherein the index SQL configuration table is configured with the index and SQL templates with different corresponding dimensions;
determining SQL sentences corresponding to the combination of the applied dimensions and the indexes from the searched SQL templates according to the applied dimensions;
associating the dimension attribute of the applied period index data with the period index table to obtain the combination of the dimension, the dimension attribute and the index of the applied period index data comprises:
determining the name of the dimension attribute of the applied period index data and the corresponding relation between the dimension attribute and the dimension from a dimension attribute table;
and associating the dimension attributes with the period index table according to the corresponding relation between the dimension attributes and the dimensions.
2. The data processing method of claim 1, wherein the statistical time period comprises: acquiring corresponding period index data from the data warehouse according to the statistical time period according to the combination of the applied dimension and the index by day, week and month, wherein the acquiring comprises the following steps:
judging whether the statistical time period is a week or a month or not for the same SQL statement;
when the statistical time period is week or month, the period index data of the statistical time period of week or month is firstly obtained, and then the period index data of the statistical time period of day is obtained.
3. The data processing method of claim 1, wherein determining the dimension of the applied period index data and the index dimension relationship of the index from a preset index dimension relationship table comprises:
acquiring a dimension name and a dimension ID of the applied period index data from a dimension attribute table;
acquiring an index name and an index ID of the applied periodic index data from an index information table;
and searching the obtained dimension name and dimension ID and the index name and index ID in the index dimension relation table to determine the combination of the dimension and the index of the applied period index data.
4. The data processing method according to claim 3, wherein the dimension attribute table is configured with a correspondence of a dimension ID, a dimension code, a dimension name, a dimension attribute ID, an attribute value, and an attribute name.
5. The data processing method according to claim 3, wherein the index information table is configured with a correspondence relationship of an index classification ID, an index classification code, an index classification name, an index ID, an index code, and an index name.
6. A data processing apparatus for generating cycle index data, comprising:
the system comprises a receiving unit and a processing unit, wherein the receiving unit is used for receiving an application request of the period index data, and the application request carries dimensions, dimension attributes and indexes of the applied period index data;
the system comprises a first determining unit, a second determining unit and a third determining unit, wherein the first determining unit is used for determining the dimension of applied periodic index data and the index dimension relation of indexes from a preset index dimension relation table, and the index dimension relation is used for representing the combination of the dimension and the indexes;
a second determination unit for determining a statistical time period and a current date of the applied period index data;
the generating unit is used for carrying out priority sequencing on the applied period indexes according to the statistical time period to generate a period index table;
the correlation unit is used for correlating the dimension attribute of the applied period index data with the period index table to obtain the combination of the dimension, the dimension attribute and the index of the applied period index data;
the acquisition unit is used for acquiring corresponding periodic index data from a data warehouse according to the applied dimension and index combination and the statistical time period and writing the periodic index data into the periodic index table;
the obtaining unit is specifically configured to, when obtaining corresponding period index data from the data warehouse according to the statistical time period based on the combination of the applied dimension and the index:
determining SQL sentences corresponding to the combination of the applied dimensions and indexes;
inputting the current date to the SQL statement, and automatically generating a data summary time period according to the statistical time period and the current date;
executing the determined SQL statements, and acquiring cycle index data in the time period from the data warehouse;
each combination of the dimension and the index corresponds to a pre-written SQL statement;
when the acquiring unit determines the SQL statement corresponding to the combination of the applied dimension and index, the acquiring unit is specifically configured to:
searching an SQL template corresponding to the applied index from an index SQL configuration table, wherein the index SQL configuration table is configured with the index and SQL templates with different corresponding dimensions;
determining an SQL sentence corresponding to the combination of the applied dimensionality and the index from the searched SQL template according to the applied dimensionality;
the association unit is configured to associate the dimension attribute of the applied period index data with the period index table, and when a combination of the dimension, the dimension attribute, and the index of the applied period index data is obtained, the association unit is specifically configured to:
determining the name of the dimension attribute of the applied period index data and the corresponding relation between the dimension attribute and the dimension from a dimension attribute table;
and associating the dimension attributes with the period index table according to the corresponding relation between the dimension attributes and the dimensions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810646630.4A CN109165238B (en) | 2018-06-21 | 2018-06-21 | Data processing method and device for generating period index data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810646630.4A CN109165238B (en) | 2018-06-21 | 2018-06-21 | Data processing method and device for generating period index data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109165238A CN109165238A (en) | 2019-01-08 |
| CN109165238B true CN109165238B (en) | 2022-09-09 |
Family
ID=64897205
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810646630.4A Active CN109165238B (en) | 2018-06-21 | 2018-06-21 | Data processing method and device for generating period index data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109165238B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851676B (en) * | 2019-10-08 | 2022-08-09 | 支付宝(杭州)信息技术有限公司 | Index data processing method and device and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809254A (en) * | 2015-05-19 | 2015-07-29 | 郑州悉知信息技术有限公司 | Data query method and device |
| CN105243167A (en) * | 2015-11-10 | 2016-01-13 | 中国建设银行股份有限公司 | Data processing method and device |
| CN105843945A (en) * | 2016-04-08 | 2016-08-10 | 联动优势科技有限公司 | Report generation method and system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424339B (en) * | 2013-09-11 | 2019-02-22 | 腾讯科技（深圳）有限公司 | The method, apparatus and system of data analysis |
| CN104809130B (en) * | 2014-01-27 | 2019-03-08 | 腾讯科技（深圳）有限公司 | Method, equipment and the system of data query |
| US10108666B2 (en) * | 2015-03-10 | 2018-10-23 | Oracle International Corporation | Adaptive handling of skew for distributed joins in a cluster |
| US10067954B2 (en) * | 2015-07-22 | 2018-09-04 | Oracle International Corporation | Use of dynamic dictionary encoding with an associated hash table to support many-to-many joins and aggregations |
| CN106557498A (en) * | 2015-09-25 | 2017-04-05 | 北京国双科技有限公司 | Date storage method and device and data query method and apparatus |
| CN106776834A (en) * | 2016-11-28 | 2017-05-31 | 中通服公众信息产业股份有限公司 | A kind of data analysis based on index is from access method and system |
| CN107729399B (en) * | 2017-09-21 | 2020-06-05 | 北京京东尚科信息技术有限公司 | Data processing method and device |
-
2018
- 2018-06-21 CN CN201810646630.4A patent/CN109165238B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809254A (en) * | 2015-05-19 | 2015-07-29 | 郑州悉知信息技术有限公司 | Data query method and device |
| CN105243167A (en) * | 2015-11-10 | 2016-01-13 | 中国建设银行股份有限公司 | Data processing method and device |
| CN105843945A (en) * | 2016-04-08 | 2016-08-10 | 联动优势科技有限公司 | Report generation method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109165238A (en) | 2019-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10452625B2 (en) | Data lineage analysis | |
| US10747762B2 (en) | Automatic generation of sub-queries | |
| CN109033173B (en) | Data processing method and device for generating multidimensional index data | |
| CN105718490A (en) | Method and device for updating classifying model | |
| US20130024167A1 (en) | Computer-Implemented Systems And Methods For Large Scale Automatic Forecast Combinations | |
| US20160285672A1 (en) | Method and system for processing network media information | |
| US20090024407A1 (en) | Indicating which of forecasting models at different aggregation levels has a better forecast quality | |
| CN111126495B (en) | Model training method, information prediction device, storage medium and equipment | |
| CN111177200A (en) | Data processing system and method | |
| CN109669995A (en) | Data storage, quality calculation method, device, storage medium and server | |
| CN110163683B (en) | Value user key index determination method, advertisement delivery method and device | |
| CN109086339B (en) | Data processing method and device for generating index recombination rate | |
| CN110727857A (en) | Method and device for identifying key features of potential users aiming at business objects | |
| CN112926858A (en) | Power marketing service operation index design method and device | |
| CN111833018A (en) | Patent analysis method and system for science and technology project | |
| CN112036631B (en) | Purchasing quantity determining method, purchasing quantity determining device, purchasing quantity determining equipment and storage medium | |
| CN105989066A (en) | Information processing method and device | |
| CN116611888A (en) | Dynamic bidding method, storage medium and electronic device for bidding advertisement | |
| CN110737673B (en) | Data processing method and system | |
| CN109165238B (en) | Data processing method and device for generating period index data | |
| CN110737432A (en) | script aided design method and device based on root list | |
| US7197471B2 (en) | System and method for assessing demographic data accuracy | |
| CN111723004A (en) | Measuring method for agile software development, measuring data output method and device | |
| WO2023098034A1 (en) | Business data report classification method and apparatus | |
| CN116228402A (en) | Financial credit investigation feature warehouse technical support system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000 R & D room 3501, block a, building 7, Vanke Cloud City Phase I, Xingke 1st Street, Xili community, Xili street, Nanshan District, Shenzhen City, Guangdong Province Applicant after: Tubatu Group Co.,Ltd. Address before: 1001-a, 10th floor, bike technology building, No.9, Keke Road, high tech Zone, Nanshan District, Shenzhen, Guangdong 518000 Applicant before: SHENZHEN BINCENT TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |