CN110955654B - Multi-dimensional index calculation method and device - Google Patents
Multi-dimensional index calculation method and device Download PDFInfo
- Publication number
- CN110955654B CN110955654B CN201811126783.2A CN201811126783A CN110955654B CN 110955654 B CN110955654 B CN 110955654B CN 201811126783 A CN201811126783 A CN 201811126783A CN 110955654 B CN110955654 B CN 110955654B
- Authority
- CN
- China
- Prior art keywords
- data
- memory
- dimension
- analyzed
- calculated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 37
- 230000002776 aggregation Effects 0.000 claims abstract description 30
- 238000004220 aggregation Methods 0.000 claims abstract description 30
- 238000013479 data entry Methods 0.000 claims description 26
- 238000003860 storage Methods 0.000 claims description 16
- 238000001914 filtration Methods 0.000 claims description 9
- 238000012935 Averaging Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000004590 computer program Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Abstract
The application discloses a multi-dimensional index calculation method and device. The method comprises the following steps: determining a plurality of dimensions to be parsed; writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data; writing the processed data into a database, and analyzing the processed data in the database by adopting a plurality of dimensions to be analyzed to obtain a plurality of target indexes. The method and the device solve the problem of low data processing efficiency when the index is calculated in real time in a multi-dimension mode in the related technology.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a multi-dimensional index calculation method and device.
Background
In the time of increasingly abundant real-time service demands, the real-time computing engine is used for carrying out real-time display on any concerned data index, so that the user behavior can be rapidly analyzed, various indexes of data can be displayed in real time, and rapid display on all indexes flexibly is particularly important for a real-time engine, and the real-time engine cannot calculate a single index for rapidness, so that the real-time computing engine should consider rapid calculation on multiple indexes. In the existing calculation scheme, a single calculation index can be stored in a memory, data is acquired from a rear end when the front end displays, the rear end takes out the data from the memory, and then the data is returned to the front end. However, the index calculation is few, only a single index calculation can be performed, and multi-dimensional calculation is difficult. For the multi-dimensional calculation, the full data are stored in the database, when the front end is displayed, the value is taken from the database, and the index is calculated in the sql mode, however, the scheme causes too much pressure on the database, the processed data are large, and the data query is slow.
Aiming at the problem of low data processing efficiency when multi-dimensional real-time index calculation is carried out in the related technology, no effective solution is proposed at present.
Disclosure of Invention
The application mainly aims to provide a multi-dimensional index calculation method and device, which are used for solving the problem of low data processing efficiency when multi-dimensional real-time index calculation is carried out in the related technology.
In order to achieve the above object, according to one aspect of the present application, there is provided a multi-dimensional index calculation method. The method comprises the following steps: determining a plurality of dimensions to be parsed; writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data; writing the processed data into a database, and analyzing the processed data in the database by adopting the plurality of dimensions to be analyzed to obtain a plurality of target indexes.
Further, writing data items of the data to be calculated into the memory, and performing aggregation processing on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed, wherein the data after processing comprises: determining data associated with each data entry in the memory in each dimension to be parsed; and carrying out aggregation processing on the data associated with each dimension to be parsed corresponding to the same data item.
Further, writing the data entry of the data to be calculated into the memory includes: the data to be calculated is stored into a memory one by taking a data item as a unit; and according to the data associated with the data to be calculated in each dimension to be analyzed, the aggregation processing of the data items written into the memory comprises the following steps: determining the data associated with each dimension to be parsed of the currently stored data item every time one data item is stored, and searching the data items which are associated with the same data in each dimension to be parsed from the data items which are stored in the memory before; combining the currently stored data items with the searched data items, averaging index data corresponding to the combined data items, and determining an average result as index data of the aggregated data items.
Further, after determining the data associated with each of the plurality of dimensions to be parsed for each of the respective data entries in the memory, the method further comprises: and filtering the data information of other dimensions except the dimensions to be parsed in each data entry to obtain filtered data.
Further, writing the processed data into a database, parsing the processed data in the database by adopting the plurality of dimensions to be parsed to obtain a plurality of target indexes, and then the method further comprises: and displaying the calculated target indexes.
Further, the database is a relational database.
In order to achieve the above object, according to one aspect of the present application, there is provided a multi-dimensional index calculation device including: a determining unit configured to determine a plurality of dimensions to be parsed; the processing unit is used for writing data items of the data to be calculated into the memory, and carrying out aggregation processing on the data items written into the memory according to the data associated with each dimension to be analyzed of the data to be calculated to obtain processed data; and the analysis unit is used for writing the processed data into a database, and analyzing the processed data in the database by adopting the plurality of dimensions to be analyzed to obtain a plurality of target indexes.
The processing unit further includes: the determining module is used for determining the data associated with each data item in the memory in each dimension to be analyzed; and the processing module is used for carrying out aggregation processing on the data items corresponding to the data associated with each dimension to be analyzed.
In order to achieve the above object, according to one aspect of the present application, there is provided a storage medium including a stored program, wherein the program performs the multi-dimensional index calculation method of any one of the above.
In order to achieve the above object, according to one aspect of the present application, there is provided a processor for executing a program, wherein the program executes the multi-dimensional index calculation method according to any one of the above.
According to the application, the following steps are adopted: determining a plurality of dimensions to be parsed; writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data; the processed data is written into a database, a plurality of dimensions to be analyzed are adopted in the database for analyzing the processed data, a plurality of target indexes are obtained, and the problem of low data processing efficiency when the indexes are calculated in real time in a multi-dimensional manner in the related technology is solved. The data associated with the plurality of dimensions to be analyzed in the memory are preprocessed, the processed data are written into the database, multidimensional analysis is carried out on the data in the database, a plurality of target indexes are obtained, the performance cost of data processing in the database is reduced, and the efficiency of acquiring the plurality of target indexes in real time is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a flow chart of a multi-dimensional index calculation method provided according to an embodiment of the present application; and
fig. 2 is a schematic diagram of a multi-dimensional index calculating device according to an embodiment of the present application.
Detailed Description
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the application herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an embodiment of the application, a multi-dimensional index calculation method is provided.
Fig. 1 is a flowchart of a multi-dimensional index calculation method according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S101, determining a plurality of dimensions to be parsed.
For example, the determined plurality of dimension to be parsed is profile_id, country, provice, city, sex.
Step S102, writing the data items of the data to be calculated into the memory, and performing aggregation processing on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain processed data.
It should be noted that, in the embodiment of the present application, the data in the memory is stored in real time.
Optionally, in the multi-dimensional index calculation method provided by the embodiment of the present application, data entries of data to be calculated are written into a memory, and according to data associated with each dimension to be analyzed by the data to be calculated, aggregation processing is performed on the data entries written into the memory, so as to obtain processed data, where the data includes: determining data associated with each data entry in the memory in each dimension to be parsed; and carrying out aggregation processing on the data items corresponding to the same data associated with each dimension to be parsed. Assuming that the dimensions to be parsed include dimension 1 and dimension 2, for two data entries, if the data associated with both data entries in dimension 1 are data a and the data associated with both data entries in dimension 2 are data B, then the data associated with both data entries in each dimension to be parsed is considered to correspond the same and the two data entries subsequently need to be aggregated.
Writing the data item of the data to be calculated into the memory comprises: taking the data item as a unit, storing the data to be calculated into a memory one by one; and according to the data associated with the data to be calculated in each dimension to be analyzed, the aggregation processing of the data items written into the memory comprises the following steps: determining the data associated with each dimension to be parsed of the currently stored data item every time one data item is stored, and searching the data items which are associated with the same data in each dimension to be parsed from the data items which are stored in the memory before; combining the currently stored data items with the searched data items, averaging index data corresponding to the combined data items, and determining an average result as index data of the aggregated data items.
For example, the data associated with each of the plurality of dimensions to be parsed stored in the memory for the predetermined period of time is shown in table 1 below:
TABLE 1
If the index data is the average income of all people, it is assumed that the data in the memory are stored one by one in the above order, and the index data corresponding to the combined data entries are averaged, so the aggregate processing in the memory is calculated as follows.
After the first piece of data is stored, the values in the memory are as follows:
| Profile_id | Country | Province | City | Sex | Avgsalary |
| 1001 | China | beijing | Beijing | Man's body | 10000 |
After the second piece of data is stored, the values in the memory are as follows:
| Profile_id | Country | Province | City | Sex | Avgsalary |
| 1001 | China | beijing | Beijing | Man's body | 10000 |
| 1001 | China | Sichuan (Sichuan) | Chengdu | Man's body | 8000 |
After the third piece of data is stored, because the first dimensions of the third piece of data and the first piece of data are the same, aggregation can be directly performed, and the memory values are as follows:
| Profile_id | Country | Province | City | Sex | Avgsalary |
| 1001 | China | beijing | Beijing | Man's body | 10000 |
| 1001 | China | Sichuan (Sichuan) | Chengdu | Man's body | 8000 |
After the fourth piece of data is stored:
after the fifth piece of data is stored:
| Profile_id | Country | Province | City | Sex | Avgsalary |
| 1001 | China | beijing | Beijing | Man's body | 10000 |
| 1001 | China | Sichuan (Sichuan) | Chengdu | Man's body | 8000 |
| 1002 | China | Beijing | Beijing | Man's body | 10000 |
| 1002 | USA | New York | New York | Man's body | 20000 |
After the sixth piece of data is stored:
| Profile_id | Country | Province | City | Sex | Avgsalary |
| 1001 | China | beijing | Beijing | Man's body | 10000 |
| 1001 | China | Sichuan (Sichuan) | Chengdu | Man's body | 8000 |
| 1002 | China | Beijing | Beijing | Man's body | 10000 |
| 1002 | USA | New York | New York | Man's body | 20000 |
| 1002 | China | Sichuan (Sichuan) | Chengdu | Female | 8000 |
After the seventh piece of data is stored:
after the eighth piece of data is stored:
| Profile_id | Country | Province | City | Sex | Avgsalary |
| 1001 | China | beijing | Beijing | Man's body | 10000 |
| 1001 | China | Sichuan (Sichuan) | Chengdu | Man's body | 8000 |
| 1002 | China | Beijing | Beijing | Man's body | 10000 |
| 1002 | USA | New York | New York | Man's body | 20000 |
| 1002 | China | Sichuan (Sichuan) | Chengdu | Female | 8000 |
| 1003 | China | Beijing | Beijing | Man's body | 11000 |
| 1003 | China | Beijing | Beijing | Female | 12000 |
After the ninth piece of data is stored, the processed data is obtained:
optionally, in the multi-dimensional index calculation method provided by the embodiment of the present application, after determining data associated with each of a plurality of dimensions to be parsed by each data entry in the memory, the method further includes: and filtering the data information of other dimensions except the dimensions to be parsed in each data entry to obtain filtered data.
For example, each piece of data includes a plurality of data information which is irrelevant to a plurality of dimensions to be parsed, in order to improve the calculation efficiency and reduce unnecessary data interference, the data information of other dimensions except the dimensions to be parsed in each piece of data is filtered, and the filtered data is obtained.
Step S103, writing the processed data into a database, and analyzing the processed data in the database by adopting a plurality of dimensions to be analyzed to obtain a plurality of target indexes.
The database may be a relational database.
For example, the average result is determined to be index data of the aggregated data item, the index data is written into a database, and the processed data is parsed in the database by adopting a plurality of dimensions to be parsed, so that a plurality of target indexes, such as the following index 1, index 2 and index 3, can be obtained.
Index 1: obtaining average wages for all users
Select avg(Avgsalary)from salary
Index 2: obtaining average wages in China
Select avg (Avgsalary) from salary where country = 'china'
Index 3: obtaining average payroll for all users with profile_id 1001
Select avg(Avgsalary)from salary where profile_id＝1001
By the scheme, the data volume in the relational database is greatly reduced because the preprocessing is carried out in the memory according to the dimension, and in this way, the real-time indexes of various dimensions can be calculated according to the requirement, the performance cost of data processing in the database is reduced, and the efficiency of acquiring a plurality of target indexes in real time is improved.
In order to facilitate the user to acquire the target index in real time, in the multi-dimensional index calculation method provided by the embodiment of the application, the processed data is written into the database, the processed data is parsed by adopting a plurality of dimensions to be parsed in the database, and after a plurality of target indexes are acquired, the method further comprises: and displaying the calculated multiple target indexes.
In summary, according to the multi-dimensional index calculation method provided by the embodiment of the application, a plurality of dimensions to be analyzed are determined; writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data; the processed data is written into a database, a plurality of dimensions to be analyzed are adopted in the database for analyzing the processed data, a plurality of target indexes are obtained, and the problem of low data processing efficiency when the indexes are calculated in real time in a multi-dimensional manner in the related technology is solved. The data associated with the plurality of dimensions to be analyzed in the memory are preprocessed, the processed data are written into the database, multidimensional analysis is carried out on the data in the database, a plurality of target indexes are obtained, the performance cost of data processing in the database is reduced, and the efficiency of acquiring the plurality of target indexes in real time is improved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
The embodiment of the application also provides a multi-dimensional index calculation device, and it is to be noted that the multi-dimensional index calculation device of the embodiment of the application can be used for executing the multi-dimensional index calculation method provided by the embodiment of the application. The following describes a multi-dimensional index calculation device provided by the embodiment of the application.
Fig. 2 is a schematic diagram of a multi-dimensional index calculation device according to an embodiment of the application. As shown in fig. 2, the apparatus includes: a determination unit 10, a processing unit 20 and a profiling unit 30.
Specifically, the determining unit 10 is configured to determine a plurality of dimensions to be parsed;
the processing unit 20 is configured to write data entries of the data to be calculated into the memory, and aggregate the data entries written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed, so as to obtain processed data;
the parsing unit 30 is configured to write the processed data into a database, parse the processed data in the database by using a plurality of dimensions to be parsed, and obtain a plurality of target indexes.
According to the multi-dimensional index calculation device provided by the embodiment of the application, a plurality of dimensions to be analyzed are determined through the determining unit 10; the processing unit 20 writes the data items of the data to be calculated into the memory, and performs aggregation processing on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed, so as to obtain processed data; the parsing unit 30 writes the processed data into a database, parses the processed data in the database by adopting a plurality of dimensions to be parsed, and obtains a plurality of target indexes, thereby solving the problem of lower data processing efficiency when the indexes are calculated in real time in a multi-dimension manner in the related art. The data associated with the plurality of dimensions to be analyzed in the memory are preprocessed, the processed data are written into the database, multidimensional analysis is carried out on the data in the database, a plurality of target indexes are obtained, the performance cost of data processing in the database is reduced, and the efficiency of acquiring the plurality of target indexes in real time is improved.
Optionally, in the multi-dimensional index calculating device provided by the embodiment of the present application, the processing unit 20 further includes: the determining module is used for determining the data associated with each data item in the memory in each dimension to be analyzed; and the processing module is used for carrying out aggregation processing on the data items corresponding to the same data associated with each dimension to be analyzed.
Optionally, in the multi-dimensional index calculating device provided by the embodiment of the present application, the processing unit 20 further includes: the storage module is used for storing the data to be calculated into the memory one by taking the data item as a unit; the searching module is used for determining the data associated with each dimension to be analyzed of the currently stored data item and searching the data items which are associated with the same data in each dimension to be analyzed from the data items which are stored in the memory before; the processing module is used for combining the currently stored data items with the searched data items, averaging index data corresponding to the combined data items, and determining an averaged result as index data of the aggregated data items.
Optionally, in the multi-dimensional index computing device provided by the embodiment of the present application, the device further includes: and the filtering unit is used for filtering the data information of other dimensions except the dimensions to be analyzed in each data item after determining the data associated with each dimension in the dimensions to be analyzed in the memory, so as to obtain filtered data.
Optionally, in the multi-dimensional index computing device provided by the embodiment of the present application, the device further includes: the display module is used for writing the processed data into a database, analyzing the processed data in the database by adopting a plurality of dimensions to be analyzed, and displaying the calculated target indexes after obtaining the target indexes.
Optionally, in the multi-dimensional index computing device provided by the embodiment of the present application, the database is a relational database.
The multi-dimensional index calculation device includes a processor and a memory, the above-mentioned determination unit 10, processing unit 20, parsing unit 30, and the like are stored as program units in the memory, and the above-mentioned program units stored in the memory are executed by the processor to realize the corresponding functions.
The processor includes a kernel, and the kernel fetches the corresponding program unit from the memory. The kernel can be provided with one or more than one, and the data processing efficiency is improved when the index is calculated in real time by adjusting the kernel parameters.
The memory may include volatile memory, random Access Memory (RAM), and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), among other forms in computer readable media, the memory including at least one memory chip.
The embodiment of the application provides a storage medium, on which a program is stored, which when executed by a processor, implements a multi-dimensional index calculation method.
The embodiment of the application provides a processor, which is used for running a program, wherein the program runs to execute a multi-dimensional index calculation method.
The embodiment of the application provides equipment, which comprises a processor, a memory and a program stored in the memory and capable of running on the processor, wherein the processor realizes the following steps when executing the program: determining a plurality of dimensions to be parsed; writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data; writing the processed data into a database, and analyzing the processed data in the database by adopting the plurality of dimensions to be analyzed to obtain a plurality of target indexes.
The following steps may also be implemented when the processor executes the program: writing data items of the data to be calculated into a memory, and carrying out aggregation processing on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed, wherein the obtained processed data comprises the following steps: determining data associated with each data entry in the memory in each dimension to be parsed; and carrying out aggregation processing on the data associated with each dimension to be parsed corresponding to the same data item.
The following steps may also be implemented when the processor executes the program: writing the data item of the data to be calculated into the memory comprises: the data to be calculated is stored into a memory one by taking a data item as a unit; and according to the data associated with the data to be calculated in each dimension to be analyzed, the aggregation processing of the data items written into the memory comprises the following steps: and each time a data item is stored, determining the data associated with the currently stored data item in each dimension to be analyzed, searching the data item which is associated with the same data in each dimension to be analyzed in the data items which are stored in the memory before, combining the currently stored data item with the searched data item, averaging index data corresponding to the combined data item, and determining the average result as the index data of the aggregated data item.
The following steps may also be implemented when the processor executes the program: after determining the data associated with each of the plurality of dimensions to be parsed for each of the respective data entries in the memory, the method further includes: and filtering the data information of other dimensions except the dimensions to be parsed in each data entry to obtain filtered data.
The following steps may also be implemented when the processor executes the program: writing the processed data into a database, analyzing the processed data in the database by adopting the plurality of dimensions to be analyzed to obtain a plurality of target indexes, and then, the method further comprises the steps of: and displaying the calculated target indexes.
The following steps may also be implemented when the processor executes the program: the database is a relational database.
The device herein may be a server, PC, PAD, cell phone, etc.
The application also provides a computer program product adapted to perform, when executed on a data processing device, a program initialized with the method steps of: determining a plurality of dimensions to be parsed; writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data; writing the processed data into a database, and analyzing the processed data in the database by adopting a plurality of dimensions to be analyzed to obtain a plurality of target indexes.
The initialization is performed by a program comprising the following method steps: writing data items of the data to be calculated into a memory, and carrying out aggregation processing on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed, wherein the obtained processed data comprises the following steps: determining data associated with each data entry in the memory in each dimension to be parsed; and carrying out aggregation processing on the data items corresponding to the same data associated with each dimension to be parsed.
The initialization is performed by a program comprising the following method steps: writing the data item of the data to be calculated into the memory comprises: taking the data item as a unit, storing the data to be calculated into a memory one by one; and according to the data associated with the data to be calculated in each dimension to be analyzed, the aggregation processing of the data items written into the memory comprises the following steps: and each time a data item is stored, determining the data associated with the currently stored data item in each dimension to be analyzed, searching the data item which is associated with the same data in each dimension to be analyzed in the data items which are stored in the memory before, combining the currently stored data item with the searched data item, averaging index data corresponding to the combined data item, and determining the average result as the index data of the aggregated data item.
The initialization is performed by a program comprising the following method steps: after determining the data associated with each of the plurality of dimensions to be parsed for the respective data entry in the memory, the method further includes: and filtering the data information of other dimensions except the dimensions to be parsed in each data entry to obtain filtered data.
The initialization is performed by a program comprising the following method steps: writing the processed data into a database, analyzing the processed data in the database by adopting a plurality of dimensions to be analyzed, and obtaining a plurality of target indexes, wherein the method further comprises the following steps: and displaying the calculated multiple target indexes.
The initialization is performed by a program comprising the following method steps: the database is a relational database.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (6)
1. A multi-dimensional index calculation method, comprising:
determining a plurality of dimensions to be parsed;
writing data items of the data to be calculated into a memory, and carrying out aggregation treatment on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed to obtain the treated data;
writing the processed data into a database, and analyzing the processed data in the database by adopting the plurality of dimensions to be analyzed to obtain a plurality of target indexes;
writing data items of the data to be calculated into a memory, and performing aggregation processing on the data items written into the memory according to the data associated with the data to be calculated in each dimension to be analyzed, wherein the data after processing comprises:
determining data associated with each data entry in the memory in each dimension to be parsed;
carrying out aggregation processing on the data items corresponding to the same data associated with each dimension to be analyzed;
writing the data item of the data to be calculated into the memory comprises:
the data to be calculated is stored into a memory one by taking a data item as a unit;
and according to the data associated with the data to be calculated in each dimension to be analyzed, the aggregation processing of the data items written into the memory comprises the following steps:
determining the data associated with each dimension to be parsed of the currently stored data item every time one data item is stored, and searching the data items which are associated with the same data in each dimension to be parsed from the data items which are stored in the memory before;
combining the currently stored data items with the searched data items, averaging index data corresponding to the combined data items, and determining an averaged result as index data of the aggregated data items;
wherein, the data items written into the memory are stored in real time;
after determining the data associated with each of the plurality of dimensions to be parsed for each of the respective data entries in the memory, the method further includes: and filtering the data information of other dimensions except the dimensions to be parsed in each data entry to obtain filtered data.
2. The method of claim 1, wherein the processed data is written into a database, wherein the plurality of dimensions to be parsed are used to parse the processed data in the database to obtain a plurality of target metrics, and wherein the method further comprises: and displaying the calculated target indexes.
3. The method of claim 1, wherein the database is a relational database.
4. A multi-dimensional index computing device, comprising:
a determining unit configured to determine a plurality of dimensions to be parsed;
the processing unit is used for writing data items of the data to be calculated into the memory, and carrying out aggregation processing on the data items written into the memory according to the data associated with each dimension to be analyzed of the data to be calculated to obtain processed data;
a parsing unit, configured to write the processed data into a database, and parse the processed data in the database by using the multiple dimensions to be parsed to obtain multiple target indexes;
wherein the processing unit further comprises:
the determining module is used for determining the data associated with each data item in the memory in each dimension to be analyzed;
the processing module is used for carrying out aggregation processing on the data items corresponding to the same data associated with each dimension to be analyzed;
the processing unit further includes: the storage module is used for storing the data to be calculated into the memory one by taking the data item as a unit; the searching module is used for determining the data associated with each dimension to be analyzed of the currently stored data item and searching the data items which are associated with the same data in each dimension to be analyzed from the data items which are stored in the memory before; the processing module is used for combining the currently stored data items with the searched data items, averaging index data corresponding to the combined data items and determining an averaged result as index data of the aggregated data items;
the device is also used for storing the data items in the writing memory in real time;
the apparatus further comprises: and the filtering unit is used for filtering the data information of other dimensions except the dimensions to be analyzed in each data item after determining the data associated with each dimension in the dimensions to be analyzed in the memory, so as to obtain filtered data.
5. A storage medium comprising a stored program, wherein the program performs the multi-dimensional index calculation method of any one of claims 1 to 3.
6. A processor for running a program, wherein the program runs on performing the multi-dimensional index calculation method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811126783.2A CN110955654B (en) | 2018-09-26 | 2018-09-26 | Multi-dimensional index calculation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811126783.2A CN110955654B (en) | 2018-09-26 | 2018-09-26 | Multi-dimensional index calculation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110955654A CN110955654A (en) | 2020-04-03 |
| CN110955654B true CN110955654B (en) | 2023-10-31 |
Family
ID=69966183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811126783.2A Active CN110955654B (en) | 2018-09-26 | 2018-09-26 | Multi-dimensional index calculation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110955654B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002003251A2 (en) * | 2000-06-29 | 2002-01-10 | Alphablox Corporation | Caching scheme for multi-dimensional data |
| WO2012040576A1 (en) * | 2010-09-24 | 2012-03-29 | International Business Machines Corporation | Evidence profiling |
| CN104317958A (en) * | 2014-11-12 | 2015-01-28 | 北京国双科技有限公司 | Method and system for processing data in real time |
| CN104408179A (en) * | 2014-12-15 | 2015-03-11 | 北京国双科技有限公司 | Method and device for processing data from data table |
| CN104484398A (en) * | 2014-12-12 | 2015-04-01 | 北京国双科技有限公司 | Method and device for aggregation of data in datasheet |
| CN106709012A (en) * | 2016-12-26 | 2017-05-24 | 北京锐安科技有限公司 | Method and device for analyzing big data |
| CN108073349A (en) * | 2016-11-08 | 2018-05-25 | 北京国双科技有限公司 | The transmission method and device of data |
| CN108173679A (en) * | 2017-12-21 | 2018-06-15 | 北京计算机技术及应用研究所 | A kind of storage system for IT system O＆M monitoring data |
| CN108268536A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | Database aggregation processing method and device |
| CN108304454A (en) * | 2017-11-27 | 2018-07-20 | 大象慧云信息技术有限公司 | Invoice data real time aggregation device based on big data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10191963B2 (en) * | 2015-05-29 | 2019-01-29 | Oracle International Corporation | Prefetching analytic results across multiple levels of data |
-
2018
- 2018-09-26 CN CN201811126783.2A patent/CN110955654B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2002003251A2 (en) * | 2000-06-29 | 2002-01-10 | Alphablox Corporation | Caching scheme for multi-dimensional data |
| WO2012040576A1 (en) * | 2010-09-24 | 2012-03-29 | International Business Machines Corporation | Evidence profiling |
| CN104317958A (en) * | 2014-11-12 | 2015-01-28 | 北京国双科技有限公司 | Method and system for processing data in real time |
| CN104484398A (en) * | 2014-12-12 | 2015-04-01 | 北京国双科技有限公司 | Method and device for aggregation of data in datasheet |
| CN104408179A (en) * | 2014-12-15 | 2015-03-11 | 北京国双科技有限公司 | Method and device for processing data from data table |
| CN108073349A (en) * | 2016-11-08 | 2018-05-25 | 北京国双科技有限公司 | The transmission method and device of data |
| CN106709012A (en) * | 2016-12-26 | 2017-05-24 | 北京锐安科技有限公司 | Method and device for analyzing big data |
| CN108268536A (en) * | 2016-12-30 | 2018-07-10 | 北京国双科技有限公司 | Database aggregation processing method and device |
| CN108304454A (en) * | 2017-11-27 | 2018-07-20 | 大象慧云信息技术有限公司 | Invoice data real time aggregation device based on big data |
| CN108173679A (en) * | 2017-12-21 | 2018-06-15 | 北京计算机技术及应用研究所 | A kind of storage system for IT system O＆M monitoring data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110955654A (en) | 2020-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3117347B1 (en) | Systems and methods for rapid data analysis | |
| US10754877B2 (en) | System and method for providing big data analytics on dynamically-changing data models | |
| WO2016004813A1 (en) | Data storage method, query method and device | |
| CN107408114B (en) | Identifying join relationships based on transactional access patterns | |
| CN107203640B (en) | Method and system for establishing physical model through database operation record | |
| CN105550270B (en) | Data base query method and device | |
| US11550762B2 (en) | Implementation of data access metrics for automated physical database design | |
| CN112241420A (en) | Government affair service item recommendation method based on association rule algorithm | |
| US8589451B1 (en) | Systems and methods for generating a common data model for relational and object oriented databases | |
| US20190303421A1 (en) | Histogram sketching for time-series data | |
| CN106874332B (en) | Database access method and device | |
| US20170195449A1 (en) | Smart proxy for datasources | |
| US20180341709A1 (en) | Unstructured search query generation from a set of structured data terms | |
| CN110825764A (en) | SQL script generation method, system, storage medium and processor | |
| CN110955654B (en) | Multi-dimensional index calculation method and device | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN107515916B (en) | Performance optimization method and device for data query | |
| CN106933909B (en) | Multi-dimensional data query method and device | |
| CN112506953A (en) | Query method, device and storage medium based on Structured Query Language (SQL) | |
| CN110019771B (en) | Text processing method and device | |
| US9244988B2 (en) | Dynamic relevant reporting | |
| CN108062329B (en) | Data import method and device | |
| CN111159214A (en) | API access method and device, electronic equipment and storage medium | |
| CN108073596B (en) | Data deletion method and device for OLAP database | |
| CN111125155B (en) | Access path-based data query method, device, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |