CN111026759B - Report generation method and device based on Hbase - Google Patents
Report generation method and device based on Hbase Download PDFInfo
- Publication number
- CN111026759B CN111026759B CN201911264252.4A CN201911264252A CN111026759B CN 111026759 B CN111026759 B CN 111026759B CN 201911264252 A CN201911264252 A CN 201911264252A CN 111026759 B CN111026759 B CN 111026759B
- Authority
- CN
- China
- Prior art keywords
- report
- generated
- item
- dimension
- hbase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000005259 measurement Methods 0.000 claims abstract description 85
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 4
- 238000013480 data collection Methods 0.000 claims description 2
- 238000013500 data storage Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/221—Column-oriented storage; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention discloses a report generation method and device based on Hbase, wherein the method comprises the following steps: receiving a selection instruction input from the outside, wherein the selection instruction is used for selecting one or more dimension items to be generated into a report and one or more measurement items associated with each dimension item; inquiring data of each dimension item corresponding to each measurement item of a report to be generated from a pre-constructed Hbase wide table according to each dimension item and each measurement item selected by a selection instruction, wherein the Hbase wide table comprises: a first column cluster for storing the statistical data of each dimension item and a second column cluster for storing the statistical data of each measure item; and generating the report to be generated according to the data of each dimension item of the report to be generated, which corresponds to each measurement item. According to the method and the system, the statistics dimension and the measurement index selected by the user can be inquired or counted in real time, and then a report is generated.
Description
Technical Field
The invention relates to the technical field of big data, in particular to a report generation method and device based on Hbase.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Along with the rapid development of networks, more and more data are generated on the networks, so that how to process massive data to generate reports for presenting data from various dimensions can greatly mine valuable information from the massive data.
Currently, the prior art mainly generates various reports in the following two ways: (1) writing data into a wide table corresponding to each data mart in advance, and directly inquiring and displaying in a report page; (2) after the data is preliminarily summarized, the data is written into a relational database (for example, an SQL database), the relational database is temporarily assembled and inquired according to the temporarily selected display dimension or measurement item of the report page, and the report page is returned after the data is counted.
Because mass data can be generated in a big data environment, the following defects exist by utilizing the existing report generation method: (1) when the data volume is too large, a wide table is constructed for each report in advance, the data redundancy is too much, and the storage pressure is very high; (2) some Hbase, HDFS, hive and the like are suitable for databases in big data environments, and are not efficient in real-time data query and statistics because SQL (structured query language) is not supported.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the invention provides a report generation method based on Hbase, which is used for solving the technical problems that in the prior art, report data are stored by adopting a relational database, excessive redundant data can be generated in a big data environment, and report data are stored by adopting a big data distributed database, so that the real-time query and statistics efficiency of the data are low, and the method comprises the following steps: receiving a selection instruction input from the outside, wherein the selection instruction is used for selecting one or more dimension items to be generated into a report and one or more measurement items associated with each dimension item; inquiring data of each dimension item corresponding to each measurement item of a report to be generated from a pre-constructed Hbase wide table according to each dimension item and each measurement item selected by a selection instruction, wherein the Hbase wide table comprises: a first column cluster for storing the statistical data of each dimension item and a second column cluster for storing the statistical data of each measure item; generating a report to be generated according to the data of each dimension item of the report to be generated, which corresponds to each measurement item; the method further comprises the steps of: acquiring each pre-configured dimension item and a measurement item associated with each dimension item; and constructing an Hbase wide table according to each dimension item and each measurement item which are preconfigured and the dimension item and the measurement item contained in each report to be generated, wherein each row of the Hbase wide table corresponds to the dimension item and the measurement item of one data record in one report to be generated.
The embodiment of the invention also provides a report generating device based on Hbase, which is used for solving the technical problems that in the prior art, report data is stored by adopting a relational database, excessive redundant data can be generated in a big data environment, report data is stored by adopting a big data distributed database, and the real-time query and statistics efficiency of the data are low, and the device comprises: the system comprises an instruction receiving module, a storage module and a storage module, wherein the instruction receiving module is used for receiving an externally input selection instruction, and the selection instruction is used for selecting one or more dimension items of a report to be generated and one or more measurement items associated with each dimension item; the data query module is used for querying data of each dimension item corresponding to each measurement item of a report to be generated from a pre-constructed Hbase wide table according to each dimension item and each measurement item selected by the selection instruction, wherein the Hbase wide table comprises: a first column cluster for storing the statistical data of each dimension item and a second column cluster for storing the statistical data of each measure item; the report generation module is used for generating a report to be generated according to the data of each dimension item of the report to be generated, which corresponds to each measurement item; the apparatus further comprises: the configuration module is used for acquiring each dimension item which is configured in advance and a measurement item which is associated with each dimension item; and the Hbase wide table construction module is used for constructing a Hbase wide table according to each dimension item and each measurement item which are configured in advance and the dimension item and the measurement item contained in each report to be generated, wherein each row of the Hbase wide table corresponds to the dimension item and the measurement item of one data record in one report to be generated.
The embodiment of the invention also provides a computer device which is used for solving the technical problems that the prior art stores report data by adopting a relational database, excessive redundant data can be generated in a big data environment, and the big data distributed database stores the report data, and the real-time query and the statistics efficiency of the data are low.
The embodiment of the invention also provides a computer readable storage medium for solving the technical problems that in the prior art, the relational database is adopted to store report data, excessive redundant data can be generated in a big data environment, and the big data distributed database is adopted to store the report data, so that the real-time data query and the statistics efficiency are low.
In the embodiment of the invention, an Hbase wide table is constructed in an Hbase database in advance according to the statistical dimension and the measurement index contained in all the reports to be generated so as to write the statistical data of all the reports to be generated into the Hbase wide table, and after receiving a selection instruction input by a user, the data of each dimension item corresponding to each measurement item of the reports to be generated are inquired from the Hbase wide table constructed in advance according to each dimension item and each measurement item selected by the selection instruction, and then the reports to be generated are generated according to the data of each dimension item corresponding to each measurement item of the reports to be generated.
According to the embodiment of the invention, the statistical dimension and the measurement index selected by the user can be customized, and the statistical report data can be queried or counted in real time, so that a report can be generated.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a flowchart of a report generation method based on Hbase, which is provided in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a table structure according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a report generating device based on Hbase according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are open-ended terms, meaning including, but not limited to. Reference to the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The sequence of steps involved in the embodiments is used to schematically illustrate the practice of the present application, and is not limited thereto and may be appropriately adjusted as desired.
The embodiment of the invention provides a report generation method based on Hbase, which can be applied to but not limited to a cross-cluster big data association query system. According to the embodiment of the invention, the data is written into the HBASE wide table in advance, and query calculation is performed in real time according to the selection of the statistical dimension and the measurement index during page display.
FIG. 1 is a flowchart of a report generating method based on Hbase, which is provided in an embodiment of the present invention, and as shown in FIG. 1, the method may include the following steps:
s101, receiving an externally input selection instruction, wherein the selection instruction is used for selecting one or more dimension items to be generated into a report, and one or more measurement items associated with each dimension item.
It should be noted that, in the embodiment of the present invention, each dimension item corresponds to one statistical dimension; each measurement item corresponds to one measurement index, and the selection instruction received in S101 may be an instruction for the user to check the statistical dimension and the measurement index of the report to be generated on the report display page. Optionally, in the report presentation page provided by the embodiment of the invention, each dimension item or measurement item can be set to realize linkage selection, and only the combination of the dimension item and measurement item with statistical relationship is allowed to be checked, namely when a certain item or several dimension or measurement items are selected, only other dimension or measurement items related to the certain item or measurement item are allowed to be continuously checked.
In general, in an information system, the dimension terms and the measure terms used for statistical analysis are limited, so that according to the embodiment of the invention, a transverse table of the table structure shown in fig. 2 is created in advance according to all the statistical dimensions and measure indexes of each report to be generated, and each row of the transverse table represents the dimension terms and the measure terms contained in one table to be generated.
For example, for a network management system, dimension terms for statistics may include: time, province, city, district, device type, device vendor, port type, etc.; the metric term may include: the number of devices, the number of ports, the average upstream traffic, the maximum transmit optical power, etc. By arbitrarily combining the dimensions and associated metrics, a data statistics of various dimensions can be generated.
S102, according to each dimension item and each measurement item selected by a selection instruction, inquiring data of each dimension item corresponding to each measurement item of a report to be generated from a pre-constructed Hbase wide table, wherein the Hbase wide table comprises: a first column cluster for storing the statistics of each dimension item and a second column cluster for storing the statistics of each measure item.
As an optional implementation manner, the Hbase-based report generating method provided by the embodiment of the invention can construct a Hbase wide table by the following steps: acquiring each pre-configured dimension item and a measurement item associated with each dimension item; and constructing an Hbase wide table according to each dimension item and each measurement item which are preconfigured and the dimension item and the measurement item contained in each report to be generated, wherein each row of the Hbase wide table corresponds to the dimension item and the measurement item of one data record in one report to be generated. According to the embodiment of the invention, all the statistical data related to the report are written into the same set of wide tables in the Hbase database, and the statistical data can be layered according to the combination of the dimension and the measurement, so that the dimension item and the measurement item to be checked can be temporarily selected when the page queries the report. In addition, after all dimensions and measurement items to be checked are selected, the data of a designated column is queried from the Hbase wide table, a report is generated, and the report is displayed in real time.
It should be noted that, as shown in fig. 2, the table structure of the Hbase wide table constructed in the embodiment of the present invention also constructs the Hbase table in the Hbase database, and defines at least two column clusters, one for managing columns of each statistical dimension (i.e., the first column cluster) and one for managing columns of each metric index (i.e., the second column cluster).
Further, after constructing the Hbase width table according to each dimension item and each measurement item which are preconfigured, and each dimension item and measurement item contained in each report to be generated, the method further comprises the following steps: collecting statistical data of each report to be generated; and writing the statistical data of each report to be generated into the Hbase wide table.
After the original data of each report to be generated are collected, carrying out layered summarization, and writing summarization results into an Hbase wide table constructed by an Hbase database. It should be noted that, when writing data, if a certain column dimension term or a certain column dimension term in the Hbase wide table does not relate to the layer statistics, the field data is set to be-1 or null.
Optionally, when collecting the statistical data of each report to be generated, the method can be implemented as follows: collecting statistical data of a first report to be generated; and generating the statistical data of the second report to be generated according to the statistical data of the first report to be generated, wherein dimension items or measurement items contained in the second report to be generated are all in the first report to be generated.
It should be noted that when the hierarchical summary data is written into the Hbase wide table, the hierarchical summary data can be respectively summarized according to the sequence from more to less according to the number of dimensions; if the dimension and the measure of a certain layer of summary are all subsets of a certain item of summary before, performing secondary summary directly according to the result of the previous summary, if the number of devices is summarized according to the date+region+device type before, then performing summary directly on the basis of the previous result when the devices are summarized according to the date+region later; and if the stored results are not summarized for the second time, statistics is directly carried out according to the original data.
And S103, generating the report to be generated according to the data of each dimension item of the report to be generated, which corresponds to each measurement item.
Specifically, in S103, after the user selects all dimension items and corresponding measurement items of the report to be generated in the checking mode, the HBase database may be queried for data of a specified column, and the generated report may be displayed on the report display page in real time.
It can be seen that, according to the Hbase-based report generating method provided by the embodiment of the present invention, by constructing an Hbase wide table in the Hbase database in advance according to the statistical dimensions and the measurement indexes included in all the reports to be generated, so as to write the statistical data of all the reports to be generated into the Hbase wide table, after receiving the selection instruction input by the user, according to each dimension item and each measurement item selected by the selection instruction, querying the data of each dimension item of the reports to be generated corresponding to each measurement item from the Hbase wide table constructed in advance, and further generating the reports to be generated according to the data of each dimension item of the reports to be generated corresponding to each measurement item.
According to the Hbase-based report generation method provided by the embodiment of the invention, the report data can be queried or counted in real time according to the statistical dimension and the measurement index selected by the user in a self-defining way, so that a report is generated.
Based on the same inventive concept, the embodiment of the invention also provides a report generating device based on Hbase, as described in the following embodiment. Because the principle of solving the problem of the embodiment of the device is similar to that of the report generation method based on Hbase, the implementation of the embodiment of the device can be referred to the implementation of the method, and the repetition is omitted.
Fig. 3 is a schematic diagram of a report generating device based on Hbase according to an embodiment of the present invention, where, as shown in fig. 3, the device may include: an instruction receiving module 31, a data query module 32 and a report generating module 33.
The instruction receiving module 31 is configured to receive a selection instruction input from the outside, where the selection instruction is used to select one or more dimension items of the report to be generated, and one or more measurement items associated with each dimension item; the data query module 32 is configured to query, from a pre-constructed Hbase wide table, data of each dimension item corresponding to each measurement item in the report to be generated according to each dimension item and each measurement item selected by the selection instruction, where the Hbase wide table includes: a first column cluster for storing the statistical data of each dimension item and a second column cluster for storing the statistical data of each measure item; the report generating module 33 is configured to generate a report to be generated according to the data corresponding to each dimension item of the report to be generated.
In an optional embodiment, the Hbase-based report generating apparatus provided in the embodiment of the present invention may further include: a configuration module 34, configured to obtain each dimension item configured in advance, and a metric item associated with each dimension item; and the Hbase wide table construction module 35 is configured to construct an Hbase wide table according to each dimension item and each measurement item which are configured in advance, and the dimension item and the measurement item contained in each report to be generated, wherein each row of the Hbase wide table corresponds to the dimension item and the measurement item of one data record in one report to be generated.
In an optional embodiment, the Hbase-based report generating apparatus provided in the embodiment of the present invention may further include: the data acquisition module 36 is used for acquiring the statistical data of each report to be generated; the data storage module 37 is configured to write the statistics of each report to be generated into the Hbase wide table.
Optionally, the data collection module 36 is further configured to collect statistical data of the first report to be generated; and generating the statistical data of the second report to be generated according to the statistical data of the first report to be generated, wherein the dimension items or the measurement items contained in the second report to be generated are all in the first report to be generated.
The embodiment of the invention also provides a computer device which is used for solving the technical problems that the prior art stores report data by adopting a relational database, excessive redundant data can be generated in a big data environment, and the big data distributed database stores the report data, and the real-time query and the statistics efficiency of the data are low.
The embodiment of the invention also provides a computer readable storage medium for solving the technical problems that in the prior art, the relational database is adopted to store report data, excessive redundant data can be generated in a big data environment, and the big data distributed database is adopted to store the report data, so that the real-time data query and the statistics efficiency are low.
In summary, the embodiments of the present invention provide a method, an apparatus, a computer device, and a computer readable storage medium for generating a report based on Hbase, by listing all statistical dimensions that may be involved in the report, and constructing an analysis table and an Hbase wide table according to the dimensions that may be simultaneously presented in the same report and metric items related to the dimensions. And (3) listing all dimension items and measurement items which are possibly used for statistical analysis in the analysis table, arranging all the dimension items and measurement items into a transverse table by taking the columns as columns, and writing each combination which possibly forms a report into a row so as to support a user to select and realize the selection combination of the dimension items and the measurement items. In the statistical wide table, adopting a data structure consistent with the analysis table; and carrying out layered summarization on the original data by utilizing the constructed statistical HBase wide table, and inquiring the data of a designated column in the HBase after each dimension item and each measurement item of the report to be generated are selected through the analysis table, so as to generate report display at the front end in real time.
According to the embodiment of the invention, only one complete Hbase wide table is constructed for the report business of the same large class, the data is written into the Hbase wide table in advance, all the dimensions and index information with relevance are written into one table to be realized, and the fact tables with relevance are combined and recombined, so that interactive real-time report construction logic can be realized.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. The report generation method based on Hbase is characterized by comprising the following steps:
receiving an externally input selection instruction, wherein the selection instruction is used for selecting one or more dimension items to be generated into a report, and one or more measurement items associated with each dimension item;
inquiring data of each dimension item of the report to be generated corresponding to each measurement item from a pre-constructed Hbase wide table according to each dimension item and each measurement item selected by the selection instruction, wherein the Hbase wide table comprises: a first column cluster for storing statistics of each dimension item and a second column cluster for storing statistics of each dimension item;
generating the report to be generated according to the data of each dimension item of the report to be generated, which corresponds to each measurement item;
the method further comprises the steps of:
acquiring each pre-configured dimension item and a measurement item associated with each dimension item;
and constructing an Hbase wide table according to each dimension item and each measurement item which are preconfigured and the dimension item and the measurement item contained in each report to be generated, wherein each row of the Hbase wide table corresponds to the dimension item and the measurement item of one data record in one report to be generated.
2. The method of claim 1, wherein after constructing the Hbase width table from the preconfigured individual dimension terms and individual measure terms, and the individual dimension terms and measure terms included in the report to be generated, the method further comprises:
collecting statistical data of each report to be generated;
and writing the statistical data of each report to be generated into the Hbase wide table.
3. The method of claim 2, wherein collecting statistics for each report to be generated comprises:
collecting statistical data of a first report to be generated;
generating the statistical data of a second report to be generated according to the statistical data of the first report to be generated, wherein dimension items or measurement items contained in the second report to be generated are all in the first report to be generated.
4. A report generating device based on Hbase, comprising:
the system comprises an instruction receiving module, a storage module and a storage module, wherein the instruction receiving module is used for receiving an externally input selection instruction, and the selection instruction is used for selecting one or more dimension items of a report to be generated and one or more measurement items associated with each dimension item;
the data query module is configured to query, from a pre-constructed Hbase wide table, data of each dimension item corresponding to each measurement item of the report to be generated according to each dimension item and each measurement item selected by the selection instruction, where the Hbase wide table includes: a first column cluster for storing statistics of each dimension item and a second column cluster for storing statistics of each dimension item;
the report generation module is used for generating the report to be generated according to the data of each dimension item of the report to be generated, which corresponds to each measurement item;
the apparatus further comprises:
the configuration module is used for acquiring each dimension item which is configured in advance and a measurement item which is associated with each dimension item;
and the Hbase wide table construction module is used for constructing a Hbase wide table according to each dimension item and each measurement item which are configured in advance and the dimension item and the measurement item contained in each report to be generated, wherein each row of the Hbase wide table corresponds to the dimension item and the measurement item of one data record in one report to be generated.
5. The apparatus of claim 4, wherein the apparatus further comprises:
the data acquisition module is used for acquiring the statistical data of each report to be generated;
and the data storage module is used for writing the statistical data of each report to be generated into the Hbase wide table.
6. The apparatus of claim 5, wherein the data collection module is further configured to collect statistics of a first report to be generated; and generating the statistical data of a second report to be generated according to the statistical data of the first report to be generated, wherein dimension items or measurement items contained in the second report to be generated are all in the first report to be generated.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the Hbase-based report generating method according to any one of claims 1 to 3 when said computer program is executed.
8. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the Hbase-based report generating method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911264252.4A CN111026759B (en) | 2019-12-11 | 2019-12-11 | Report generation method and device based on Hbase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911264252.4A CN111026759B (en) | 2019-12-11 | 2019-12-11 | Report generation method and device based on Hbase |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111026759A CN111026759A (en) | 2020-04-17 |
| CN111026759B true CN111026759B (en) | 2024-03-12 |
Family
ID=70208798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911264252.4A Active CN111026759B (en) | 2019-12-11 | 2019-12-11 | Report generation method and device based on Hbase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026759B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563082B (en) * | 2020-04-30 | 2023-09-19 | 中国银行股份有限公司 | Report generation method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1492336A (en) * | 2003-09-04 | 2004-04-28 | 上海格尔软件股份有限公司 | Information system auditing method based on data storehouse |
| US7143107B1 (en) * | 2003-06-26 | 2006-11-28 | Microsoft Corporation | Reporting engine for data warehouse |
| WO2016041373A1 (en) * | 2014-09-16 | 2016-03-24 | 中兴通讯股份有限公司 | Data query method and device |
| CN107451109A (en) * | 2017-07-05 | 2017-12-08 | 北京京东尚科信息技术有限公司 | Report form generation method and system |
| CN108664638A (en) * | 2018-05-15 | 2018-10-16 | 口碑(上海)信息技术有限公司 | Report form generation method based on index system and device |
| CN109508338A (en) * | 2018-11-23 | 2019-03-22 | 成都四方伟业软件股份有限公司 | Data sheet correlating method and device |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN110297855A (en) * | 2019-05-22 | 2019-10-01 | 深圳壹账通智能科技有限公司 | Report Dynamic Display method, apparatus, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020158B (en) * | 2012-11-26 | 2016-09-07 | 中兴通讯股份有限公司 | A kind of report form creation, device and system |
-
2019
- 2019-12-11 CN CN201911264252.4A patent/CN111026759B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7143107B1 (en) * | 2003-06-26 | 2006-11-28 | Microsoft Corporation | Reporting engine for data warehouse |
| CN1492336A (en) * | 2003-09-04 | 2004-04-28 | 上海格尔软件股份有限公司 | Information system auditing method based on data storehouse |
| WO2016041373A1 (en) * | 2014-09-16 | 2016-03-24 | 中兴通讯股份有限公司 | Data query method and device |
| CN107451109A (en) * | 2017-07-05 | 2017-12-08 | 北京京东尚科信息技术有限公司 | Report form generation method and system |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN108664638A (en) * | 2018-05-15 | 2018-10-16 | 口碑(上海)信息技术有限公司 | Report form generation method based on index system and device |
| CN109508338A (en) * | 2018-11-23 | 2019-03-22 | 成都四方伟业软件股份有限公司 | Data sheet correlating method and device |
| CN110297855A (en) * | 2019-05-22 | 2019-10-01 | 深圳壹账通智能科技有限公司 | Report Dynamic Display method, apparatus, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于数据仓库的指标体系研究;刘黎志;;计算机技术与发展(01);全文 * |
| 大型企业报表引擎的优化设计与实现;何拥军;曾文权;;计算机技术与发展(08);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111026759A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102521416B (en) | Data correlation query method and data correlation query device | |
| US20240078229A1 (en) | Generating, accessing, and displaying lineage metadata | |
| CN108038222B (en) | System of entity-attribute framework for information system modeling and data access | |
| CN107622103B (en) | Managing data queries | |
| US9824127B2 (en) | Systems and methods for interest-driven data visualization systems utilized in interest-driven business intelligence systems | |
| CN107016001B (en) | Data query method and device | |
| US9836519B2 (en) | Densely grouping dimensional data | |
| WO2015041714A1 (en) | Interest-driven business intelligence systems including event-oriented data | |
| US20100153409A1 (en) | Computer-Implemented Systems And Methods For Providing Paginated Search Results From A Database | |
| CN103778135A (en) | Method for distribution storage and paging querying of real-time data | |
| US20060271583A1 (en) | Dimension member sliding in online analytical processing | |
| CN107273482A (en) | Alarm data storage method and device based on HBase | |
| CN105117442B (en) | A kind of big data querying method based on probability | |
| Chevalier et al. | How can we implement a multidimensional data warehouse using NoSQL? | |
| WO2015041731A1 (en) | Interest-driven business intelligence systems including segment data | |
| CN103036921A (en) | User behavior analysis system and method | |
| CN105022791A (en) | Novel KV distributed data storage method | |
| CN111026759B (en) | Report generation method and device based on Hbase | |
| CN114064660A (en) | Data structured analysis method based on ElasticSearch | |
| CN104866508B (en) | The method and apparatus of file is managed under cloud environment | |
| Khalil et al. | New approach for implementing big datamart using NoSQL key-value stores | |
| KR20050044380A (en) | Data joining/displaying method | |
| CN115658680A (en) | Data storage method, data query method and related device | |
| CN114443742A (en) | K line graph display method, device and equipment | |
| CN112667859A (en) | Data processing method and device based on memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |