CN114510488A - Statistical method, system, equipment and medium for upward summarization of layer data - Google Patents
Statistical method, system, equipment and medium for upward summarization of layer data Download PDFInfo
- Publication number
- CN114510488A CN114510488A CN202210168216.3A CN202210168216A CN114510488A CN 114510488 A CN114510488 A CN 114510488A CN 202210168216 A CN202210168216 A CN 202210168216A CN 114510488 A CN114510488 A CN 114510488A
- Authority
- CN
- China
- Prior art keywords
- data
- hierarchical
- fields
- fact
- hierarchy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a statistical method, a statistical system, statistical equipment and a statistical medium for upward summarization of hierarchical data, wherein the statistical method comprises the following steps: receiving an operation instruction, and acquiring a hierarchy label and a hierarchy data relation table according to the operation instruction; determining a corresponding fact data table by combining with the operation instruction, and acquiring to-be-processed fact data according to the fact data table, wherein the to-be-processed fact data comprises a plurality of fields; judging whether the fields need to be summarized upwards according to preset requirements; when the dry fields need to be summarized upwards, the fact data table is associated with the level data table, and whether level fields exist in the fields or not is judged according to the level data table; and if the hierarchical field exists, splitting the hierarchical field, acquiring the unit field, performing statistics again on the fact data to be processed according to the unit field, and performing upward summary through a hierarchical data relation table. The invention can reduce redundant data, reduce disk occupation and flexibly control the data processing mode.
Description
Technical Field
The invention relates to the technical field of industrial internet, in particular to a statistical method, a statistical system, a statistical device and a statistical medium for upwards summarizing level data.
Background
In the prior art, in order to meet the universality, the data formats of the industrial internet platform are more, but the hierarchical data traffic of an organization can be automatically summarized upwards, and for example, the following hierarchical relations exist: a is the previous stage of B, B is the previous stage of C, if the fact data occurs on C, then A and B will generate a piece of data respectively when counting data, resulting in data redundancy. The existing statistical method of the hierarchy data summarization upwards, namely, is realized by the upward redundant data, for example, data C and D with ID 1 and 2 exist now, as shown in Table 1, the upper level of C has A and B, the upper level of D has A, when the fact repair data occurs once at C and D, respectively, the existing data will have the redundant data with ID3-5 as shown in Table 2.
TABLE 1
| Id | Column | Count |
| 1 | C | 1 |
| 2 | D | 2 |
TABLE 2
| Id | Column | Count |
| 1 | C | 1 |
| 2 | B | 1 |
| 3 | A | 1 |
| 4 | D | 1 |
| 5 | A | 1 |
However, in this way, when the data hierarchy is deep, a large amount of redundant data exists, which causes disk waste and is inconvenient for data maintenance, and in addition, each piece of fact data is generated and is sequentially stored according to the hierarchy structure, so that the fact data and the redundant data cannot be distinguished when data query is performed, and only upward summary is possible.
Therefore, a statistical method for summarization up is needed to reduce redundant data, facilitate data maintenance, and provide flexible data processing.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a statistical method, system, device and medium for aggregating hierarchical data upwards.
A statistical method for upward summarization of hierarchical data comprises the following steps: receiving an operation instruction, and acquiring a hierarchy label and a hierarchy data relation table according to the operation instruction; determining a corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, and acquiring fact data to be processed according to the fact data table, wherein the fact data to be processed comprises a plurality of fields; judging whether the fields need to be summarized upwards according to preset requirements; if the fields need to be summarized upwards, associating the fact data table with a hierarchical data table, and judging whether hierarchical fields exist in the fields according to the hierarchical data table; and if the hierarchical field exists, splitting the hierarchical field, acquiring a unit field, performing statistics again on the fact data to be processed according to the unit field, and performing upward summary through the hierarchical data relation table.
In one embodiment, before the receiving the instruction, according to the received operation instruction, and according to the operation instruction, obtaining the hierarchy tag and the hierarchy data relationship table, the method further includes: creating a first table and a second table, storing the fact data and the hierarchy data in the first table and the second table respectively, and acquiring a fact data table and a hierarchy data table; creating a third table, storing the hierarchical relationship in a hierarchical data table in the third table, and acquiring the hierarchical data relationship table, wherein fields of the hierarchical data relationship table comprise hierarchical label types, the hierarchical data table, associated fields, and fields and arrays of the hierarchical data table.
In one embodiment, the determining a corresponding fact data table according to the operation instruction, the hierarchical tag, and the hierarchical data relationship table, and acquiring the fact data to be processed according to the fact data table specifically include: acquiring an associated field in the fact data table according to the operation instruction; determining a corresponding hierarchical data table in the hierarchical data relation table according to the hierarchical label; determining a corresponding fact data table according to the hierarchy data table; and performing association query in the fact data table according to the association field to acquire the fact data to be processed.
In one embodiment, the determining whether the fields need to be summarized upwards according to a preset requirement further includes: and when the fields do not need to be summarized upwards, counting the fact data to be processed according to the fields.
In one embodiment, if the fields need to be summarized upwards, the associating the fact data table with a hierarchical data table, and determining whether a hierarchical field exists in the fields according to the hierarchical data table further includes: and when no hierarchical field exists in the fields, counting the fact data to be processed according to the fields, and carrying out upward summary through a corresponding hierarchical data relation table.
In one embodiment, if there is a hierarchical field, splitting the hierarchical field, acquiring a unit field, performing statistics again on the fact data to be processed according to the unit field, and performing upward summarization through the hierarchical data relationship table, specifically including: acquiring a hierarchy label type according to the hierarchy field, and inquiring to obtain an associated fact data table according to the hierarchy label type; inquiring field information in the associated fact data table according to the associated field; splitting the hierarchical field, acquiring a unit field, and acquiring corresponding unit field information according to the unit field; and constructing a new data table according to the unit field and the unit field information, performing statistics again on the fact data to be processed, and performing upward summary according to the new data table based on the hierarchical data relation table.
A statistical system for upward summarization of hierarchical data, comprising: the operation instruction receiving module is used for receiving an operation instruction and acquiring a hierarchy label and a hierarchy data relation table according to the operation instruction; the fact data acquisition module is used for determining a corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, and acquiring fact data to be processed according to the fact data table, wherein the fact data to be processed comprises a plurality of fields; the upward gathering judgment module is used for judging whether the fields need to be gathered upward according to a preset requirement; the hierarchical field judging module is used for associating the fact data table with a hierarchical data table when the fields need to be summarized upwards, and judging whether hierarchical fields exist in the fields or not according to the hierarchical data table; and the data summarizing and counting module is used for splitting the hierarchy field when the hierarchy field exists, acquiring the unit field, carrying out statistics again on the fact data to be processed according to the unit field, and carrying out upward summarization through the hierarchy data relation table.
An apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the statistical method for aggregating hierarchical data up described in the above embodiments.
A medium on which a computer program is stored, which when executed by a processor implements the steps of the statistical method for aggregating hierarchical data described in the various embodiments above.
Compared with the prior art, the invention has the advantages and beneficial effects that: the invention can acquire the hierarchy label and the hierarchy data relation table according to the operation instruction by receiving the operation instruction, and determine the corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, thereby realizing data processing by adopting a mode of processing the fact data table, acquiring the fact data to be processed according to the fact data table, wherein the fact data to be processed comprises a plurality of fields, judging whether the fields need to be summarized upwards according to the preset requirement, avoiding generating excessive redundant data, reducing the occupation of a disk, associating the fact data table with the hierarchy data table when the fields need to be summarized upwards, judging whether the hierarchy fields exist in the fields according to the hierarchy data table, splitting the hierarchy fields when the hierarchy fields exist, acquiring the unit fields, and performing statistics again according to the unit fields, and the data processing method carries out upward summarization through the hierarchical data relation table, thereby being capable of carrying out data processing in an upward summarization mode and realizing flexible control of the data processing mode.
Drawings
FIG. 1 is a flow diagram illustrating a statistical method for layer-level data summarization in one embodiment;
FIG. 2 is a block diagram of a statistical system for layer-level data summarization up, according to an embodiment;
fig. 3 is a schematic diagram of the internal structure of the apparatus in one embodiment.
Detailed Description
Before describing the embodiments of the present invention, the overall concept of the present invention will be described as follows:
the invention is mainly developed according to the process of data up-gathering, and realizes the gathering of the fact data mainly by an up-redundancy data mode at present, but when the hierarchical relationship is deeper, a large amount of redundancy data exists, which causes the waste of a magnetic disk and is inconvenient for data maintenance, and simultaneously, as each piece of fact data is generated, the fact data needs to be stored in sequence according to the hierarchical structure, only the up-gathering can be realized during the data query, and the flexible control cannot be realized.
The inventor finds that the main reason of the above problems is that when existing data are summarized upwards, the existing data need to be stored sequentially according to a hierarchical structure, so that fact data and redundant data cannot be distinguished, and a data processing mode cannot be flexibly controlled. The invention provides a statistical method for upward summarization of hierarchical data, which comprises the steps of receiving an operation instruction, obtaining a hierarchical label and a hierarchical data relation table according to the operation instruction, determining a corresponding fact data table according to the operation instruction, the hierarchical label and the hierarchical data relation table, realizing data processing by adopting a mode of processing the fact data table, obtaining to-be-processed fact data according to the fact data table, wherein the to-be-processed fact data comprises a plurality of fields, judging whether the fields need to be summarized upwards according to a preset requirement, avoiding generating excessive redundant data, reducing the occupation of a disk, associating the fact data table with the hierarchical data table when the fields need to be summarized upwards, judging whether hierarchical fields exist in the fields according to the hierarchical data table, splitting the hierarchical fields when the hierarchical fields exist, obtaining unit fields, and performing statistics again on the fact data to be processed according to the unit fields, and performing upward summarization through the hierarchical data relation table, so that data processing can be performed in an upward summarization mode, and flexible control of the data processing mode is realized.
In order to facilitate subsequent operations, before receiving an operation instruction, the fact data and the hierarchy data need to be stored in the fact data table and the hierarchy data table respectively, the hierarchy relationship in the hierarchy data table needs to be stored in the hierarchy data relationship table, and the fact data table is directly processed subsequently, so that operations such as maintenance and query of data are facilitated.
When the fact data to be processed is obtained, the associated bullet is obtained from the fact data table according to the operation instruction, the corresponding hierarchical data table is determined from the hierarchical data relation table according to the hierarchical label, the corresponding fact data table is determined according to the hierarchical data table, and finally the associated inquiry is carried out in the fact data table through the associated field to obtain the fact data to be processed, so that the subsequent judgment on the fields in the fact data and the requirement of summarizing are facilitated.
When the fields do not need to be summarized upwards, the fact data to be processed is counted according to the fields, namely the fact data to be processed directly is counted again in the fact data table, so that the data are convenient to maintain, query and the like, upward redundant data are avoided, and the memory occupation is reduced.
When the fields need to be summarized upwards but hierarchical fields do not exist, the fact data to be processed are counted according to the fields, and are summarized upwards through the corresponding hierarchical data relation table, so that data processing is realized conveniently in an upwards summarizing mode.
When a plurality of fields need to be gathered upwards and hierarchical fields exist, the hierarchical label type is obtained according to the hierarchical fields, the associated fact data table is obtained through inquiry according to the hierarchical label type, the field information is inquired in the associated fact data table according to the associated fields, the hierarchical fields are split, the unit fields are obtained, the corresponding unit field information is obtained according to the unit fields, the new data table is built according to the unit field information and the unit fields, the fact data to be processed is counted again, and the data are gathered upwards according to the new data table based on the hierarchical data relation table, so that the data processing is realized in a data-up gathering mode, and the data processing mode is flexibly controlled.
Having described the general concept of the present invention, the present invention will be further described in detail with reference to the following detailed description and accompanying drawings in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In one embodiment, as shown in fig. 1, a statistical method for level data up-summarization is provided, which includes the following steps:
and S101, receiving an operation instruction, and acquiring a hierarchy label and a hierarchy data relation table according to the operation instruction.
Specifically, an operation instruction is received, wherein the operation instruction can be a data maintenance instruction, a data statistics instruction and the like, and a hierarchy label and a hierarchy data relation table are obtained according to the operation instruction.
The hierarchical label is used for indicating a level corresponding to the hierarchical data, for example, the data A, B, C exists, B is subdata of a, and C is subdata of B, and A, B and C are labeled by the hierarchical label, so that the data can be conveniently searched, and the hierarchical relationship of the data is determined. The hierarchical data relationship table is shown in table 3, and includes fields such as hierarchical tag types, hierarchical data tables, associated fields, and fields and arrays of the hierarchical data tables.
TABLE 3 hierarchical data relationship Table
And S102, determining a corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, and acquiring to-be-processed fact data according to the fact data table, wherein the to-be-processed fact data comprises a plurality of fields.
Specifically, when the operation instruction is data maintenance, determining data to be maintained according to the operation instruction, and acquiring a hierarchy label and a hierarchy data relation table corresponding to the data to be maintained; searching corresponding hierarchy data according to the hierarchy label, determining a hierarchy data table according to the hierarchy data and a corresponding hierarchy data relation table, and determining a corresponding fact data table according to the hierarchy data table; extracting the associated fields in the hierarchical data according to the operation instruction, performing associated query in the fact data table according to the associated fields, and acquiring the fact data to be maintained, wherein the fact data to be maintained comprises a plurality of fields, and the fields can be unit fields and/or hierarchical fields. When the operation instruction is data statistics, the fact data to be counted is acquired.
And step S103, judging whether the plurality of fields need to be summarized upwards according to preset requirements.
Specifically, when an operation instruction is sent out or a table is created, a preset requirement whether corresponding data needs to be reported upwards is set, after fact data to be processed is obtained, whether the fact data to be processed needs to be reported upwards is judged, if yes, the fact data table is associated with a hierarchical data table, the fact data to be processed is processed, it is determined that all fields of data statistics are unit fields, the fact data to be processed are counted according to the unit fields, and finally, upwards summary is carried out based on the hierarchical data relation, so that the operation instruction is completed; if not, counting the data to be counted according to the fields to complete the operation instruction, so that the data can be processed in a mode of processing or upwards summarizing the fact data table, and the data processing mode can be flexibly controlled.
And step S104, if upward summarization is needed, associating the fact data table with the level data table, and judging whether level fields exist in the fields according to the level data table.
Specifically, when a plurality of fields in the fact data to be processed need to be summarized upwards, the fact data table is associated with the corresponding hierarchical data table, whether hierarchical fields exist in the fields is judged according to the hierarchical data table, and the hierarchical fields are fields which include at least two unit fields. The fact data and the hierarchy data are respectively stored, so that data query can be realized by directly querying the fact data table, data distinction is facilitated, only the fact data table can be maintained when the fact data does not need to be summarized upwards, maintenance operation is simple, and data expansion is facilitated; when fact data need upwards gather, the level data table is associated, upwards gather is carried out on the basis of the level data relation table, data processing is conveniently achieved through the upwards gather mode, and data maintenance efficiency is improved.
It should be noted that there may be more than one hierarchical data table corresponding to the fact data table, which is used to represent the unit fields corresponding to several fields in the fact data table.
And step S105, if the hierarchical field exists, splitting the hierarchical field, acquiring the unit field, performing statistics again on the fact data to be processed according to the unit field, and performing upward summarization through a hierarchical data relation table.
Specifically, when a hierarchical field exists in a plurality of fields, the hierarchical field is divided into unit fields, the fact data to be processed is re-counted according to the unit fields, upward summarization is performed according to the re-counted data based on a hierarchical data relation table, and data processing is conveniently achieved in an upward summarization mode.
In the embodiment, by receiving an operation instruction, obtaining a hierarchy label and a hierarchy data relation table according to the operation instruction, and determining a corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, data processing can be realized by adopting a method of processing the fact data table, obtaining fact data to be processed according to the fact data table, wherein the fact data to be processed comprises a plurality of fields, judging whether the fields need to be summarized upwards according to a preset requirement, avoiding generating excessive redundant data, reducing disk occupation, associating the fact data table with the hierarchy data table when the fields need to be summarized upwards, judging whether hierarchy fields exist in the fields according to the hierarchy data table, splitting the hierarchy fields when the hierarchy fields exist, obtaining unit fields, and performing statistics again according to the unit fields, therefore, the fact data can be processed in a mode of directly processing the fact data table or in an upward summarizing mode, and flexible control of data processing is achieved.
Before step S101, the method further includes: creating a first table and a second table, storing the fact data and the hierarchy data in the first table and the second table respectively, and acquiring a fact data table and a hierarchy data table; creating a third table, storing the hierarchical relationship in the hierarchical data table in the third table, and acquiring the hierarchical data relationship table, wherein fields of the hierarchical data relationship table comprise hierarchical label types, the hierarchical data table, associated fields, and fields and arrays of the hierarchical data table.
Specifically, before receiving an operation instruction, by creating a first table and a second table, storing fact data and hierarchy data respectively, obtaining a fact data table and a hierarchy data table, wherein the hierarchy data table corresponds to one fact table, the fact table may correspond to a plurality of hierarchy data tables, and the relationship between fields in the hierarchy data table is reflected through the hierarchy data tables. And storing the hierarchical relationship in the hierarchical data table by creating a third table, acquiring the hierarchical data relationship table, and reflecting the hierarchy among the hierarchical data according to the hierarchical data relationship table. The fields of the hierarchical data relationship table comprise hierarchical label types, hierarchical data tables, association fields, fields of the hierarchical data tables and arrays. The fact data and the hierarchy data are stored in different tables respectively, independent maintenance or query can be conducted on the fact data and the hierarchy data, when the data need to be gathered upwards, data are gathered upwards through the related hierarchy data tables, and flexible control of data processing is achieved.
Wherein, step S102 specifically includes: acquiring an associated field in the fact data table according to the operation instruction; determining a corresponding hierarchical data table in a hierarchical data relation table according to the hierarchical label; determining a corresponding fact data table according to the hierarchy data table; and performing association query in the fact data table according to the association fields to acquire the fact data to be processed.
Specifically, the operation instruction includes instruction information, for example, when the maintenance city relationship is instruction information, the city is a keyword, and corresponding associated fields, such as province, district, etc., are obtained in the fact data table through the keyword city; assuming that the hierarchy label types are 1 type and 2 type, determining that the corresponding hierarchy label is 2 type according to the hierarchy label, and searching a hierarchy data table corresponding to the 2 type, such as a city information table, in the hierarchy data table; because the hierarchy data table corresponds to one fact data table, the corresponding fact data table can be obtained according to the hierarchy data table; and performing association query in the fact data table according to the association fields to acquire the fact data to be processed.
Wherein, step S103 further includes: and when the fields do not need to be summarized upwards, counting the fact data to be processed according to the fields.
Specifically, when a plurality of fields do not need to be gathered upwards, the fact data to be processed can be counted according to the fields, only the content in the fact data is changed, the statistics does not need to be gathered upwards, data maintenance is facilitated, upward redundant data are avoided, and the occupation of a disk is reduced.
Wherein, step S104 further includes: and when the plurality of fields do not have hierarchical fields, counting the fact data to be processed according to the plurality of fields, and carrying out upward summary through the corresponding hierarchical data relation table.
Specifically, when a plurality of fields need to be counted upwards, but no hierarchical field exists, the fact data table and the hierarchical data table are associated, it is determined that no hierarchical field exists in the field corresponding to the data to be processed, that is, the fact data to be processed can be counted according to the existing field, the hierarchical relationship is determined through the corresponding hierarchical data relationship table, upward summary is performed according to the hierarchical relationship, and therefore data processing can be performed in a data upward summary mode.
Wherein, step S105 specifically includes: acquiring a hierarchy label type according to the hierarchy field, and inquiring to obtain an associated fact data table according to the hierarchy label type; inquiring field information in an association fact data table according to the association field; splitting the hierarchy field, acquiring a unit field, and acquiring corresponding unit field information according to the unit field; and constructing a new data table according to the unit field and the unit field information, performing statistics again on the fact data to be processed, and performing upward summary according to the new data table based on the hierarchical data relation table.
Specifically, the hierarchical label types of the corresponding fields are obtained according to the hierarchical fields, and associated fact data tables are obtained in a hierarchical data relation table according to the hierarchical label types, namely a plurality of fact data tables corresponding to the hierarchical label types exist at the moment; in the association fact data table, performing association query according to the association fields to acquire field information corresponding to the association fields; splitting the hierarchical field based on the hierarchical data table to obtain a unit field, and obtaining corresponding unit field information in the associated fact data table according to the unit field to realize the correspondence between the unit field and the unit field information; and constructing a new data table according to the unit fields and the unit field information, realizing the re-statistics of the fact data to be processed, acquiring the hierarchical relationship based on the hierarchical data relationship table, and adopting the new data table to perform upward summarization according to the hierarchical relationship, thereby facilitating the data processing through a data upward summarization mode.
As shown in fig. 2, a statistical system 20 for level data up-aggregation is provided, comprising: an operation instruction receiving module 21, a fact data acquiring module 22, an upward summary judging module 23, a hierarchy field judging module 24 and a data summary counting module 25; wherein:
the operation instruction receiving module 21 is configured to receive an operation instruction, and obtain a hierarchy label and a hierarchy data relationship table according to the operation instruction;
the fact data acquisition module 22 is configured to determine a corresponding fact data table according to the operation instruction, the hierarchical label, and the hierarchical data relationship table, and acquire fact data to be processed according to the fact data table, where the fact data to be processed includes a plurality of fields;
the upward summarizing judging module 23 is used for judging whether the fields need to be summarized upwards according to preset requirements;
the hierarchical field judging module 24 is configured to, when the fields need to be summarized upwards, associate the fact data table with a hierarchical data table, and judge whether a hierarchical field exists in the fields according to the hierarchical data table;
and the data summarizing and counting module 25 is configured to split the hierarchical field when the hierarchical field exists, acquire the unit field, perform statistics again on the fact data to be processed according to the unit field, and perform upward summarization through a hierarchical data relationship table.
In one embodiment, the fact data obtaining module 22 is specifically configured to: acquiring an associated field in the fact data table according to the operation instruction; determining a corresponding hierarchical data table in a hierarchical data relation table according to the hierarchical label; determining a corresponding fact data table according to the hierarchy data table; and performing association query in the fact data table according to the association fields to acquire the fact data to be processed.
In one embodiment, the upward summary judgment module 23 is further configured to: and when the fields do not need to be summarized upwards, counting the fact data to be processed according to the fields.
In one embodiment, the hierarchy field determination module 24 is further configured to: and when the plurality of fields do not have hierarchical fields, counting the fact data to be processed according to the plurality of fields, and carrying out upward summary through the corresponding hierarchical data relation table.
In one embodiment, the data summary statistics module 25 is specifically configured to: acquiring a hierarchy tag type according to the hierarchy field, and inquiring to obtain an associated fact data table according to the hierarchy tag type; inquiring field information in an associated fact data table according to the associated field; splitting the hierarchy field, acquiring a unit field, and acquiring corresponding unit field information according to the unit field; and constructing a new data table according to the unit field and the unit field information, performing statistics again on the fact data to be processed, and performing upward summary according to the new data table based on the hierarchical data relation table.
In one embodiment, a device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 3. The device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the device is configured to provide computing and control capabilities. The memory of the device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the device is used for storing configuration templates and also for storing target web page data. The network interface of the device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a statistical method of level data rollup.
Those skilled in the art will appreciate that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation on the devices to which the present application may be applied, and that a particular device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a medium may be further provided, the medium storing a computer program, the computer program comprising program instructions, which when executed by a computer, which may be part of one of the above-mentioned statistical systems for level data summarization up, cause the computer to perform the method according to the previous embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented in a general purpose computing device, they may be centralized in a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented in program code executable by a computing device, such that they may be stored on a computer storage medium (ROM/RAM, magnetic disk, optical disk) for execution by a computing device, and in some cases, the steps shown or described may be performed in an order different from that described herein, or they may be separately fabricated as individual integrated circuit modules, or multiple ones of them may be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The foregoing is a more detailed description of the present invention that is presented in conjunction with specific embodiments, and the practice of the invention is not to be considered limited to those descriptions. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (9)
1. A statistical method for upward summarization of hierarchical data is characterized by comprising the following steps:
receiving an operation instruction, and acquiring a hierarchy label and a hierarchy data relation table according to the operation instruction;
determining a corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, and acquiring fact data to be processed according to the fact data table, wherein the fact data to be processed comprises a plurality of fields;
judging whether the fields need to be summarized upwards according to preset requirements;
if the fields need to be summarized upwards, associating the fact data table with a hierarchical data table, and judging whether hierarchical fields exist in the fields according to the hierarchical data table;
and if the hierarchical field exists, splitting the hierarchical field, acquiring a unit field, performing statistics again on the fact data to be processed according to the unit field, and performing upward summary through the hierarchical data relation table.
2. The statistical method for upward summarization of hierarchical data according to claim 1, further comprising, before the receiving an instruction, according to the received operation instruction, and according to the operation instruction, obtaining a hierarchical label and a hierarchical data relationship table, further comprising:
creating a first table and a second table, storing the fact data and the hierarchy data in the first table and the second table respectively, and acquiring a fact data table and a hierarchy data table;
creating a third table, storing the hierarchical relationship in a hierarchical data table in the third table, and acquiring the hierarchical data relationship table, wherein fields of the hierarchical data relationship table comprise hierarchical label types, the hierarchical data table, associated fields, and fields and arrays of the hierarchical data table.
3. The statistical method for upward summarization of hierarchical data according to claim 2, wherein the determining a corresponding fact data table according to the operation instruction, the hierarchical label and the hierarchical data relationship table, and acquiring the fact data to be processed according to the fact data table specifically comprises:
acquiring an associated field in the fact data table according to the operation instruction;
determining a corresponding hierarchical data table in the hierarchical data relation table according to the hierarchical label;
determining a corresponding fact data table according to the hierarchy data table;
and performing association query in the fact data table according to the association field to acquire the fact data to be processed.
4. The method of claim 1, wherein the determining whether the plurality of fields need to be summarized upwards according to a preset requirement further comprises:
and when the fields do not need to be summarized upwards, counting the fact data to be processed according to the fields.
5. The statistical method for hierarchical data summarization according to claim 1, wherein if the fields need to be summarized upwards, associating the fact data table with a hierarchical data table, and determining whether a hierarchical field exists in the fields according to the hierarchical data table, further comprising:
and when no hierarchical field exists in the fields, counting the fact data to be processed according to the fields, and carrying out upward summary through a corresponding hierarchical data relation table.
6. The method according to claim 3, wherein if there is a hierarchical field, splitting the hierarchical field, obtaining a unit field, performing statistics again on the fact data to be processed according to the unit field, and performing upward summarization through the hierarchical data relationship table specifically includes:
acquiring a hierarchy label type according to the hierarchy field, and inquiring to obtain an associated fact data table according to the hierarchy label type;
inquiring field information in the associated fact data table according to the associated field;
splitting the hierarchical field, acquiring a unit field, and acquiring corresponding unit field information according to the unit field;
and constructing a new data table according to the unit field and the unit field information, performing statistics again on the fact data to be processed, and performing upward summary according to the new data table based on the hierarchical data relation table.
7. A statistical system for upward summarization of hierarchical data, comprising:
the operation instruction receiving module is used for receiving an operation instruction and acquiring a hierarchy label and a hierarchy data relation table according to the operation instruction;
the fact data acquisition module is used for determining a corresponding fact data table according to the operation instruction, the hierarchy label and the hierarchy data relation table, and acquiring fact data to be processed according to the fact data table, wherein the fact data to be processed comprises a plurality of fields;
the upward gathering judgment module is used for judging whether the fields need to be gathered upward according to a preset requirement;
the hierarchical field judging module is used for associating the fact data table with a hierarchical data table when the fields need to be summarized upwards, and judging whether hierarchical fields exist in the fields or not according to the hierarchical data table;
and the data summarizing and counting module is used for splitting the hierarchy field when the hierarchy field exists, acquiring the unit field, carrying out statistics again on the fact data to be processed according to the unit field, and carrying out upward summarization through the hierarchy data relation table.
8. An apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 6 are implemented when the computer program is executed by the processor.
9. A medium on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210168216.3A CN114510488A (en) | 2022-02-23 | 2022-02-23 | Statistical method, system, equipment and medium for upward summarization of layer data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210168216.3A CN114510488A (en) | 2022-02-23 | 2022-02-23 | Statistical method, system, equipment and medium for upward summarization of layer data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114510488A true CN114510488A (en) | 2022-05-17 |
Family
ID=81553447
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210168216.3A Pending CN114510488A (en) | 2022-02-23 | 2022-02-23 | Statistical method, system, equipment and medium for upward summarization of layer data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114510488A (en) |
-
2022
- 2022-02-23 CN CN202210168216.3A patent/CN114510488A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321344B (en) | Information query method and device for associated data, computer equipment and storage medium | |
| CN102402605B (en) | Mixed distribution model for search engine indexing | |
| US8112421B2 (en) | Query selection for effectively learning ranking functions | |
| CN102725755B (en) | Method and system of file access | |
| CN105550225A (en) | Index construction method and query method and apparatus | |
| CN104063376A (en) | Multi-dimensional grouping operation method and system | |
| CN112612827B (en) | Database paging query method and device, computer equipment and storage medium | |
| CN110837520A (en) | Data processing method, platform and system | |
| CN102024046A (en) | Data repeatability checking method and device as well as system | |
| WO2022267457A1 (en) | Method, apparatus, and device for processing spreadsheet data, and storage medium | |
| US10990573B2 (en) | Fast index creation system for cloud big data database | |
| CN109299101B (en) | Data retrieval method, device, server and storage medium | |
| US10482087B2 (en) | Storage system and method of operating the same | |
| CN111125199B (en) | Database access method and device and electronic equipment | |
| US20230020618A1 (en) | Spreadsheet Flat Data Extractor | |
| CN116719822B (en) | Method and system for storing massive structured data | |
| CN117609181A (en) | Method and system for migrating TCHouse database | |
| US7672925B2 (en) | Accelerating queries using temporary enumeration representation | |
| CN114510488A (en) | Statistical method, system, equipment and medium for upward summarization of layer data | |
| US20080162441A1 (en) | Accelerating queries using secondary semantic column enumeration | |
| CN101405727B (en) | Management of statistical views in a database system | |
| CN113849520B (en) | Intelligent recognition method and device for abnormal SQL, electronic equipment and storage medium | |
| CN112711627B (en) | Data importing method, device and equipment of Greemplum database | |
| WO2008085359A1 (en) | Accelerating queries using delayed value projection of enumerated storage | |
| CN114020446A (en) | Cross-multi-engine routing processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |