CN105956071A - Memory optimization method and memory optimization device for OLAP aggregation operation - Google Patents

Memory optimization method and memory optimization device for OLAP aggregation operation Download PDF

Info

Publication number
CN105956071A
CN105956071A CN201610279001.3A CN201610279001A CN105956071A CN 105956071 A CN105956071 A CN 105956071A CN 201610279001 A CN201610279001 A CN 201610279001A CN 105956071 A CN105956071 A CN 105956071A
Authority
CN
China
Prior art keywords
key
coding
olap
initial data
index
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610279001.3A
Other languages
Chinese (zh)
Inventor
杨海乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LeTV Holding Beijing Co Ltd
LeTV Information Technology Beijing Co Ltd
Original Assignee
LeTV Holding Beijing Co Ltd
LeTV Information Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LeTV Holding Beijing Co Ltd, LeTV Information Technology Beijing Co Ltd filed Critical LeTV Holding Beijing Co Ltd
Priority to CN201610279001.3A priority Critical patent/CN105956071A/en
Publication of CN105956071A publication Critical patent/CN105956071A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation
    • G06F16/24534Query rewriting; Transformation
    • G06F16/24549Run-time optimisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a memory optimization method and a memory optimization device for OLAP aggregation operation, and the method comprises the following steps: creating corresponding index codes for keys in original data according to a preset coding mode; storing the keys in the original data and the corresponding index codes into a key value table of a non-relational database, wherein the index codes are stored as keys of the key value table, and the keys in the original data are stored as values of the key value table; when performing OLAP aggregation operation, reading the index codes from the key value table to a memory to complete the OLAP aggregation operation. During the operation process, the index codes substitute the keys in the original data, and thus, memory usage during the OLAP aggregation operation process can be reduced.

Description

Memory Optimize Method during OLAP aminated polyepichlorohydrin and device
Technical field
The invention belongs to field of computer technology, specifically, when relating to a kind of OLAP aminated polyepichlorohydrin Memory Optimize Method and device.
Background technology
On-line analytical processing (On-Line Analytical Processing, OLAP) is data warehouse Main application, support complicated analysis operation, stress decision support, and visual and understandable looking into is provided Ask result.For OLAP system, time most, data base is the most running is form operation, Perform the SQL operation of substantially polymeric type, such as packet (group by).
The aminated polyepichlorohydrin of OLAP, such as summation (Sum), maximum (Max), minima (Min), It is averaged (Ave) etc., it usually needs calculate the metric corresponding to each key (Key).At a report In table, each push-to is often made up of the attribute of multiple dimensions, such as name (name), address (adress) Etc..When calculating, the various combination between each dimension is utilized to calculate the degree of correspondence respectively as key Value, then, key to be put into time internal memory carries out computing, each dimension the key combined is all Character string (String) type, takies memory headroom relatively big, and dimension is the highest, shared Memory headroom is the biggest.
Summary of the invention
In view of this, Memory Optimize Method during a kind of OLAP aminated polyepichlorohydrin is embodiments provided And device, ask in order to solve the technology that in prior art, during OLAP aminated polyepichlorohydrin, EMS memory occupation is excessive Topic.
In order to solve above-mentioned technical problem, internal memory when the invention discloses a kind of OLAP aminated polyepichlorohydrin is excellent Change method, including: according to pre-arranged code mode, the key in initial data is created corresponding index and encode; The index storing the key in described initial data and correspondence in the key assignments table of non-relational database is compiled Code, described index coding is saved as the key of described key assignments table, and the key in described initial data is saved as The value of described key assignments table;When carrying out OLAP aminated polyepichlorohydrin, from described key assignments table, described index is compiled Code reads and completes OLAP aminated polyepichlorohydrin in internal memory.
In order to solve above-mentioned technical problem, internal memory when the invention discloses a kind of OLAP aminated polyepichlorohydrin is excellent Gasifying device, including: coding module, right for the key in initial data being created according to pre-arranged code mode The index coding answered;Mapping block, described original for storing in the key assignments table of non-relational database Key in data and the index coding of correspondence, described index coding is saved as the key of described key assignments table, Key in described initial data is saved as the value of described key assignments table;Computing module, for when carrying out OLAP During aminated polyepichlorohydrin, from described key assignments table, described index coding is read and internal memory completes OLAP gather Close computing.
Compared with prior art, the internal memory optimization side during OLAP aminated polyepichlorohydrin that the embodiment of the present invention provides Method and device, set up index coding for the key in initial data, encoded by index in calculating process Instead of the key in initial data, it is possible to reduce the EMS memory occupation during OLAP aminated polyepichlorohydrin.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to reality Execute the required accompanying drawing used in example or description of the prior art to be briefly described, it should be apparent that under, Accompanying drawing during face describes is some embodiments of the present invention, for those of ordinary skill in the art, On the premise of not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the system architecture schematic diagram of the embodiment of the present invention;
Fig. 2 is the schematic diagram of the dictionary tree that the embodiment of the present invention creates in internal memory;
Fig. 3 is the schematic diagram of the dictionary tree that the embodiment of the present invention creates in internal memory;
Fig. 4 is the schematic diagram of the dictionary tree that the embodiment of the present invention creates in internal memory;
Fig. 5 is the stream of Memory Optimize Method during a kind of OLAP aminated polyepichlorohydrin of embodiment of the present invention offer Cheng Tu;
Fig. 6 is the stream of Memory Optimize Method during a kind of OLAP aminated polyepichlorohydrin of embodiment of the present invention offer Cheng Tu;
Fig. 7 is the frame of internal memory optimization device during a kind of OLAP aminated polyepichlorohydrin of embodiment of the present invention offer Figure.
Detailed description of the invention
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearer, below in conjunction with this Accompanying drawing in bright embodiment, is clearly and completely described the technical scheme in the embodiment of the present invention, Obviously, described embodiment is a part of embodiment of the present invention rather than whole embodiments.Based on Embodiment in the present invention, those of ordinary skill in the art are obtained under not making creative work premise The every other embodiment obtained, broadly falls into the scope of protection of the invention.
Key (Key) in the initial data that OLAP aminated polyepichlorohydrin is used by the embodiment of the present invention is set up Index coding, and records in the key assignments table of non-relational database (nosql), index is encoded as Key in key assignments table, by the key in initial data as the value in key assignments table, carries out OLAP aminated polyepichlorohydrin Time, the index coding in key assignments table is read and internal memory completes aminated polyepichlorohydrin, by index coding generation Replace the key in initial data, thus reduce the EMS memory occupation in calculating process.Further, it is also possible to Internal memory sets up corresponding dictionary tree according to the coding that indexes read, further reduces and need maintenance Internal storage data amount.
Fig. 1 is the system architecture schematic diagram of the embodiment of the present invention, including distributed memory system 10 and end End equipment 11.Distributed memory system 10 can use HBase, and (Hadoop Database, based on Hadoop The data base of system architecture), all data files are stored in HDFS (Hadoop distributed document System, Hadoop Distributed File System) in file system, including substantial amounts of Key-Value Data.Terminal unit 10 can be computer equipment or the service equipment of other offers OLAP.
In the Key-Value data preserved in distributed memory system 10, key (Key) is various dimensions (Dimension) combination, such as, the dimension of the data such as " time, place, product " collectively constitutes one Individual key;Value (Value) is the value under corresponding key (dimension combination), such as, " price, sales volume " Deng.These Key-Value data may be respectively stored in the different clusters of distributed memory system, also It is that terminal unit 11 carries out the initial data that OLAP aminated polyepichlorohydrin is to be used.
In the embodiment of the present invention, in the meta data server of distributed memory system 10, it is possible to use Non-relational database (nosql) creates a concordance list, and this concordance list is a key assignments (Key-Value) Table.In this key assignments table, by the key of the initial data of preservation in distributed memory system 10 as key The value of value table, creates corresponding index coding according to the key that pre-arranged code mode is initial data, and should Index coding is as key corresponding in key assignments table.Such as, by initial data by " time, place, produce Product " etc. the key " in January, 2016, Shanghai, intelligent television " that formed of dimension be stored as the value of key assignments table, According to pre-arranged code mode create index coding " 121 ", will " 121 " as in key assignments table be worth " 2016 Year January, Shanghai, intelligent television " corresponding key.So, by the key of this key assignments table, (index is compiled Code) just can navigate to the tolerance (such as " price, sales volume " etc.) of the key of initial data and correspondence.
Pre-arranged code mode includes following several ways.
(1) ten's digit coding is used
Each in coding can use the numeral of 0-9, if coding figure place is three, then and can generation Table 10+102+103Plant different dimension combinations.Numeral 0-9 data type can use integer, relative to character The memory headroom that type takies is less.
(2) binary digital encoding of boolean type is used
In this coding, the numeral of use is all the 0 and 1 of boolean type, and each only takes up one Individual byte, best to the effect of optimization of internal memory.But owing to each can only represent two kinds of situations, work as number According to dimension the highest, various dimension combination quantity the biggest time, the figure place of coding can become considerably long.When original Data volume less (time within such as 100GB), it may be preferred to use this coded system, make internal memory excellent Change better.
(3) two hexadecimal character codings are used
Each in coding can use the character of A-Z, and each can represent 26 kinds of feelings Shape, if coding figure place is 3, then can represent 26+262+263Plant different dimension combinations.When original Data volume is very big (such as during more than 10TB), it may be preferred to consider this coded system.
(1st) kind coded system is preferably used in the embodiment of the present invention, both can reach good internal memory Effect of optimization, it is also possible to deal with more original data volume.
When terminal unit 11 carries out OLAP aminated polyepichlorohydrin, from this key assignments table, index coding is read Get in internal memory, and utilize this index to encode OLAP aminated polyepichlorohydrin.Now, it is no longer necessary to will The key of initial data reads in internal memory, owing to the key of initial data is typically character string type, therefore passes through Index coding instead of the key in initial data, it is possible to reduces the internal memory during OLAP aminated polyepichlorohydrin and accounts for With.
For further memory optimization space, the index in key assignments table is being encoded reading by terminal unit 11 Time in internal memory, in internal memory, build dictionary tree according to the pre-arranged code mode of index coding.
If index coding employing (1st) plant coded system, then the dictionary tree created as in figure 2 it is shown, The next stage node of each node can include these ten child nodes of 0-9.If index coding employing the (2) plant coded system, then the dictionary tree created is as it is shown on figure 3, the next stage node of each node 0 and 1 the two child node can be included.If index coding employing (3rd) plants coded system, then Create dictionary tree as shown in Figure 4, the next stage node of each node can include A-Z this 26 Individual child node.
When carrying out OLAP aminated polyepichlorohydrin, the index read in key assignments table encodes the word in audit memory again Allusion quotation tree completes OLAP aminated polyepichlorohydrin.
After building dictionary tree, for the identical coding of one and same coding position in index coding, can be the most logical Cross a node just can safeguard, such as, index coding " 121 " and " 122 ", front two " 1 " and " 2 " are identical, by a first nodes " 1 " and a two-level node " 2 ", it is possible to tie up simultaneously Protect the front two of the two index coding, thus have compressed internal storage data amount further, reduce internal memory and open Pin, makes memory headroom be optimized further.
When utilizing non-relational database (nosql) to create concordance list, it is also possible to create one more simultaneously Reverse indexing table.In this reverse indexing table, by the key of initial data as the key in reverse indexing table, By index coding as the value in reverse indexing table.So, index coded query aminated polyepichlorohydrin is being utilized During result, it is possible to navigated to the key of initial data by reverse indexing table.
Based on discussed above, embodiments provide internal memory optimization during a kind of OLAP aminated polyepichlorohydrin Method, as it is shown in figure 5, include:
S20, creates corresponding index according to pre-arranged code mode to the key in initial data and encodes;
S21, stores the key in initial data and the rope of correspondence in the key assignments table of non-relational database Drawing coding, index coding is saved as the key of key assignments table, and the key in initial data is saved as key assignments table Value;
S22, when carrying out OLAP aminated polyepichlorohydrin, reads index coding in internal memory from key assignments table Complete OLAP aminated polyepichlorohydrin.
The key of initial data is typically character string type, instead of the key in initial data by index coding, The EMS memory occupation during OLAP aminated polyepichlorohydrin can be reduced.
In one embodiment, as shown in Figure 6, step S22 is further comprising the steps.
S221, creates the dictionary tree corresponding with pre-arranged code mode in internal memory;
S222, when carrying out OLAP aminated polyepichlorohydrin, reads index coding and queries dictionary tree completes OLAP aminated polyepichlorohydrin.
By setting up dictionary tree compression memory data volume further in internal memory, memory headroom is enable to obtain To optimizing further.
In the above embodiment of the present invention, step S20 may further include:
Utilize ten's digit coding that the key in initial data creates corresponding index coding.Can reach Good internal memory optimization effect, copes with again the situation that initial data is more.
This step 20 can further include:
The binary digital encoding utilizing boolean type creates corresponding index to the key in initial data Coding.The binary digital encoding of boolean type each take a byte, can reach more in Deposit effect of optimization.
Additionally, this step 20 can further include:
Utilize two hexadecimal character codings that the key in initial data creates corresponding index coding.Coding In each can have 26 kinds of changes, index effect is more preferable, it is adaptable to original data volume is bigger Or the more situation of combination of data dimension.
In one embodiment, this step 21 is further comprising the steps.
S211, stores in initial data in the key assignments table and reverse indexing table of non-relational database respectively Key and correspondence index coding;In key assignments table, index coding is saved as the key of key assignments table, former Key in beginning data is saved as the value of key assignments table;In reverse indexing table, index coding is saved as value, Key in initial data is saved as key.
When inquiring about the result of this aminated polyepichlorohydrin, can be inquired about by this reverse indexing table, by index Coding navigates to the key of initial data.
Assembly of the invention embodiment is presented herein below, for performing the said method embodiment of the present invention.
Fig. 7 is internal memory optimization device during a kind of OLAP aminated polyepichlorohydrin of embodiment of the present invention offer, its Including:
Coding module 30, for creating corresponding index according to pre-arranged code mode to the key in initial data Coding;
Mapping block 31, for store in the key assignments table of non-relational database key in initial data with And the index coding of correspondence, index coding is saved as the key of key assignments table, and the key in initial data is saved Value for described key assignments table;
Computing module 32, for when carrying out OLAP aminated polyepichlorohydrin, reads index coding from key assignments table Get and internal memory completes OLAP aminated polyepichlorohydrin.
In one embodiment, this computing module 32 farther includes:
Process submodule, for creating the dictionary tree corresponding with pre-arranged code mode in internal memory;
Operator module, for when carrying out OLAP aminated polyepichlorohydrin, reads index coding queries dictionary Set OLAP aminated polyepichlorohydrin.
In the above embodiment of the present invention, this coding module 30 may further include:
First encoding submodule, for utilizing ten's digit coding that the key in initial data is created correspondence Index coding.
This coding module 30 can further include:
Second encoding submodule, for utilizing the binary digital encoding of boolean type to initial data In key create corresponding index coding.
This coding module 30 can further include:
3rd encoding submodule, for utilizing two hexadecimal character codings to create the key in initial data Corresponding index coding.
In one embodiment, mapping block 31 may further include:
Mapping submodule, for storing respectively in the key assignments table and reverse indexing table of non-relational database Key in initial data and the index coding of correspondence;In key assignments table, index coding is saved as key assignments The key of table, the key in initial data is saved as the value of key assignments table;In reverse indexing table, index coding Being saved as value, the key in initial data is saved as key.
Each functional module that internal memory optimization device during above-mentioned OLAP aminated polyepichlorohydrin provides may be located at same Equipment (is such as all located at terminal unit 11 or the Metadata Service of distributed memory system 10 of Fig. 1 Device), or, above-mentioned each functional module can also be positioned at distinct device (such as, coding module 30 He Mapping block 31 is positioned at the meta data server of distributed memory system 10 in Fig. 1, and computing module 32 It is positioned at terminal unit 11).
Come real additionally, the embodiment of the present invention can be passed through hardware processor (hardware processor) Existing each functional module above-mentioned.
Device embodiment described above is only schematically, wherein said illustrates as separating component Unit can be or may not be physically separate, the parts shown as unit can be or Person may not be physical location, i.e. may be located at a place, or can also be distributed to multiple network On unit.Some or all of module therein can be selected according to the actual needs to realize the present embodiment The purpose of scheme.Those of ordinary skill in the art are not in the case of paying performing creative labour, the most permissible Understand and implement.
Through the above description of the embodiments, those skilled in the art is it can be understood that arrive each reality The mode of executing can add the mode of required general hardware platform by software and realize, naturally it is also possible to by firmly Part.Based on such understanding, the portion that prior art is contributed by technique scheme the most in other words Dividing and can embody with the form of software product, this computer software product can be stored in computer can Read in storage medium, such as ROM/RAM, magnetic disc, CD etc., including some instructions with so that one Computer equipment (can be personal computer, server, or the network equipment etc.) performs each to be implemented The method described in some part of example or embodiment.
Last it is noted that above example is only in order to illustrate technical scheme, rather than to it Limit;Although the present invention being described in detail with reference to previous embodiment, the ordinary skill of this area Personnel it is understood that the technical scheme described in foregoing embodiments still can be modified by it, or Person carries out equivalent to wherein portion of techniques feature;And these amendments or replacement, do not make corresponding skill The essence of art scheme departs from the spirit and scope of various embodiments of the present invention technical scheme.

Claims (10)

1. a Memory Optimize Method during OLAP aminated polyepichlorohydrin, it is characterised in that including:
According to pre-arranged code mode, the key in initial data is created corresponding index to encode;
The rope of the key in described initial data and correspondence is stored in the key assignments table of non-relational database Drawing coding, described index coding is saved as the key of described key assignments table, and the key in described initial data is protected Save as the value of described key assignments table;
When carrying out OLAP aminated polyepichlorohydrin, from described key assignments table, described index coding is read internal memory In complete OLAP aminated polyepichlorohydrin.
Method the most according to claim 1, it is characterised in that described when carrying out OLAP polymerization During computing, from described key assignments table, described index coding is read and internal memory completes OLAP polymerization fortune Including:
The dictionary tree corresponding with described pre-arranged code mode is created in internal memory;
When carrying out OLAP aminated polyepichlorohydrin, read described index coding and inquire about described dictionary tree and complete OLAP aminated polyepichlorohydrin.
Method the most according to claim 1, it is characterised in that described according to pre-arranged code mode Key in initial data creates corresponding index coding include:
Utilize ten's digit coding that the key in initial data creates corresponding index coding.
Method the most according to claim 1, it is characterised in that described according to pre-arranged code mode Key in initial data creates corresponding index coding include:
The binary digital encoding utilizing boolean type creates corresponding index to the key in initial data Coding.
Method the most according to claim 1, it is characterised in that described according to pre-arranged code mode Key in initial data creates corresponding index coding include:
Utilize two hexadecimal character codings that the key in initial data creates corresponding index coding.
6. an internal memory optimization device during OLAP aminated polyepichlorohydrin, it is characterised in that including:
Coding module, compiles for the key in initial data being created corresponding index according to pre-arranged code mode Code;
Mapping block, for storing the key in described initial data in the key assignments table of non-relational database And the index coding of correspondence, described index coding is saved as the key of described key assignments table, described original number Key according to is saved as the value of described key assignments table;
Computing module, for when carrying out OLAP aminated polyepichlorohydrin, by described index from described key assignments table Coding reads and completes OLAP aminated polyepichlorohydrin in internal memory.
Device the most according to claim 6, it is characterised in that described computing module includes:
Process submodule, for creating the dictionary tree corresponding with described pre-arranged code mode in internal memory;
Operator module, for when carrying out OLAP aminated polyepichlorohydrin, reads described index coding and inquires about Described dictionary tree completes OLAP aminated polyepichlorohydrin.
Device the most according to claim 6, it is characterised in that described coding module includes:
First encoding submodule, for utilizing ten's digit coding that the key in initial data is created correspondence Index coding.
Device the most according to claim 6, it is characterised in that described coding module includes:
Second encoding submodule, for utilizing the binary digital encoding of boolean type to initial data In key create corresponding index coding.
Device the most according to claim 6, it is characterised in that described coding module includes:
3rd encoding submodule, for utilizing two hexadecimal character codings to create the key in initial data Corresponding index coding.
CN201610279001.3A 2016-04-28 2016-04-28 Memory optimization method and memory optimization device for OLAP aggregation operation Pending CN105956071A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610279001.3A CN105956071A (en) 2016-04-28 2016-04-28 Memory optimization method and memory optimization device for OLAP aggregation operation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610279001.3A CN105956071A (en) 2016-04-28 2016-04-28 Memory optimization method and memory optimization device for OLAP aggregation operation

Publications (1)

Publication Number Publication Date
CN105956071A true CN105956071A (en) 2016-09-21

Family

ID=56916768

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610279001.3A Pending CN105956071A (en) 2016-04-28 2016-04-28 Memory optimization method and memory optimization device for OLAP aggregation operation

Country Status (1)

Country Link
CN (1) CN105956071A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982103A (en) * 2012-11-06 2013-03-20 东南大学 On-line analytical processing (OLAP) massive multidimensional data dimension storage method
CN103366015A (en) * 2013-07-31 2013-10-23 东南大学 OLAP (on-line analytical processing) data storage and query method based on Hadoop
CN103425772A (en) * 2013-08-13 2013-12-04 东北大学 Method for searching massive data with multi-dimensional information
CN103942266A (en) * 2014-03-27 2014-07-23 上海巨数信息科技有限公司 Data analysis method capable of achieving self-defining of complex service computational logic on basis of OLAP

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982103A (en) * 2012-11-06 2013-03-20 东南大学 On-line analytical processing (OLAP) massive multidimensional data dimension storage method
CN103366015A (en) * 2013-07-31 2013-10-23 东南大学 OLAP (on-line analytical processing) data storage and query method based on Hadoop
CN103425772A (en) * 2013-08-13 2013-12-04 东北大学 Method for searching massive data with multi-dimensional information
CN103942266A (en) * 2014-03-27 2014-07-23 上海巨数信息科技有限公司 Data analysis method capable of achieving self-defining of complex service computational logic on basis of OLAP

Similar Documents

Publication Publication Date Title
KR101806055B1 (en) Generating a multi-column index for relational databases by interleaving data bits for selectivity
CN102053992B (en) Clustering method and system
CN107408114B (en) Identifying join relationships based on transactional access patterns
Stabno et al. RLH: Bitmap compression technique based on run-length and Huffman encoding
CN105631003A (en) Intelligent index establishing, inquiring and maintaining method supporting mass data classification and counting
US9760593B2 (en) Data dictionary with a reduced need for rebuilding
US20220164345A1 (en) Managed query execution platform, and methods thereof
CN105574212B (en) A kind of image search method of more index disk hash data structures
CN109241159B (en) Partition query method and system for data cube and terminal equipment
CN105701200A (en) Data warehouse security OLAP method on memory cloud computing platform
CN102609462A (en) Method for compressed storage of massive SQL (structured query language) by means of extracting SQL models
Su et al. Taming massive distributed datasets: data sampling using bitmap indices
JP3452531B2 (en) Method and system for data mining
CN105144157A (en) System and method for compressing data in database
CN113672230A (en) Industrial element identification coding method
US10726005B2 (en) Virtual split dictionary for search optimization
CN111159180A (en) Data processing method and system based on data resource directory construction
CN108009290A (en) A kind of data modeling and storage method of track traffic command centre gauze big data
CN107644050A (en) A kind of querying method and device of the Hbase based on solr
CN105302915A (en) High-performance data processing system based on memory calculation
CN103345527B (en) Intelligent data statistical system
CN105117447A (en) Processing method and system of XML (Extensive Markup Language) document data
CN114741368A (en) Log data statistical method based on artificial intelligence and related equipment
CN111274454A (en) Spatio-temporal data processing method and device, electronic equipment and storage medium
CN107291935A (en) The CPIR V arest neighbors privacy protection enquiring methods encoded based on Spark and Huffman

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20160921

WD01 Invention patent application deemed withdrawn after publication