CN104317966B - A kind of dynamic index method inquired about for electric power big data Rapid Combination - Google Patents
A kind of dynamic index method inquired about for electric power big data Rapid Combination Download PDFInfo
- Publication number
- CN104317966B CN104317966B CN201410654100.6A CN201410654100A CN104317966B CN 104317966 B CN104317966 B CN 104317966B CN 201410654100 A CN201410654100 A CN 201410654100A CN 104317966 B CN104317966 B CN 104317966B
- Authority
- CN
- China
- Prior art keywords
- index
- data
- big data
- electric power
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 17
- 238000005516 engineering process Methods 0.000 claims abstract description 27
- 239000000203 mixture Substances 0.000 claims abstract description 12
- 238000010586 diagram Methods 0.000 claims abstract description 9
- 238000001914 filtration Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 239000002131 composite material Substances 0.000 abstract description 8
- 230000000750 progressive effect Effects 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000011218 segmentation Effects 0.000 description 5
- 230000005611 electricity Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003034 coal gas Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012772 sequence design Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2264—Multidimensional index structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/328—Management therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of dynamic index method inquired about for electric power big data Rapid Combination, it is characterised in that methods described specifically includes following steps：SS1 utilizes dynamic index diagram technology, is that electric power big data sets up a set of three-dimensional directory system；SS2 creates index using many condition query composition method；SS3 sets up electric power big data Rapid Combination query scheme.The beneficial effect that the present invention is reached：Using dynamic index diagram technology, the foundation and Rapid Combination inquiry of many condition column index are realized, composite index is created by setting up index map for each inquiry is special, it is to avoid the full table progressive scan of carry out, greatly improve the speed of electric power big data query composition.
Description
Technical field
The utility model is related to a kind of dynamic index method inquired about for electric power big data Rapid Combination, belongs to electric power letter
Breathization technical field.
Background technology
With the propulsion of electric power digital process, power system have accumulated substantial amounts of hair, defeated, electricity consumption data.At present
The whole province's power information data that only power information system in Jiangsu Province's is preserved over the years have reached tens TB, how using existing
Big data analytical technology, excavates the potential value of electric power big data, electric power enterprise is provided more preferable service for client, is one
It is worth the problem of research.And 2013《China Power big data develops white paper》Issue, by China electric power big data grind
Study carefully and pushed a new starting point to, have epoch-making meaning to the research and application of China Power big data.
Its feature of electric power big data can be summarized as 3 " V " and 3 " E ", and 3 " V " represent the scale of construction greatly (Volume), and type is more
(Variety) and speed is fast (Velocity), 3 " E " represent data i.e. energy (Energy), data i.e. interaction (Exchange),
Data are common feelings (Empathy).It is such to summarize equally applicable in electricity consumption big data.
Although it is very difficult that efficient index is created on big data basis, but it will be apparent that big data is to index
Demand is more urgent compared to traditional database：Traditional database needs to use rope in the case of hundreds of thousands, millions of data volumes
The query performance for meeting and requiring could be provided by drawing, then be absorbed in processing easily hundreds of hundred million, the big data skill of several hundred billion data volumes
If art does not provide how about index can meet performance requirementThe index of traditional database is all a kind of single index knot in fact
Structure, although many big data products based on Hadoop can support composite index, but this composite index its essence is still
It is that single index, i.e. one query can only use an index, so-called composite index is also simply by multiple field simple concatenations.Single index
Efficiency can meet the inquiry of user's wall scroll part, and traditional composite index is excessively simple due to the technology that it splices, therefore
Also single inquiry can only be supported, if the querying condition of user is more complicated, conditional combination is more flexible, it can not just expire completely
The demand of sufficient user.
Big data solution relatively common at present is Hadoop+HBase, and the solution is by building distribution
Software frame and distributed memory system are handled, distributed storage and the inquiry of big data is realized.HBase is carried out by Rowkey
Sort and store, need to retrieve data block by row when carrying out data query, but inquiry velocity can not far be met in real time
Demand.
The content of the invention
To overcome the defect that prior art is present, above-mentioned technical problem is solved, the present invention is a kind of fast for electric power big data
The dynamic index method of fast query composition.
The present invention is adopted the following technical scheme that：A kind of dynamic index method inquired about for electric power big data Rapid Combination,
Characterized in that, methods described specifically includes following steps：
SS1 utilizes dynamic index diagram technology, is that electric power big data sets up a set of three-dimensional directory system；
SS2 creates index using many condition query composition method；
SS3 sets up electric power big data Rapid Combination query scheme.
Preferably, step SS1 includes：It is ranked up first with first domain, sets up some index starting points, then make
It will be indexed and be segmented with hash technologies, build the three-dimensional index segmented system of a multistage.
Preferably, step SS2 includes：When the combination of user's use condition carries out data query, database engine can foundation
The exclusive mechanism of itself dynamically independently creates the data query that index provides many condition of any combination originally using these；
Preferably, step SS2 also includes：If using without the field and other words for having created index for creating index
Section is combined inquiry, and system intelligently goes to judge first, it is found that several fields therein have been indexed, will be preferentially several using this
Individual field is tentatively judged with filtering, and obtains one group of intermediate queries result；For and do not set up other fields of index, it is necessary to right again
Intermediate result data is scanned one by one.
Preferably, step SS3 specifically includes following steps：
1) user inputs sql command from client；
2）Index data base is connected to by JDBC and HBase；
3）Sql command is parsed, corresponding index file is found from index data base；
4）Index file is trimmed, the dynamic index figure for specific querying command is formed；
5）By dynamic index figure, obtain needing the HFile of inquiry RowKey；
6）HBase according to RowKey from HDFS fetch evidence；
7）Query Result is returned into user.
Preferably, the step 2 in step SS3）Including：When HBase reads in newly-increased data, all data syn-chronizations are sent to
The inquiry specified accelerates server, the statistics of numerical value is carried out to certain field by nominal key and date, and set up inquiry rope
Draw；When user sends inquiry request to HBase, the request is sent to special query engine immediately, is returned according to querying condition
Corresponding index address is returned, initial data, and returning result are found by index address.
The implication of above-mentioned term：DIG（dynamic index graph）That is dynamic index diagram technology.
" hash " is done in Hash, general translation, is exactly the input random length（It is called and does preliminary mapping, pre-image）,
By hashing algorithm, the output of regular length is transformed into, the output is exactly hashed value.
SQL (Structured Query Language) is SQL, is a kind of data base querying and journey
Sequence design language, for accessing data and inquiry, renewal and administrative relationships Database Systems；It is also database script text simultaneously
The extension name of part.
HBase is Hadoop Database, be a high reliability, high-performance, towards row, telescopic distribution
Storage system.
JDBC (Java Data Base Connectivity) is the connection of java databases, is that one kind is used to perform SQL languages
The Java API of sentence, can provide unified access, it is by one group of class write with Java language and connects for a variety of relational databases
Mouth composition.
RowKey is equivalent to the primary key in mysql databases, and it is exactly the combination of that several primary key column, row
Order and the sequence consensus defined in primary key.
HDFS is Hadoop Distributed File System, is a distributed file system.
The beneficial effect that the present invention is reached：Using dynamic index diagram technology, the foundation of many condition column index is realized and fast
Fast query composition, composite index is created by setting up index map for each inquiry is special, it is to avoid the full table progressive scan of carry out, greatly
The big speed for improving electric power big data query composition.
Brief description of the drawings
Fig. 1 is the schematic diagram of an index embodiment of the dynamic index figure of the present invention.
Fig. 2 is the schematic flow sheet of the electric power big data query composition of the present invention.
Embodiment
The invention will be further described below in conjunction with the accompanying drawings.Following examples are only used for clearly illustrating the present invention
Technical scheme, and can not be limited the scope of the invention with this.
In order to solve the efficiency of big data inquiry, while the limitation brought of conventional composite index technology is avoided,
The present invention proposes a kind of composite index technology suitable for electricity consumption big data --- dynamic index diagram technology (dynamic
index graph, DIG)。
DIG technologies be it is a kind of be based on distributed storage, the index framework of Distributed Calculation, it establishes a set of vertical to data
The directory system of body.This set directory system is ranked up first with first domain, is set up some index starting points, is used hash
Technology will be indexed and is segmented, and the segmentation of the next field is pointed to by these starting points in first domain, by that analogy, build a multistage
Three-dimensional index segmented system.It is appropriate to merge reduction segments when a certain segmentation is more loose, when a certain segmentation comparatively dense
When, appropriate separation sets up segmentation, to reach the balance between the storage reading efficiency of segmentation and search efficiency.When an inquiry
During beginning, by one or more starting points, recursive query is carried out according to constraints.In the final inquiry for determining destination node
Hold.
DIG takes full advantage of the buffer scheduling of cloud equipment, and multinuclear is calculated, and the index of isolated establishment is connected into index system
The schematic diagram of system, as shown in Fig. 1 an index embodiment of dynamic index figure of the invention.When user performs query task
When, the examination query type of intelligence is inquired about scale, optimal search algorithm is chosen automatically by system.In three-dimensional directory system
In, evaded using the optimal algorithm of selection and searched for one by one, fully pre-processed using system between the multiple index produced and index
Anticipation is pre-read in association index, index, multi-threading parallel process.It is finally reached the effect for greatly improving inquiry velocity.
Because most of inquiries in ordinary size data system are can be completed in Miao Ji chronomeres, and these
Operation will often rise for mass data as minute level, hour level operation, DIG technologies by inquire about mass data when
Widely apply from time-consuming some minutes, accelerating to only needs some seconds, so that the response time of system is compressed to user's wait
Within the scope of psychological endurance.
With four equipment, exemplified by 4,000,000,000 datas, it is assumed that have five fields per data, each 10 bytes of field are determined
It is long.Its full table content is about 200GB, and every equipment handles 50GB data, with processing 3GB per minute hard disk upper limit disposal ability
Calculate, one query needs more than 15 minutes.Under the conditions of homepage inquiry is more excellent also more than 5 minutes.And use first after DIG technologies
Page query time can be foreshortened to 10-20 seconds, so that query time is fallen into the range of the psychological endurance of user's wait.
Index is a supplementary means for traditional database, if user has used an inquiry to combine, but this
Inquiry is combined and is not set up index, and it is also an acceptable solution to carry out inquiry using full table scan technology temporarily.
But when the data volume for being assigned to every common computer it is big to a certain extent when, progressive scanning technology entirely without
When method meets the performance requirement of system, the efficient index under big data is not only then the auxiliary that inquiry accelerates, but inquiry
Necessary condition.Therefore, the design of big data Rapid Combination inquiry must is fulfilled for two requirements of speed and versatility.
To meet the rate request of Rapid Combination inquiry, search efficiency lifting is carried out in terms of following two：
（1）From the data storage layer of the bottom, realize that high-performance big data is stored using big data Virtual File System,
Good basis is provided for big data quick search；
（2）Using multi-dimensional database the processing mode optimized is provided for data.
From the perspective of versatility, because the requirement that big data is inquired about to index is no longer limited only to provide for inquiry
A kind of miscellaneous function of acceleration, but all inquiries have to the technology used, therefore, the index technology under big data technology must
Must can by any many condition the combination that is possible to.
The index user that DIG technologies are created need not go to consider the possibility quantity of the combination of any many condition, it is only necessary to right
The corresponding field of querying condition that may be used creates index.When user uses the conditional combination being made up of these conditions to enter
During row data query, database engine dynamically can independently be created index using these and provided originally according to the exclusive mechanism of itself appoints
The data query of many condition of meaning combination.
If being combined inquiry using without the field and other fields for having created index for creating index, system is first
First intelligently go to judge, discovery several fields therein have been indexed, preferentially will tentatively be judged with filtering using these fields,
Obtain one group of intermediate queries result；Due to some other fields and index is not set up, it is therefore desirable to again to intermediate result data
Scanned one by one.Because being filtered using the several fields indexed, therefore carry out intermediate result one by one
During comparison, the scale of data set has been greatly reduced.Therefore, even if having used only a few not create index in advance once in a while
Field inquired about, under this paper query engine, pretty good search efficiency can also be provided.
With the popularization of intelligent electric meter, the data volume of power industry increases in blowout.Power industry is current by terminal
Spread to one of rare several industries in each corner of huge numbers of families（Similar also has the industries such as water, coal gas）.
Electric power data has the features such as format, data volume is big, periodicity is obvious.By taking the electric power of Jiangsu as an example, if each
Hour collection data, then a hour will produce the data of 30,000,000 magnitudes, this data volume can also be adopted with data
Collect the lifting of frequency and the growth of electricity consumption Board Lot is exponentially increased.
The mass data produced in face of periodicity, the relatively advanced HBase in big data field is stored with locating as big data
The basic platform of reason.Although HBase also provides relatively good big data disposal ability, but it can not still provide any many
The index technology of condition query.
Because HBase is stored by row, and row race concept is supported, the inquiry timeliness of a rigid condition is done to a table
Rate is very high；But generally require to carry out the query composition of multiple conditions during general inquiry, and HBase does not support the group of multiple conditions
Close inquiry.Therefore HBase self-characteristic is combined, DIG technologies is introduced and is very important with the efficiency for improving query composition.
User realizes the intercommunication of database by JDBC and HBase, and completes statistics pretreatment in real time and set up inquiry rope
Draw, when HBase reads in newly-increased data, all data syn-chronizations are sent to the inquiry specified and accelerate server, by nominal key
The statistics of numerical value is carried out to certain field with the date, and sets up search index；, should when user sends inquiry request to HBase
Request is sent to special query engine immediately, is returned to corresponding index address according to querying condition, is found by index address
Initial data, and returning result.
As shown in Fig. 2 the schematic flow sheet of electric power big data query composition of the invention.Quick group of electric power big data
Query scheme is closed to comprise the following steps：
1) user inputs sql command from client；
2）Index data base is connected to by JDBC and HBase；
3）Sql command is parsed, corresponding index file is found from index data base；
4）Index file is trimmed, the dynamic index figure for specific querying command is formed；
5）By dynamic index figure, obtain needing the HFile of inquiry RowKey；
6）HBase according to RowKey from HDFS fetch evidence；
7）Query Result is returned into user.
Based on the inquiry of DIG technologies, no matter data total amount how much, the rate request of inquiry is less than 5 seconds.It is real by this programme
HBase any configuration need not be changed by having showed, while without any programming, you can realize statistics under the pressure of magnanimity big data
With the second level response of inquiry.
Described above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, without departing from the technical principles of the invention, some improvement and deformation can also be made, these improve and deformed
Also it should be regarded as protection scope of the present invention.
Claims (2)
1. a kind of dynamic index method inquired about for electric power big data Rapid Combination, it is characterised in that methods described is specifically wrapped
Include following steps：
SS1 utilizes dynamic index diagram technology, is that electric power big data sets up a set of three-dimensional directory system；
SS2 creates index using many condition query composition method；The step SS2 includes：Carried out when user's use condition is combined
During data query, database engine dynamically can provide any according to the exclusive mechanism of itself using these originally independent indexes that create
The data query of many condition of combination；Step SS2 also includes：If having been created using without the field for creating index with other
The field of index is combined inquiry, and system intelligently goes to judge first, it is found that several fields therein have been indexed, will be preferential
Tentatively judged with filtering using these fields, obtain one group of intermediate queries result；For and do not set up other fields of index,
Need again to scan intermediate result data one by one；
SS3 sets up electric power big data Rapid Combination query scheme；The step SS3 specifically includes following steps：
1) user inputs sql command from client；
2) index data base is connected to by JDBC and HBase；Step 2 in the step SS3) include：When HBase is read in newly
When increasing data, all data syn-chronizations are sent to the inquiry specified and accelerate server, by nominal key and date to certain field
The statistics of numerical value is carried out, and sets up search index；When user sends inquiry request to HBase, the request is sent to spy immediately
The query engine of system, returns to corresponding index address according to querying condition, finds initial data by index address, and return to knot
Really；
3) sql command is parsed, corresponding index file is found from index data base；
4) index file is trimmed, forms the dynamic index figure for specific querying command；
5) by dynamic index figure, obtain needing the HFile of inquiry RowKey；
6) HBase according to RowKey from HDFS fetch evidence；
7) Query Result is returned into user.
2. a kind of dynamic index method inquired about for electric power big data Rapid Combination according to claim 1, its feature
It is, the step SS1 includes：It is ranked up first with first domain, some index starting points is set up, then using hash
Technology will be indexed and is segmented, and build the three-dimensional index segmented system of a multistage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410654100.6A CN104317966B (en) | 2014-11-18 | 2014-11-18 | A kind of dynamic index method inquired about for electric power big data Rapid Combination |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410654100.6A CN104317966B (en) | 2014-11-18 | 2014-11-18 | A kind of dynamic index method inquired about for electric power big data Rapid Combination |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104317966A CN104317966A (en) | 2015-01-28 |
| CN104317966B true CN104317966B (en) | 2017-08-08 |
Family
ID=52373198
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410654100.6A Active CN104317966B (en) | 2014-11-18 | 2014-11-18 | A kind of dynamic index method inquired about for electric power big data Rapid Combination |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104317966B (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9471710B2 (en) | 2013-06-14 | 2016-10-18 | International Business Machines Corporation | On-the-fly encoding method for efficient grouping and aggregation |
| US9672248B2 (en) | 2014-10-08 | 2017-06-06 | International Business Machines Corporation | Embracing and exploiting data skew during a join or groupby |
| US9922064B2 (en) | 2015-03-20 | 2018-03-20 | International Business Machines Corporation | Parallel build of non-partitioned join hash tables and non-enforced N:1 join hash tables |
| US10650011B2 (en) | 2015-03-20 | 2020-05-12 | International Business Machines Corporation | Efficient performance of insert and point query operations in a column store |
| US10108653B2 (en) | 2015-03-27 | 2018-10-23 | International Business Machines Corporation | Concurrent reads and inserts into a data structure without latching or waiting by readers |
| US10831736B2 (en) * | 2015-03-27 | 2020-11-10 | International Business Machines Corporation | Fast multi-tier indexing supporting dynamic update |
| CN105069665B (en) * | 2015-09-06 | 2020-09-01 | 北京京东尚科信息技术有限公司 | Method and device for displaying sequencing of multi-channel off-site advertisement report data |
| CN106571940A (en) * | 2015-10-08 | 2017-04-19 | 中国移动通信集团广东有限公司 | Method and device of fusing network management data and resource data |
| CN105445579B (en) * | 2015-11-17 | 2018-06-22 | 国网山东省电力公司电力科学研究院 | The oil chromatography online monitoring data visualization system and method for multiple dimensioned Time-Series analysis |
| CN105574204B (en) * | 2016-01-08 | 2020-07-24 | 国网冀北电力有限公司 | Retrieval method and system for distributed power grid regulation and control operation data |
| CN105975585A (en) * | 2016-05-05 | 2016-09-28 | 云神科技投资股份有限公司 | Quick query method used for power big data |
| CN107515867B (en) * | 2016-06-15 | 2021-06-29 | 阿里巴巴集团控股有限公司 | Data storage and query method and device of NoSQL database and generation method and device of rowKey full combination |
| CN106126553B (en) * | 2016-06-16 | 2020-02-14 | 西安科技大市场有限公司 | Storage method based on scientific and technological resource big data |
| CN107239517B (en) * | 2017-05-23 | 2020-09-29 | 中国联合网络通信集团有限公司 | Multi-condition searching method and device based on Hbase database |
| CN107766452B (en) * | 2017-09-26 | 2021-07-06 | 广西电网有限责任公司电力科学研究院 | Indexing system suitable for high-speed access of power dispatching data and indexing method thereof |
| CN108667929A (en) * | 2018-05-08 | 2018-10-16 | 浪潮软件集团有限公司 | Method for synchronizing data to elastic search based on HBase coprocessor |
| CN109582643A (en) * | 2018-11-20 | 2019-04-05 | 中国石油大学(华东) | A kind of real-time dynamic data management system based on HBase |
| CN109688014B (en) * | 2019-01-03 | 2022-04-08 | 杭州电子科技大学 | Keyword-driven Web service automatic combination method |
| CN112765171B (en) * | 2021-01-12 | 2023-05-23 | 湖北宸威玺链信息技术有限公司 | Optimization algorithm for multi-field combined index fetch of block chain data uplink |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102253990A (en) * | 2011-07-05 | 2011-11-23 | 广东星海数字家庭产业技术研究院有限公司 | Interactive application multimedia data query method and device |
| CN103500183A (en) * | 2013-09-12 | 2014-01-08 | 国家计算机网络与信息安全管理中心 | Storage structure based on multiple-relevant-field combined index and building, inquiring and maintaining method |
| CN103955538A (en) * | 2014-05-19 | 2014-07-30 | 携程计算机技术（上海）有限公司 | HBase data persistence and query methods and HBase system |
| CN103984745A (en) * | 2014-05-23 | 2014-08-13 | 何震宇 | Distributed video vertical searching method and system |
-
2014
- 2014-11-18 CN CN201410654100.6A patent/CN104317966B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102253990A (en) * | 2011-07-05 | 2011-11-23 | 广东星海数字家庭产业技术研究院有限公司 | Interactive application multimedia data query method and device |
| CN103500183A (en) * | 2013-09-12 | 2014-01-08 | 国家计算机网络与信息安全管理中心 | Storage structure based on multiple-relevant-field combined index and building, inquiring and maintaining method |
| CN103955538A (en) * | 2014-05-19 | 2014-07-30 | 携程计算机技术（上海）有限公司 | HBase data persistence and query methods and HBase system |
| CN103984745A (en) * | 2014-05-23 | 2014-08-13 | 何震宇 | Distributed video vertical searching method and system |
Non-Patent Citations (1)
| Title |
|---|
| 基于HDFS开源架构与多级索引表的海量数据检索mDHT算法;汤羽等;《计算机科学》;20130228;第40卷(第2期);第195-199页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104317966A (en) | 2015-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104317966B (en) | A kind of dynamic index method inquired about for electric power big data Rapid Combination | |
| CN109299102B (en) | HBase secondary index system and method based on Elastcissearch | |
| CN103631909B (en) | System and method for combined processing of large-scale structured and unstructured data | |
| CN102184222B (en) | Quick searching method in large data volume storage | |
| CN102402617A (en) | Easily compressed database index storage system using fragments and sparse bitmap, and corresponding construction, scheduling and query processing methods | |
| CN102332030A (en) | Data storing, managing and inquiring method and system for distributed key-value storage system | |
| CN105488231A (en) | Self-adaption table dimension division based big data processing method | |
| Wang et al. | Distributed storage and index of vector spatial data based on HBase | |
| CN103226608B (en) | A kind of parallel file searching method based on directory level telescopic Bloom Filter bitmap table | |
| CN106611053A (en) | Data cleaning and indexing method | |
| CN109947791A (en) | A kind of database statement optimization method, device, equipment and storage medium | |
| CN101051309A (en) | Researching system and method used in digital labrary | |
| CN114328779A (en) | Geographic information cloud disk based on cloud computing efficient retrieval and browsing | |
| Ghotiya et al. | Migration from relational to NoSQL database | |
| KR101955376B1 (en) | Processing method for a relational query in distributed stream processing engine based on shared-nothing architecture, recording medium and device for performing the method | |
| CN103177046B (en) | A kind of data processing method based on row storage data base and equipment | |
| CN104268158A (en) | Structural data distributed index and retrieval method | |
| Huang et al. | R-HBase: A multi-dimensional indexing framework for cloud computing environment | |
| CN107273443B (en) | Mixed indexing method based on metadata of big data model | |
| Sheng et al. | Dynamic top-k range reporting in external memory | |
| CN111680072B (en) | System and method for dividing social information data | |
| CN117743369A (en) | Data query method and device, storage medium and terminal equipment | |
| CN105975585A (en) | Quick query method used for power big data | |
| CN108319604B (en) | Optimization method for association of large and small tables in hive | |
| CN104252537A (en) | Index fragmentation method based on mail characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |