CN105159845A - Memory reading method - Google Patents
Memory reading method Download PDFInfo
- Publication number
- CN105159845A CN105159845A CN201510562788.XA CN201510562788A CN105159845A CN 105159845 A CN105159845 A CN 105159845A CN 201510562788 A CN201510562788 A CN 201510562788A CN 105159845 A CN105159845 A CN 105159845A
- Authority
- CN
- China
- Prior art keywords
- index
- data
- user message
- message table
- main memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides a memory reading method, which comprises the following steps: storing all foreign key index data corresponding to a user information sheet into a persistent storage module; and caching index high-frequency data into a main memory by an index high-frequency data caching module based on the main memory, carrying out cache replacement according to the cumulative frequency of data access, and utilizing a cache structure based on a consistent hash to manage a cache region of the main memory. The invention provides the memory reading method which can more accurately capture data access characteristics according to the access frequency of memory blocks so as to substantially improve data query and access efficiency.
Description
Technical field
The present invention relates to data access, particularly a kind of memory reading method.
Background technology
In order to successfully manage storage and the searching and managing of mass data, people use distributed data-storage system substitutional relation data base management system (DBMS) more and more.Under the large market demand demand driving of urgent enterprise, there is non-relation data storage system at present.But, because existing non-relation data storage system lacks outer key index ability, need to scan full table, cause search efficiency low, the data query that being difficult to satisfies the demand responds fast and statistical study scene.
Summary of the invention
For solving the problem existing for above-mentioned prior art, the present invention proposes a kind of memory reading method, comprising:
By all external key index datastore corresponding for user message table in permanent memory module; Utilize the index high-frequency data cache module based on main memory, be buffered in main memory by index high-frequency data, the cumulative frequency according to data access carries out buffer memory replacement, utilizes the buffer area of buffer structure to main memory based on consistance hash to manage.
Preferably, described by all external key index datastore corresponding for user message table in permanent memory module, comprise further:
For the external key attribute be kept in user message table sets up concordance list, and concordance list is kept in non-relation data storage system, each concordance list is used for the index of the external key attribute to be checked in storage administration user message table, and the external key attribute drawn for lithol yet to be built in user message table is defined as follows the concordance list major key of form:
< user message table index column name, user message table index train value, user message table major key >
Wherein, index column name is the another name of indexed attribute in user message table; The address of indexed record in user message table is provided, obtains record indexed in user message table by user message table major key; Take querying attributes as concordance list major key, concordance list comprises the part field of user message table, only needs the field of complementary access to leave in the external key attribute of concordance list by inquiry; For the query composition of multiple external key attribute column, set up index based on multiple queries attribute column, use comma to do separator, after multiple attribute column establishes index, query composition is converted to the inquiry based on concordance list major key;
Index high-frequency data, based on the index high-frequency data cache module of main memory, is buffered in main memory, comprises further by described utilization:
Main memory high frequency index data buffer memory adopts following major key form:
< user message table index column name, user message table index train value >
Wherein, user message table index column name is identical with permanent index stores layer with the implication of user message table index train value, each index major key in main memory indexed cache layer correspond to the index record set that has same index train value, this set contains all concordance list data records corresponding with this index value, also contains other external key attributes needing access in set, complete main memory index data form is as follows:
Index major key: < user message table index column name, user message table index train value >
Index set: { < user message table major key, { the frequent Access Column name of <, frequent access train value >}>}
User data record is obtained in user message table by the user message table major key of this set correspondence, index main memory cache layer data are stored in main storage data base, and automatically complete above-mentioned hash and query script by main storage data base, this query script comprises: first arrive index main memory cache layer inquiry high frequency index data, if not hit, be then forwarded to index permanent storage layer and retrieve in buffer memory by inquiry.
Preferably, the described cumulative frequency according to data access carries out buffer memory replacement, comprises further:
The number of times that periodically cumulative buffer memory index set is accessed, and obtain frequency values be kept in cache metadata by periodically cumulative for access algorithm, the cumulative frequency of all records is sorted, selects the index record of the highest predetermined number of cumulative frequency to be cached in main memory;
By user message table, concordance list and numerical tabular are all stored in the index permanent storage layer based on non-relation data storage system, indexed cache storage organization is based on the set of main storage data base, main storage data base is also carry out organising data with key-value pair form, the key of main storage data base is done with the index major key of high-frequency data, indexed set cooperation is that the value of main storage data base is kept in main memory buffer memory, be bundled in same set to make the record with same index train value, based on the query hit of index train value in units of gathering, each set can add up its access times in a computation period, when performing inquiry request, the service processes of main memory buffer memory is to the access times in the every bar index data minute book cycle had access to, and the data of main memory buffer memory are not replaced, until when inquiry request number of times reaches predeterminated frequency computation period, the renewal that service processes triggers buffer memory is replaced, after to all record calculated rates, sort according to frequency, the highest K of a frequency sequence set record is cached in main memory, the record number limit calculation that wherein selection of numerical value K can be held according to spatial cache goes out frequency threshold, set higher than threshold value is cached in main memory.
Preferably, described data-storage system uses ZooKeeper to detect the active state of service processes on hosting node, the each hosting node service processes of index main memory cache layer sets up the session with ZooKeeper respectively, and creating transient node to represent the active state of self, each hosting node service processes observes the active state of other node processes from ZooKeeper system image;
According to the difference according to data entry mode, index creation is divided into towards stream data and the index creation towards batching data; Index creation process is all read a record of user message table, external key attribute generates an index record, if meet buffer memory condition, generates the index data of main memory cache layer simultaneously, finally index data is updated to respectively permanent storage layer and main memory cache layer, and upgrades numerical tabular; Described data-storage system utilizes MapReduce to carry out parallelization and performs static data index creation, first MapReduce task obtains inputting <Row, Result>, wherein Row is the line unit of user message table, Result is the non-relation data storage system record obtained by Row, then the index data of its correspondence is generated according to index information, and index data is inserted in level index, the parallelization processing power provided to utilize MapReduce is to accelerate index creation process.
The present invention compared to existing technology, has the following advantages:
The present invention proposes a kind of memory reading method, according to the feature of memory block access frequency capture-data access more accurately, significantly improve the efficiency of data query access.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of the memory reading method according to the embodiment of the present invention.
Embodiment
Detailed description to one or more embodiment of the present invention is hereafter provided together with the accompanying drawing of the diagram principle of the invention.Describe the present invention in conjunction with such embodiment, but the invention is not restricted to any embodiment.Scope of the present invention is only defined by the claims, and the present invention contain many substitute, amendment and equivalent.Set forth many details in the following description to provide thorough understanding of the present invention.These details are provided for exemplary purposes, and also can realize the present invention according to claims without some in these details or all details.
An aspect of of the present present invention provides a kind of memory reading method based on data-storage system.Fig. 1 is the memory reading method process flow diagram according to the embodiment of the present invention.Data-storage system of the present invention is based on the level index of non-relation data storage system and inquiry.Data-storage system is divided into two-layer: 1) permanent storage layer, is used for all external key index datas corresponding to storing subscriber information table; 2) based on the cache layer of main memory, utilize based on index high-frequency data cache replacement policy, partial index high-frequency data is buffered in main memory.And consider the cumulative frequency of data access, thus the feature of capture-data access more accurately.Simultaneously spatial cache less than in, improve buffer memory and insert strategy, ensure that spatial cache is fully used, cache hit rate can obtain fast lifting at data loading phase and tend towards stability, meanwhile, the main memory buffer structure based on consistance hash is that data-storage system cache layer of the present invention provides good extensibility and fault-tolerance.
Full table scan during in order to avoid inquiring about external key, data-storage system of the present invention is that the external key attribute be kept in user message table sets up concordance list, and is kept at by concordance list in non-relation data storage system.The index of certain external key attribute to be checked that each concordance list is used in storage administration user message table.The present invention is the concordance list major key that external key attribute that in user message table, lithol yet to be built draws is defined as follows form:
< user message table index column name, user message table index train value, user message table major key >
Wherein, index column name is the another name of indexed attribute in user message table.By row name being mapped to a brief another name, the expense of concordance list major key storage space can be reduced.Ensure that the uniqueness of concordance list major key; The address of indexed record in user message table is provided, by user message table major key, record indexed in user message table can be obtained fast.Take querying attributes as concordance list major key, concordance list comprises the part field of user message table, only needs the field of complementary access to leave in the external key attribute of concordance list by inquiry, avoids again calling party information table and the secondary disk access that causes.
There is the demand of multiple external key combinations of attributes inquiry in actual applications, for this reason, be similar to the multifield index in database, need the combined index creating multiple external key attribute column.For the query composition situation of multiple external key attribute column, the present invention is based on multiple queries attribute column and set up index.(separator of data-storage system of the present invention is configurable, and for integer, float and regular length character string type, system is specified and used ‘ 0 ' as separator to use comma to do separator.For varying string type, user needs self-defined separator).After multiple attribute column establishes index, query composition just converts the inquiry based on concordance list major key to, and Index process is identical with single attribute query principle with query script.
Index data high for those access frequencys in index is buffered in main memory as high-frequency data by the further aspect of the present invention, forms level index stores and inquiry mechanism, improves the inquiry velocity of index further.Main memory high frequency index data buffer memory form in data-storage system is different from the indexed format in permanent storage, and the major key form of main memory buffer memory index is:
< user message table index column name, user message table index train value >
Wherein, user message table index column name and the implication of user message table index train value and identical in permanent index stores layer.Each index major key in main memory indexed cache layer correspond to the index record set that has same index train value, and this set contains all concordance list data records corresponding with this index value.The same with permanent index stores layer, also contains other external key attributes needing access in set.Therefore, complete main memory index data form is as follows:
Index major key: < user message table index column name, user message table index train value >
Index set: { < user message table major key, { the frequent Access Column name of <, frequent access train value >}>}
Corresponding user data record can be obtained in user message table by the user message table major key of this set correspondence.Index main memory cache layer data are stored in main storage data base in realization, and automatically complete above-mentioned hash and fast query process by main storage data base.
The query script of level index stores model is: first arrive index main memory cache layer inquiry high frequency index data, if not hit in buffer memory, then inquiry is forwarded to index permanent storage layer and retrieves, reduce disk access expense, improve overall query performance.
Invention introduces consistance hash to complete the storage administration of index high-frequency data in main memory.When memory node changes, only need the data of moving fraction.And by being mapped to by memory node on the pseudo-random distribution point of annulus, the load balancing between each memory node effectively can be ensured.Main memory buffer memory needs to find the actual position of manipulative indexing record by twice hash: the server node being found index data place for the first time by consistance hash; Second time then finds the index data address in node by the hashing mechanism of main storage data base.
The present invention stores numerical tabular in index permanent storage layer, is used for recording in an orderly manner and store the set of all values of index attributes, to support efficiently based on the range query of index main memory buffer memory.In the concordance list of permanent storage layer, in concordance list major key, add user message table major key to safeguard its uniqueness.In numerical tabular, the present invention only preserves index train value, and therefore the record number of numerical tabular can be more much smaller than user message table.When user carries out range query to level index storage system, numerical tabular can be accessed frequently, and therefore numerical tabular can be buffered in main memory by underlying file systems, greatly can improve the search efficiency of numerical tabular.
Index column value is carried out granularity segmentation by the present invention, the corresponding numerical value table record of the multiple index train values in each segmentation limit.Index record in index column span corresponding for numerical value table record is stored in same main memory buffer memory index set (as main memory buffer memory index structure).Therefore, after adopting such scheme, remain the corresponding main storage data base set of a numerical value table record.When process range is inquired about, first inquire about numerical tabular, obtain dropping in query context, meet all numerical tabular records of querying condition.Then the monodrome initiating batch with numerical tabular record value for request is inquired about, and aggregate query result.Therefore not coverage query script.
Need to select suitable victim to eliminate out buffer memory after buffer memory is full, Here it is cache replacement policy.In cache replacement policy of the present invention, the number of times that periodically cumulative buffer memory index set is accessed, and obtain frequency values be kept at periodically cumulative for access algorithm in cache metadata.And then, the cumulative frequency of all records is sorted, selects the index record of the highest predetermined number of cumulative frequency to be cached in main memory.
User message table, concordance list and numerical tabular are all stored in the index permanent storage layer based on non-relation data storage system.In main memory buffer memory, the present invention preserves index high-frequency data.Indexed cache storage organization is the set based on main storage data base, and main storage data base is also carry out organising data with key-value pair form.The index major key of high-frequency data does the key of main storage data base, and the value that indexed set cooperation is main storage data base is kept in main memory buffer memory.Obviously, the record with same index train value is bundled in same set, based on the query hit of index train value in units of gathering.Meanwhile, they are also the cumulative base units of frequency, and each set can add up its access times in a computation period.
In order to reduce calculating that frequency computation part brings and upgrade expense, when performing inquiry request, the service processes of main memory buffer memory, to the access times in the every bar index data minute book cycle had access to, is not now replaced the data of main memory buffer memory.Until when inquiry request number of times reaches predeterminated frequency computation period, the renewal that service processes triggers buffer memory is replaced.According to frequency totalization formula to all record calculated rates, sort according to frequency, the highest K of a frequency sequence set record is cached in main memory, the record number comprised in set is unfixed, so the record number limit calculation that the selection of numerical value K can be held according to spatial cache goes out frequency threshold, the set higher than threshold value is cached in main memory.
Whole level index storage system is divided into following module by data-storage system of the present invention, and system function module divides.
1) index creation administration module.The metadata of management index, the information such as concordance list title, index column that this metadata record user message table is corresponding, and realize for the stream data of non-relation data storage system and the index creation method of static data two kinds of different qualities data, comprise the insertion of support concordance list and numerical tabular, deletion, renewal rewards theory.
2) permanent storage administration module.There is provided the permanent storage of concordance list and numerical tabular, non-relation data storage system provides extensibility and fault-tolerance for permanent storage data.
3) index main memory cache module.The buffer memory of management index high-frequency data, renewal and address maps, realize frequency and to add up cache replacement policy, the data of frequent access recently can be cached in main memory.
4) query execution module.The inquiry request of user is translated into the order of system identification, call corresponding method and perform inquiry, and Query Result is gathered return to client.
Data-storage system of the present invention employs the active state that ZooKeeper reliably detects service processes on hosting node.The each hosting node service processes of index main memory cache layer can set up the session with ZooKeeper respectively, and creates transient node to represent the active state of self.Each hosting node service processes can observe the active state of other node processes from ZooKeeper system image.System by realizing the failure detection of hosting node and process of reaching the standard grade to the monitoring of distributed memory node state, thus realizes the high availability of index main memory cache layer.
According to the difference of data entry mode, index creation can be divided into towards stream data and the index creation towards batching data.Index creation process is all read a record of user message table, and external key attribute generates an index record, if meet buffer memory condition, generates the index data of main memory cache layer simultaneously.Finally index data is updated to respectively permanent storage layer and main memory cache layer, and upgrades numerical tabular.Because static data is generally relatively large, in order to accelerate the establishment speed of static data index, the present invention utilizes MapReduce to carry out parallelization and performs static data index creation.First MapReduce task obtains inputting <Row, Result>, and wherein Row is the line unit of user message table, and Result is the non-relation data storage system record obtained by Row.Then generate the index data of its correspondence according to index information, and index data is inserted in level index.Whole process does not need the Reduce stage to complete, simultaneously owing to being separate between user profile table record, so can make full use of parallelization processing power that MapReduce provides to accelerate index creation process.
Data query process comprises monodrome inquiry and range query.
(1) monodrome inquiry
The present invention, by setting up the index on external key attribute, supports the monodrome inquiry in efficient foreign key column and range query.Monodrome is inquired about, and namely inquiry request conditional attribute is defined the unique inquiry of value.For monodrome inquiry, the basic procedure of client is as follows:
1) obtain the address of ZooKeeper from configuration file, set up the connection with ZooKeeper, obtain all service processess of registration, determine current all service processes positional informations that main memory buffer memory is provided.
2) service processes to main memory cache layer initiates inquiry request.If main memory cache layer hits, then return the result that main memory cache layer provides, terminate inquiry.
3) if main memory cache layer is miss, then inquiry request initiated by the concordance list to non-relation data storage system.After obtaining result, return the result of inquiring about and obtaining, terminate inquiry.
Can find out, if in the hit of main memory cache layer, whole querying flow all can not have access to disk, decreases disk access expense, can increase substantially response speed.In addition, can client be buffered in the main memory cache layer service processes information that ZooKeeper access obtains, the communication overhead of data query can be reduced further in follow-up access.
(2) range query
The said range query of the present invention, namely the value of inquiry request conditional attribute is the inquiry of scope.Main memory cache layer improves query performance by the method for consistance hash by Data distribution8 to each memory node, and hash destroys the order of index column, so the present invention needs all values of recording indexes table major key, is kept in numerical tabular.By this numerical tabular, the present invention can obtain the train value of index major key all existence within the scope of certain.For range query, the basic procedure of client is as follows:
1) obtain the address of ZooKeeper from configuration file, set up the connection with ZooKeeper, obtain all service processess of registration, determine current all service processes positional informations that main memory buffer memory is provided.
2) inquiry train values all in query context is obtained from numerical tabular.
3) monodrome inquiry request is initiated successively.Query Result is gathered and returns.
In order to promote search efficiency further, data-storage system of the present invention carries out following optimization and improvement to range query:
1) obtain the address of ZooKeeper from configuration file, set up the connection with ZooKeeper, obtain all service processess of registration, determine current all service processes positional informations that main memory buffer memory is provided.
2) according to the condition of range query, all index train values that client exists between acquisition scope from numerical tabular.
3) for the index train value of all existence, calculate memory node address according to consistance hashing algorithm, thus the index train value of all existence is got up to relevant node address one_to_one corresponding.
4) concomitantly inquiry request is initiated to interdependent node, wherein, a batch request will be merged into the multiple queries request that same node is initiated.
5) the main memory index service process on each node responds inquiry request, if the content of inquiry is in main memory, then directly returns the data in main memory; Otherwise, the inquiry that service processes will be initiated lasting accumulation layer, and return Query Result.
6) client gathers the Query Result returned from each service node.
In sum, the present invention proposes a kind of memory reading method, according to the feature of memory block access frequency capture-data access more accurately, significantly improve the efficiency of data query access.
Obviously, it should be appreciated by those skilled in the art, above-mentioned of the present invention each module or each step can realize with general computing system, they can concentrate on single computing system, or be distributed on network that multiple computing system forms, alternatively, they can realize with the executable program code of computing system, thus, they can be stored and be performed by computing system within the storage system.Like this, the present invention is not restricted to any specific hardware and software combination.
Should be understood that, above-mentioned embodiment of the present invention only for exemplary illustration or explain principle of the present invention, and is not construed as limiting the invention.Therefore, any amendment made when without departing from the spirit and scope of the present invention, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.In addition, claims of the present invention be intended to contain fall into claims scope and border or this scope and border equivalents in whole change and modification.
Claims (4)
1. a memory reading method, for carrying out digital independent to non-relation data storage system, is characterized in that, comprise:
By all external key index datastore corresponding for user message table in permanent memory module; Utilize the index high-frequency data cache module based on main memory, be buffered in main memory by index high-frequency data, the cumulative frequency according to data access carries out buffer memory replacement, utilizes the buffer area of buffer structure to main memory based on consistance hash to manage.
2. method according to claim 1, is characterized in that, described by all external key index datastore corresponding for user message table in permanent memory module, comprise further:
For the external key attribute be kept in user message table sets up concordance list, and concordance list is kept in non-relation data storage system, each concordance list is used for the index of the external key attribute to be checked in storage administration user message table, and the external key attribute drawn for lithol yet to be built in user message table is defined as follows the concordance list major key of form:
< user message table index column name, user message table index train value, user message table major key >
Wherein, index column name is the another name of indexed attribute in user message table; The address of indexed record in user message table is provided, obtains record indexed in user message table by user message table major key; Take querying attributes as concordance list major key, concordance list comprises the part field of user message table, only needs the field of complementary access to leave in the external key attribute of concordance list by inquiry; For the query composition of multiple external key attribute column, set up index based on multiple queries attribute column, use comma to do separator, after multiple attribute column establishes index, query composition is converted to the inquiry based on concordance list major key;
Index high-frequency data, based on the index high-frequency data cache module of main memory, is buffered in main memory, comprises further by described utilization:
Main memory high frequency index data buffer memory adopts following major key form:
< user message table index column name, user message table index train value >
Wherein, user message table index column name is identical with permanent index stores layer with the implication of user message table index train value, each index major key in main memory indexed cache layer correspond to the index record set that has same index train value, this set contains all concordance list data records corresponding with this index value, also contains other external key attributes needing access in set, complete main memory index data form is as follows:
Index major key: < user message table index column name, user message table index train value >
Index set: { < user message table major key, { the frequent Access Column name of <, frequent access train value >}>}
User data record is obtained in user message table by the user message table major key of this set correspondence, index main memory cache layer data are stored in main storage data base, and automatically complete above-mentioned hash and query script by main storage data base, this query script comprises: first arrive index main memory cache layer inquiry high frequency index data, if not hit, be then forwarded to index permanent storage layer and retrieve in buffer memory by inquiry.
3. method according to claim 2, is characterized in that, the described cumulative frequency according to data access carries out buffer memory replacement, comprises further:
The number of times that periodically cumulative buffer memory index set is accessed, and obtain frequency values be kept in cache metadata by periodically cumulative for access algorithm, the cumulative frequency of all records is sorted, selects the index record of the highest predetermined number of cumulative frequency to be cached in main memory;
By user message table, concordance list and numerical tabular are all stored in the index permanent storage layer based on non-relation data storage system, indexed cache storage organization is based on the set of main storage data base, main storage data base is also carry out organising data with key-value pair form, the key of main storage data base is done with the index major key of high-frequency data, indexed set cooperation is that the value of main storage data base is kept in main memory buffer memory, be bundled in same set to make the record with same index train value, based on the query hit of index train value in units of gathering, each set can add up its access times in a computation period, when performing inquiry request, the service processes of main memory buffer memory is to the access times in the every bar index data minute book cycle had access to, and the data of main memory buffer memory are not replaced, until when inquiry request number of times reaches predeterminated frequency computation period, the renewal that service processes triggers buffer memory is replaced, after to all record calculated rates, sort according to frequency, the highest K of a frequency sequence set record is cached in main memory, the record number limit calculation that wherein selection of numerical value K can be held according to spatial cache goes out frequency threshold, set higher than threshold value is cached in main memory.
4. method according to claim 3, it is characterized in that, described data-storage system uses ZooKeeper to detect the active state of service processes on hosting node, the each hosting node service processes of index main memory cache layer sets up the session with ZooKeeper respectively, and creating transient node to represent the active state of self, each hosting node service processes observes the active state of other node processes from ZooKeeper system image;
According to the difference according to data entry mode, index creation is divided into towards stream data and the index creation towards batching data; Index creation process is all read a record of user message table, external key attribute generates an index record, if meet buffer memory condition, generates the index data of main memory cache layer simultaneously, finally index data is updated to respectively permanent storage layer and main memory cache layer, and upgrades numerical tabular; Described data-storage system utilizes MapReduce to carry out parallelization and performs static data index creation, first MapReduce task obtains inputting <Row, Result>, wherein Row is the line unit of user message table, Result is the non-relation data storage system record obtained by Row, then the index data of its correspondence is generated according to index information, and index data is inserted in level index, the parallelization processing power provided to utilize MapReduce is to accelerate index creation process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510562788.XA CN105159845A (en) | 2015-09-07 | 2015-09-07 | Memory reading method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510562788.XA CN105159845A (en) | 2015-09-07 | 2015-09-07 | Memory reading method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105159845A true CN105159845A (en) | 2015-12-16 |
Family
ID=54800706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510562788.XA Pending CN105159845A (en) | 2015-09-07 | 2015-09-07 | Memory reading method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105159845A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021126A (en) * | 2016-05-31 | 2016-10-12 | 腾讯科技（深圳）有限公司 | Cache data processing method, server and configuration device |
| CN107846681A (en) * | 2017-10-10 | 2018-03-27 | 北京小米移动软件有限公司 | Information updating method and device |
| CN109085997A (en) * | 2017-06-13 | 2018-12-25 | 西部数据技术公司 | Memory-efficient for nonvolatile memory continues key assignments storage |
| CN109213760A (en) * | 2018-08-02 | 2019-01-15 | 南瑞集团有限公司 | The storage of high load business and search method of non-relation data storage |
| CN109739883A (en) * | 2019-01-03 | 2019-05-10 | 广东南方数码科技股份有限公司 | Promote the method, apparatus and electronic equipment of data query performance |
| CN110062098A (en) * | 2018-01-18 | 2019-07-26 | 中兴通讯股份有限公司 | A kind of method and device, the terminal of application module storage |
| CN110162528A (en) * | 2019-05-24 | 2019-08-23 | 安徽芃睿科技有限公司 | Magnanimity big data search method and system |
| CN110334119A (en) * | 2019-06-21 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of data correlation processing method, device, equipment and medium |
| CN112035528A (en) * | 2020-09-11 | 2020-12-04 | 中国银行股份有限公司 | Data query method and device |
| CN114490749A (en) * | 2021-12-28 | 2022-05-13 | 珠海大横琴科技发展有限公司 | Resource access method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101996250A (en) * | 2010-11-15 | 2011-03-30 | 中国科学院计算技术研究所 | Hadoop-based mass stream data storage and query method and system |
| CN103942289A (en) * | 2014-04-12 | 2014-07-23 | 广西师范大学 | Memory caching method oriented to range querying on Hadoop |
| CN104850572A (en) * | 2014-11-18 | 2015-08-19 | 中兴通讯股份有限公司 | HBase non-primary key index building and inquiring method and system |
-
2015
- 2015-09-07 CN CN201510562788.XA patent/CN105159845A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101996250A (en) * | 2010-11-15 | 2011-03-30 | 中国科学院计算技术研究所 | Hadoop-based mass stream data storage and query method and system |
| CN103942289A (en) * | 2014-04-12 | 2014-07-23 | 广西师范大学 | Memory caching method oriented to range querying on Hadoop |
| CN104850572A (en) * | 2014-11-18 | 2015-08-19 | 中兴通讯股份有限公司 | HBase non-primary key index building and inquiring method and system |
Non-Patent Citations (1)
| Title |
|---|
| 葛微 等: "HiBase：一种基于分层式索引的高效HBase查询技术与系统", 《第二届CCF大数据学术会议论文集》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021126A (en) * | 2016-05-31 | 2016-10-12 | 腾讯科技（深圳）有限公司 | Cache data processing method, server and configuration device |
| CN109085997A (en) * | 2017-06-13 | 2018-12-25 | 西部数据技术公司 | Memory-efficient for nonvolatile memory continues key assignments storage |
| CN107846681A (en) * | 2017-10-10 | 2018-03-27 | 北京小米移动软件有限公司 | Information updating method and device |
| CN110062098A (en) * | 2018-01-18 | 2019-07-26 | 中兴通讯股份有限公司 | A kind of method and device, the terminal of application module storage |
| CN109213760A (en) * | 2018-08-02 | 2019-01-15 | 南瑞集团有限公司 | The storage of high load business and search method of non-relation data storage |
| CN109213760B (en) * | 2018-08-02 | 2021-10-22 | 南瑞集团有限公司 | High-load service storage and retrieval method for non-relational data storage |
| CN109739883A (en) * | 2019-01-03 | 2019-05-10 | 广东南方数码科技股份有限公司 | Promote the method, apparatus and electronic equipment of data query performance |
| CN110162528A (en) * | 2019-05-24 | 2019-08-23 | 安徽芃睿科技有限公司 | Magnanimity big data search method and system |
| CN110334119A (en) * | 2019-06-21 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of data correlation processing method, device, equipment and medium |
| CN112035528A (en) * | 2020-09-11 | 2020-12-04 | 中国银行股份有限公司 | Data query method and device |
| CN112035528B (en) * | 2020-09-11 | 2024-04-16 | 中国银行股份有限公司 | Data query method and device |
| CN114490749A (en) * | 2021-12-28 | 2022-05-13 | 珠海大横琴科技发展有限公司 | Resource access method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105159845A (en) | Memory reading method | |
| US10013317B1 (en) | Restoring a volume in a storage system | |
| US20190303382A1 (en) | Distributed database systems and methods with pluggable storage engines | |
| CN107025243B (en) | Resource data query method, query client and query system | |
| CN102591970B (en) | Distributed key-value query method and query engine system | |
| CN107491523B (en) | Method and device for storing data object | |
| CN104850572A (en) | HBase non-primary key index building and inquiring method and system | |
| US20160364407A1 (en) | Method and Device for Responding to Request, and Distributed File System | |
| CN101236569B (en) | Highly effective dynamic path analysis method based on ContextFS | |
| CN103246616A (en) | Global shared cache replacement method for realizing long-short cycle access frequency | |
| CN104408111A (en) | Method and device for deleting duplicate data | |
| CN103607312A (en) | Data request processing method and system for server system | |
| US20220374407A1 (en) | Multi-tenant partitioning in a time-series database | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| US11775480B2 (en) | Method and system for deleting obsolete files from a file system | |
| CN101404649B (en) | Data processing system based on CACHE and its method | |
| CN108446308A (en) | The system and method for scalable distributed database | |
| US9239871B2 (en) | System and method for analyzing sequential data access efficiency | |
| CN103365987A (en) | Clustered database system and data processing method based on shared-disk framework | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| US20220342888A1 (en) | Object tagging | |
| CN110109931B (en) | Method and system for preventing data access conflict between RAC instances | |
| CN102737061A (en) | Distributed ticket query management system and method | |
| CN107133334B (en) | Data synchronization method based on high-bandwidth storage system | |
| WO2015015727A1 (en) | Storage device, data access method, and program recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151216 |