CN107783980A - Index data generates and data query method and device, storage and inquiry system - Google Patents

Index data generates and data query method and device, storage and inquiry system Download PDF

Info

Publication number
CN107783980A
CN107783980A CN201610720018.8A CN201610720018A CN107783980A CN 107783980 A CN107783980 A CN 107783980A CN 201610720018 A CN201610720018 A CN 201610720018A CN 107783980 A CN107783980 A CN 107783980A
Authority
CN
China
Prior art keywords
data
time
partition code
index
partition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610720018.8A
Other languages
Chinese (zh)
Other versions
CN107783980B (en
Inventor
冯驰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201610720018.8A priority Critical patent/CN107783980B/en
Publication of CN107783980A publication Critical patent/CN107783980A/en
Application granted granted Critical
Publication of CN107783980B publication Critical patent/CN107783980B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31—Indexing; Data structures therefor; Storage structures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a kind of generation of index data and data query method and device, storage and inquiry system, increase a partition code when building index data, subregion is carried out to large-scale continuous index data according to the partition code so that the index data in concordance list has preferable dispersion.Solves index data existing hot issue in storage.Further, when being inquired about based on partition code data, because index data stores in storage according to partition code, so that continuous index data realizes partitioned storage on a large scale in concordance list, and then large-scale inquiry mode is changed into the inquiry in multiple partition codes, the continuous-query of a wide range of high-volume data is realized, improves search efficiency.

Description

Index data generates and data query method and device, storage and inquiry system
Technical field
The invention belongs to data processing field, more particularly to a kind of generation of index data and data query method and device, Storage system and inquiry system.
Background technology
Unstructured storage system can support the storage of magnanimity big data and inquiry, in unstructured storage system, number Key-Value form storage according to this, data are arranged by Key natural order first, are then multiple data by data cutting Subregion (Region), finally by corresponding to distributed data storage corresponding to data partition to the data partition on machine, so as to real The distributed storage of existing data.Due to being arranged by Key natural order, unstructured storage system can be in storage mass data On the basis of, there is provided single-point is inquired about and range query.Single-point inquiry i.e. given a Key, Value corresponding to inquiry, and scope Inquiry gives Key scope, the Value inquired about in the range of this.However, this range query can only be inquired about according to Key, It can not provide similar to the search index mode in database.
Search index mode in database is similar in order to realize, two tables of establishment in unstructured storage system, one It is tables of data, one is secondary index table, and the storage organization of tables of data and secondary index table is Key-Value.Tables of data Data storage Key and data Value, secondary index table storage index Key and data Key, adds wherein index Key is index prefix Upper data Key, index prefix can realize range query, be to ensure to index Key uniqueness plus data Key.When right When data are inquired about, secondary index can be first inquired about, obtains data Key, data are obtained further according to data Key inquiry tables of data Value.For example, a data can include 3 fields uid=123, product=a, time=1000, its data Key is “uid_123；product_a；time_1000；", index Key is " product_a；time_1000；uid_123；product_ a；time_1000；", then data Key supports to specify uid and product time range queries, and uid is primary bar Part, but the time range query abilities that Key provides specified product are indexed, it is achieved thereby that inquiry specifies product at certain The function of all customer data in individual time range.
In practical application, data Key is identified (user id, abbreviation uid) with user in storage or identified with user (uid) the foundation of+object identity (product) as cutting Region, due to using user as a segmentation unit, so that it may So that the data included by an object to be cut into the data fragmentation of multiple smaller particle sizes, the dispersion of such data is preferable, The data of write-in can be dispersed on each machine.
And in order to realize a wide range of inquiry to data, the index Key in concordance list is usually to be used as cutting using product Region foundation, using, so may result in included by an object because an object often has many consumers Data are often magnanimity, so very poor as the dispersion of index data during cutting Region foundation using product so that even Index data in the continuous period is likely to writing a certain machine always so that the load of the machine is very high, index data Distribute uneven.Especially when multiple applications share a cluster, the data hot issue of superposition can be more serious.
As shown in figure 1, it is existing index Key structural representation.In Fig. 1, index Key is used as cutting using product Dispersion is very poor during Region foundation, and data are writing machine A always since 3600s, shown on the right side of Fig. 1, chooses Four index Key of data in this continuous time section of 3600s~12000s are illustrated, four index Key： product：a time：3600 data Key1, product：a time：4000 data Key2, product：a time：8000 numbers According to Key3 and product：a time：12000 data Key4 are stored on machine A.As shown in fig. 1 on the left-hand side, to concordance list Inquired about, if querying condition is product：A10800 ＞ time ＞=3600, then constantly will look into machine A Ask.According to shown in Fig. 1 structure construction index Key when, storage when concordance list when often lead to machine A load it is very high, There is the problem of data focus.
The content of the invention
The present invention provides a kind of index data generation and data query method and device, storage and inquiry system, for solving Certainly due in existing concordance list index data dispersion it is poor, cause occur the problem of data focus in index datastore.
To achieve these goals, the invention provides a kind of index data generation method, including：
Obtain the generation time of more datas of period memory storage；
According to the generation time of pieces of data, determine partition code corresponding to the pieces of data, the partition code with it is described Multiple time subregions of period are corresponding, and the partition code corresponding to different time subregion is different；
A plurality of index number corresponding with pieces of data is generated according to partition code corresponding to the pieces of data and pieces of data According to wherein the partition code is included in the index data.
To achieve these goals, the invention provides a kind of index data querying method, including：
Obtain the query time scope in inquiry request；
According to the query time scope, multiple partition codes are determined, and the inquiry request is decomposed into according to partition code Multiple subquery requests, the partition code are relative with multiple time subregions in the range of the query time of the data to be checked Should, the partition code corresponding to different time subregion is different；
According to the pre-recorded partition code and the corresponding relation of the database or memory space, each subregion is determined Database or memory space corresponding to code；
In the database or memory space corresponding to each partition code, according to the subquery corresponding with each partition code Request carries out the inquiry of data.
To achieve these goals, the invention provides a kind of storage system, including：
Storage server and at least one the first database or the first memory space for being used to store index data；Its In, the storage server is connected with the database or the memory space；
The storage server, the generation time of more datas for obtaining period memory storage, according to pieces of data The generation time, determine partition code corresponding to the pieces of data, data generation module, for according to the pieces of data and each Partition code corresponding to data generates index data corresponding with pieces of data, wherein the partition code is included in the index number In, the partition code is corresponding with multiple time subregions of the period, and the partition code corresponding to different time subregion is not Together.
To achieve these goals, the invention provides a kind of inquiry system, including：
Inquiry server, at least one the first database or the first memory space and extremely for being used to store index data Few one the second database or the second memory space for being used to store real data；Wherein, it is described inquiry server respectively with First database either first memory space and the connection of second database or second memory space；
The inquiry server, for obtaining the query time scope in inquiry request, according to the query time scope, Multiple partition codes are determined, and the inquiry request is decomposed into multiple subqueries according to partition code and asked, according to pre-recorded The partition code and first database or the corresponding relation of first memory space, are determined corresponding to each partition code First database or first memory space, and in first database corresponding to each partition code or institute State in the second memory space, according to the subquery request corresponding with each partition code in first database or described first Index data corresponding to memory space inquiry, and according to the index data inquired in second database or the Two memory spaces carry out data query；
Wherein, the partition code is corresponding with multiple time subregions in the range of the query time of the data to be checked, Partition code corresponding to different time subregion is different.
To achieve these goals, the invention provides a kind of index data generating means, including：
Acquisition module, the generation time of more datas for obtaining period memory storage；
Partition code generation module, for the generation time according to pieces of data, determine subregion corresponding to the pieces of data Code, the partition code is corresponding with multiple time subregions of the period, and the partition code corresponding to different time subregion is different；
Data generation module, for partition code generation and pieces of data according to corresponding to the pieces of data and pieces of data Corresponding index data, wherein the partition code is included in the index data.
To achieve these goals, the invention provides a kind of data query arrangement, including：
Acquisition module, for obtaining the query time scope in inquiry request；
Generation module, for according to the query time scope, determining multiple partition codes, and by the inquiry request according to Partition code is decomposed into the request of multiple subqueries, when the partition code is with multiple in the range of the query time of the data to be checked Between subregion it is corresponding, partition code corresponding to different time subregion is different；
Determining module, for the corresponding pass according to the pre-recorded partition code with the database or memory space System, determines the database or memory space corresponding to each partition code；
Enquiry module, in the database or memory space corresponding to each partition code, according to each partition code pair The subquery request answered carries out the inquiry of data.
To achieve these goals, the invention provides a kind of index data generating means, including：
Memory and processor；
The memory, for storage program；
The processor, for performing described program, for：
Obtain the generation time of more datas of period memory storage；
According to the generation time of pieces of data, determine partition code corresponding to the pieces of data, the partition code with it is described Multiple time subregions of period are corresponding, and the partition code corresponding to different time subregion is different；
Index data corresponding with pieces of data is generated according to partition code corresponding to the pieces of data and pieces of data, its Described in partition code be included in the index data in.
To achieve these goals, the invention provides a kind of data query arrangement, including：
Memory and processor；
The memory, for storage program；
The processor, for performing described program, for：
Obtain the query time scope in inquiry request；
According to the query time scope, multiple partition codes are determined, and the inquiry request is decomposed into according to partition code Multiple subquery requests, the partition code are relative with multiple time subregions in the range of the query time of the data to be checked Should, the partition code corresponding to different time subregion is different；
According to the pre-recorded partition code and the corresponding relation of the database or memory space, each subregion is determined Database or memory space corresponding to code；
In the database or memory space corresponding to each partition code, according to the subquery corresponding with each partition code Request carries out the inquiry of data.
Index data generation provided by the invention and data query method and device, storage and inquiry system, by structure Increase a partition code when indexing data, partitioned storage carried out to large-scale continuous index data according to the partition code, So that the index data in concordance list has preferable dispersion.Further, when being inquired about based on partition code data, by Stored when index data is in storage according to partition code so that continuous index data is carried out on a large scale in concordance list Partitioned storage, and then large-scale inquiry mode is changed into the inquiry in multiple partition codes, realize a wide range of high-volume data Continuous-query, improve search efficiency.
Brief description of the drawings
Fig. 1 is the storage schematic diagram of existing index data；
Fig. 2 is the schematic flow sheet for the index data generation method that the embodiment of the present invention one provides；
Fig. 3 is the schematic flow sheet for the index data generation method that the embodiment of the present invention two provides；
Fig. 4 is the storage schematic diagram of the index data generated in the embodiment of the present invention two；
Fig. 5 is the schematic flow sheet for the data query method that the embodiment of the present invention three provides；
Fig. 6 is the schematic flow sheet for the data query method that the embodiment of the present invention four provides；
Fig. 7 is the structural representation for the storage system that the embodiment of the present invention five provides；
Fig. 8 is the structural representation for the inquiry system that the embodiment of the present invention six provides；
Fig. 9 is the structural representation for the data storage and query system that the embodiment of the present invention seven provides；
Figure 10 is the structural representation for the index data generating means that the embodiment of the present invention eight provides；
Figure 11 is the structural representation for the data query arrangement that the embodiment of the present invention nine provides；
Figure 12 is the structural representation for the index table creating device that the embodiment of the present invention ten provides；
Figure 13 is the structural representation for the data query arrangement that the embodiment of the present invention 11 provides.
Embodiment
Below in conjunction with the accompanying drawings to index data provided in an embodiment of the present invention generation and data query method and device, storage It is described in detail with inquiry system.
Embodiment one
As shown in Fig. 2 it is the schematic flow sheet for the index data generation method that the embodiment of the present invention one provides.The index Data creation method comprises the following steps：
S101, obtain period memory storage more datas the generation time.
In practical application, during the actual data of storage system storage, it can be stored by Key-Value forms, can be with data Carry generation time, the user's mark of user corresponding to data of the data, and the object mark of query object corresponding to data The information such as knowledge.For example, actual data are used for representing the service condition of a certain product, the service condition can reflect different use Family to the product different time use.In the present embodiment, storage system, which is worked as, has stored data all in continuous time section Afterwards, the generation time of all data in the continuous time section can be got.
S102, the generation time according to pieces of data, determine partition code corresponding to pieces of data.
Wherein, partition code is corresponding with multiple time subregions of period, and the partition code corresponding to different time subregion is not Together.
In the present embodiment, after the generation time of pieces of data is got, according to the generation time of pieces of data, it is determined that respectively Partition code corresponding to data.
Preferably, in the present embodiment, when the generation time earliest in the generation time of pieces of data is generated as starting Between, the starting generation time is rounded, obtains time stepping information corresponding to the starting generation time.According further to Time stepping information and default time step corresponding to the starting generation time, obtain all included by continuous time section Time stepping information.Specifically, the starting generation time is rounded first to obtain a time corresponding with the starting generation time Stepping information, a default time step then is added on the starting generation time, obtains next generation time rounded, One time stepping information is obtained according to the generation time, then proceeded to default plus one on the basis of the generation time Time step, next generation time rounded is obtained, untill the generation time in the generation time of each data the latest.
Alternatively, according to default time step, subregion is carried out to the period and obtains time subregion, by the end of time subregion Point is used as time stepping information, then generates each self-corresponding partition code according to each time stepping information.
In the present embodiment, after all time stepping information is got, it can be generated according to each time stepping information Each self-corresponding partition code.Preferably, Message Digest Algorithm 5 (Message is carried out to each time stepping information Digest Algorithm5, abbreviation MD5) calculate, obtain time stepping information corresponding to MD5 values, take in MD5 values on default position Numerical value as partition code, such as take latter 3 of MD5 values to be used as partition code.Alternatively, each time stepping information is breathed out Uncommon (hash) is calculated, obtain the time stepping information corresponding to hash values, take the numerical value in hash values on default position as subregion Code, such as take latter 5 of hash values to be used as partition code.
In the present embodiment, the generation time of pieces of data is the timestamp of integer, when the starting generated in the time is generated Between corresponding timestamp round after obtain a corresponding time stepping information.For example, when timestamp precision is to the second, when Between the value stabbed be 1454466710, i.e. time 2016-02-03 10:31:50, default time step is 3600s, i.e., 1 hour Represent to carry out time stepping by 1 hour, the time obtained after timestamp 1454466710 corresponding to generating the time rounds will be originated Stepping information is 1454464800, that is, it is 2016-02-03 10 to change the time into:00:00, due to after rounding from 2016-02-03 10:00:00 starts, although from 2016-02-03 10:00:00 arrives 2016-02-03 10:31:49 no any data, still From 2016-02-03 10:31:59 start to include the data stored.Herein it is to be appreciated that needing to note when rounding timestamp Meaning, and if from 2016-02-03 11 after rounding:00:00 starts, then not from 2016-02-03 10:31:59 start to include depositing The data of storage will be lost.
S103, partition code generates a plurality of index number corresponding with pieces of data according to corresponding to pieces of data and pieces of data According to.
Specifically, after the partition code included by the period, can according to carried in pieces of data object identity, generation Time and corresponding partition code, generate index data corresponding with pieces of data.In the present embodiment, index data includes index Key and corresponding with index Key index Value.In order to ensure to index Key uniqueness, index Key includes data Key, Then data Key and index prefix corresponding with data Key form an index Key.In the present embodiment, by index data Discretization, increase a partition code formed according to the generation time of data in prefix is indexed.In practical application, index Prefix includes multiple fields：The generation time of the identification field of query object corresponding to data, partition code and data.
In the present embodiment, by included multiple fields in index prefix, respectively on predeterminated position corresponding to placement, Ran Houli Index of reference prefix and data Key corresponding with index prefix generation indexes Key.Preferably, in order to realize to index data Range query, now only have last field need to be a field that can represent scope, that is, need that the time will be generated This field is placed into the last position of index prefix.Other included fields must be to determine value in index prefix.This implementation In example, in order to realize large-scale inquiry, identification field is placed into the first place of index prefix, partition code is placed into identification field Below.The sequence of positions of each field is in index prefix：The generation time of identification field, partition code and data.Further, it is Ensure index Key uniqueness, index needs to add data Key behind prefix.That is, will index prefix and data Key is combined, generation index Key, is then indexed using indexing Key and index Value corresponding with index Key and generate one Data, the composition of the index Key in the present embodiment in constructed index data are as shown in table 1 below.
Table 1
Identification field Partition code The generation time of data Data Key
The index data generation method provided in the present embodiment, the generation of more datas by obtaining period memory storage Time, according to the generation time of pieces of data, determine multiple times of partition code corresponding to pieces of data, partition code and period Subregion is corresponding, and the partition code corresponding to different time subregion is different, according to partition code corresponding to pieces of data and pieces of data Generation a plurality of index data corresponding with pieces of data, wherein partition code are included in index data.The present embodiment is in structure rope Argument according to when increase a partition code, subregion is carried out to continuous index data according to the burst code, different partition codes causes Index data in identical product section of lower continuous time is dispersed on different machines so that index data have preferably from Divergence, and then can solve the problem that existing concordance list has the problem of data focus because dispersion is poor.
Embodiment two
As shown in figure 3, it is the schematic flow sheet for the index data generation method that the embodiment of the present invention two provides.The index Data creation method comprises the following steps：
S201, receive data write request.
In practical application, a plurality of index data may be constructed concordance list, and concordance list can be used within the storage system, is used for Two level inquiry is carried out to the real data of storage.In the present embodiment, with reference to generation of the actual data to index data and deposit Storage illustrates.When attempting to store actual data, storage system can receive data write request.In practical application, number According to this user mark or user mark plus object to be checked be identified as according to carry out subregion, so the dispersion of data compared with Good, the data of general different user can be stored in different positions, it is not easy to hot issue occur.In the present embodiment, first Data storage, then store index data.
S202, write data into storage system.
S203, the generation time for obtaining the more datas stored.
Generation time, the user of user corresponding to data of the data can be carried in practical application, in actual data Mark, and the information such as object identity of query object corresponding to data.Storage system, can be with after actual data have been stored Get the generation time of each data stored.Generally, the real data stored in storage system is one continuous Data of the different user to query object service condition in period.
S204, the generation time according to each data, determine partition code corresponding to pieces of data.
The process of tectonic reverse code can be found in the record of related content in above-described embodiment one, and this is repeated no more.
S205, partition code generates index data corresponding with pieces of data according to corresponding to pieces of data and pieces of data.
Wherein, partition code is included in index data.
The process of construction index data can be found in the record of related content in above-described embodiment, and this is repeated no more.
S206, by index datastore into storage system.
, can be according to multiple fields included in index prefix during index data is generated in practical application Information combination generation index Key, will according to index Key natural order when existing storage system stores a plurality of index data All index data stores after carrying out cutting, it may for example comprise 100 index datas, index Key is from 1~100, storage system Index Key from 1 to 30 is stored on a machine, will store onto another machine, will be deposited from 61 to 100 from 31 to 60 Store up onto the 3rd machine.
Compared with existing storage system, in the present embodiment when in the present embodiment in prefix is indexed increase a partition code Afterwards, the value of Key corresponding to original index data will change, so as to break original index Key continuous nature Sequence so that index data is fallen on different machines, and in the present embodiment, multiple index datas corresponding to same partition code can quilt Store on same database or platform machine.That is, different partition codes can correspond to disparate databases or deposit Store up space., can be multiple by a plurality of index datastore according to the partition code in the index data after index data is generated In database or memory space.
Using query object as a certain product a, exemplified by 3600~10800s continuous time sections, the index data of generation is carried out Storage, as shown in Figure 4.3600~10800 this period in Fig. 4, the default time step in Fig. 3 is 3600, to life Calculated into time 3600 and generation time 4000, obtained time stepping information is identical, when the generation time 3600 is with generating Between 4000 corresponding to partition codes 1.And generate the time for 8000 when, partition code 2 corresponding to the time stepping information that is calculated, When to generate the time be 12000, this continuous time of partition code 3,3600~10800 corresponding to the time stepping information that is calculated Corresponding 2 partition codes of section：Partition code 1 and partition code 2, wherein, memory space corresponding to partition code 1 is machine A, then partition code 2 Corresponding memory space is machine B.That is different partition codes causes the data quilt in identical product section of lower continuous time It is distributed on different machines, realizes discrete storage, reduces data focus.
In the present embodiment, increase a partition code when building concordance list, according to the partition code to the continuous of query object Index data in period carries out subregion so that index data has preferable dispersion, is stored to index data When according to partition code will in index datastore to corresponding position, reduce by continuous time section index data write-in it is same The probability of individual machine, and then can solve the problem that existing concordance list has the problem of data focus because dispersion is poor.
Embodiment three
As shown in figure 5, it is the schematic flow sheet for the data query method that the embodiment of the present invention three provides.The data query Method comprises the following steps：
Query time scope in S301, acquisition inquiry request.
In the present embodiment, the index data generation method that storage system is provided according to above-described embodiment one and two builds rope Argument evidence, then using the index data structure concordance list of generation, by concordance list storage within the storage system, in order to user's profit Index of reference table carries out secondary index to real data., can when user attempts to inquire about the real data in storage system To be asked to server input inquiry corresponding to storage system, server can be inquired about data according to the inquiry request. Because the index data that concordance list is formed in the present embodiment includes partition code, server can get from inquiry request and look into Time range is ask, to generate partition code according to the query time scope.
S302, according to query time scope, determine multiple partition codes, and inquiry request is decomposed into according to partition code multiple Subquery is asked.
Wherein partition code is corresponding with multiple time subregions in the range of the query time of data to be checked, different time point Partition code corresponding to area is different.
Specifically, server is rounded with the initial time of default time step from query time scope, until during inquiry Between scope the termination time untill, generate multiple time stepping information.
The initial time of query time scope is rounded first to obtain a time stepping letter corresponding with the initial time Breath, a default time step is then added in the initial time of query time scope, obtains next time rounded Value, a time stepping information is worth to according to the time, is then proceeded to default plus one on the basis of the time value Time step, next time value rounded is obtained, untill the termination time of query time scope.According to the method described above may be used To obtain multiple time stepping information corresponding to query time scope, then multiple subregions are generated according to multiple time stepping information Code.
Specifically, to each time stepping information carry out MD5 calculating, obtain time stepping information corresponding to MD5 values, take The numerical value on position is preset in MD5 values as partition code, such as takes latter 3 of MD5 values to be used as partition code.Alternatively, to it is each when Between stepping information carry out hash calculating, obtain the time stepping information corresponding to hash values, take in hash values the number on default position Value is used as partition code, such as takes latter 5 of hash values to be used as partition code.
In the present embodiment, server can resolve into inquiry request according to partition code after multiple partition codes are generated Multiple subquery requests.In practical application, the mark of object to be checked is also included in inquiry request.Server is asked according to inquiry Sum partition code, when obtaining subquery request, wherein multiple times in the range of the query time of partition code and data to be checked Subregion is corresponding, and the partition code corresponding to different time subregion is different.
For example, the data that user attempts to inquire about in a certain product section time are inquired about, the inquiry request of user's input The middle mark for needing to carry the product and query time scope, the mark of product can be the title of the product, for example, inquiry Request can be product：a；3600≤time ＜ 10800.After partition code is got, inquiry request can be resolved into Multiple subquery requests, subquery request product is shown on the left of Fig. 4：a；Partition code 1；3600≤time ＜ 10800, And product：a；Partition code 2；3600≤time ＜ 10800.
S303, the corresponding relation according to pre-recorded partition code and database or memory space, determine each partition code Corresponding database or memory space.
In the present embodiment, different partition codes corresponds to different database or memory space, and index data includes rope Drawing Key and data Key, index Key includes indexing prefix, wherein, index prefix includes partition code.Server is in storage rope Argument according to when, in order to realize to index data carry out partitioned storage, index data is stored according to partition code.Server exists By concordance list storage into storage system after, can be with record partitioning code and database or the corresponding relation of memory space, can be with Database or memory space according to corresponding to the corresponding relation determines each partition code.
S304, in the database or memory space corresponding to each partition code, looked into according to son corresponding with each partition code Ask the inquiry that request carries out data.
Server can according to partition code in being asked to subquery, inquire with the database corresponding to the partition code or Memory space, asked to carry out the inquiry of data according to subquery corresponding with each partition code.
Specifically, positioning database corresponding with the partition code in subquery request or memory space, can search number According to being stored in storehouse or memory space and the corresponding index data of subquery request.It is corresponding with subquery request finding After index data, the index Key that each index data includes can be got.In practical application, in order to ensure to index Key's Uniqueness, structure index Key when can index prefix behind increase data Key, wherein, index prefix be include it is to be checked right Identification field, partition code and the query time scope of elephant.Because index Key and data Key has corresponding relation, so obtaining While getting the index Key corresponding to subquery request, data Key corresponding to subquery request has just been obtained.Carrying out rope When drawing inquiry, the data Key for meeting inquiry request can be obtained, in the data storage of reality, according to data Key- data Value mode is stored, after data Key is got, it is possible to gets the data that corresponding relation be present with data Key Value。
The data query method that the present embodiment provides, by obtaining the query time scope in inquiry request, according to described Query time scope, multiple partition codes are determined, and inquiry request is decomposed into multiple subqueries according to partition code and asked, partition code It is corresponding with multiple time subregions in the range of the query time of the data to be checked, the subregion corresponding to different time subregion Code is different；According to pre-recorded partition code and the corresponding relation of database or memory space, determine corresponding to each partition code Database or memory space, in the database or memory space corresponding to each partition code, according to each partition code pair The subquery request answered carries out the inquiry of data.In the present embodiment, when being inquired about based on partition code data, due to index number Stored according in storage according to partition code, i.e., large-scale continuous index data in concordance list has been carried out into subregion deposits Storage, and then large-scale inquiry mode is changed into the inquiry in multiple partition codes, realize the continuous of a wide range of high-volume data Inquiry, and complicated processing is carried out without outside, improve search efficiency.
Example IV
As shown in fig. 6, it is the schematic flow sheet for the data query method that the embodiment of the present invention four provides.The data query Method comprises the following steps：
S401, receive inquiry request.
When attempting to inquire about data, user can send inquiry request, receive the inquiry request of the transmission of user, Can log-on data querying flow.Generally, when mark and the inquiry of object to be checked are at least carried in inquiry request Between scope.Alternatively, current time stepping information and default time step can also be carried in inquiry request, for table Show the default record number N of inquiry data Key total numbers and paging mark etc. every time.For example, object to be checked is a certain production Identification field is product=a during product, the time range of inquiry：time>=3600, time<10800, current time stepping Information can the inquiry of continuous data be changed into the inquiry in single partition code, default time step in interruption when will be a wide range of For 3600.
S402, judge current time stepping information whether is carried in inquiry request.
If current time stepping information is sky in inquiry request, S403 is performed i.e. by the starting of query time scope After time rounds, as current time stepping information, getting current time stepping information and then performing S404. Generally, when not carrying current time stepping information in the inquiry request that user directly inputs, when server according to After the inquiry request has carried out the inquiry of a partition code, a time stepping information of return can be received, the return Time stepping information will act as a new input parameter, carry in inquiry request.Further, server continues to inquire about Shi Ze carries out data query according to the inquiry request for carrying time stepping information.
If current time stepping information is not sky in inquiry request, S404 is directly performed i.e. according to the current time Stepping information directly calculate corresponding to partition code.
S403, using the initial time of query time scope as current time stepping information.
S404, partition code calculated according to current time stepping information.
The MD5 values of current time stepping information are calculated, take latter 3 of MD5 values to be used as partition code.Alternatively, calculate and work as The hash values of preceding time stepping information, latter 3 of hash values are taken to be used as partition code.
Further, according to current time stepping information and default time step, next time can be calculated Stepping information.
S405, subquery request corresponding with the partition code generated according to inquiry request and partition code.
The identification field and query time scope of object to be checked are obtained from the inquiry request received, is being got After identification field, partition code and query time scope, then subquery request corresponding with the partition code, this implementation are may be constructed In example, the order that each field is asked in subquery is identification field, partition code and query time scope.Generally by identification field In the first place of subquery request, it becomes possible to realize the number used under inquiry product under all user's specified time sections product According to.
For example, if the current stepping position in condition is sky, using the initial value 3600 of range field as current step Carry, is calculated partition code 1, and subquery request includes：Initial value is " product:a；Partition code 1；Time_3600 " is opened Begin, including 3600 in itself.End value is " product:a；Partition code 1；Time_10800 ", wherein not including 10800 Itself.
If current time stepping information is 7200, partition code 2 is calculated, subquery request includes：Initial value is “product:a；Partition code 2；Time_3600 " starts, including 3600 in itself.End value is " product:a；Partition code 2；Time_10800 ", wherein not including 10800 in itself.So far, a wide range of of inquiry is narrowed down into two partition codes It is interior, realize the small range inquiry in partition code.
S406, data in database corresponding with partition code or memory space are indexed according to subquery request Inquiry.
In the present embodiment, different partition codes corresponds to different database or memory space, and index data includes rope Drawing Key and data Key, index Key includes indexing prefix, wherein, index prefix includes partition code.Server is in storage rope Argument according to when, in order to realize to index data carry out partitioned storage, index data is stored according to partition code.Server exists By concordance list storage into storage system after, can be with record partitioning code and database or the corresponding relation of memory space, can be with Database or memory space according to corresponding to the corresponding relation determines each partition code.
Server can according to partition code in being asked to subquery, inquire with the database corresponding to the partition code or Memory space, the inquiry of data is indexed according to subquery corresponding with each partition code request, obtains meeting subquery request Index data.
Specifically, positioning database corresponding with the partition code in subquery request or memory space, can search number According to being stored in storehouse or memory space and the corresponding index data of subquery request.Wherein index data includes indexing Key With index Value, in practical application, continue to inquire about data Value according to index Key, therefore, server is looked into son finding After asking index data corresponding to request, the index Key that the index data inquired includes can be returned.
Whether S407, the record number for the index Key for judging to return are beyond default record number N.
If it is judged that the index Key returned record number is more than default record number N, illustrate current time stepping letter Index Key corresponding to breath in partition code is not run through, then performs S408 and continue in subregion corresponding to current time stepping information In code, inquired about according to subquery request, to obtain new index Key；If it is judged that the index Key returned record number Less than or equal to default record number, illustrate that the index Key corresponding to current time stepping information in partition code has been read, Then perform S411.Wherein, default record number is carried in inquiry request, for example, record number N is 500, when record number surpasses When 500, illustrate that needs continue to be inquired about in partition code.
S408, mark using the N+1 articles index Key as new paging.
Generally, the empty paging mark carried in inquiry request, when needing to carry out paging query to partition code When, the parameter that can receive return marks as paging, such as can be when being read out to index Key, more than pre- If record number after it is down more read an index Key, using index Key as paging mark.
S409, since paging mark corresponding to the N+1 articles index Key continue in partition code according to subquery ask into Row is inquired about to obtain new index Key.
When partition code needs paging query, the starting point inquired about after paging is able to know that according to paging mark, from this Point starts to continue in partition code, is inquired about according to the subquery request of structure, to obtain new index Key.The present embodiment In, the starting point of paging query is the next index Key beyond default record number, i.e. starting point is exactly N+1 bars index Key.
S410, the data Value according to corresponding to data Key included in the index Key got.
In practical application, in order to ensure to index Key uniqueness, increase data Key, such rope when structure indexes Key Draw Key and data Key and corresponding relation be present.Correspondingly, after all index Key in partition code are got, it is possible to obtain All data Key in partition code.In the data storage of reality, stored in the way of Key-Value, to subregion After the completion of all data Key in code are read, data Value is inquired according to data Key single-points.
After the completion of S411, the index Key in partition code corresponding to current time stepping information are obtained, according to current Time stepping information, default step-length and query time scope obtain next time stepping information.
S412, using next time stepping information as new current time stepping information.
After the completion of index Key in partition code corresponding to current time stepping information is obtained, according to the current time Stepping information and default time step can calculate next time stepping information, if the next time calculated Stepping information can be returned without departing from query time scope, then next time stepping information, and by next time Stepping information returns as new current time stepping information and performs S402.If the next time stepping letter calculated Breath exceeds query time scope, then next time stepping information can not be returned, it is necessary to by current time stepping information It is arranged to empty.
The data query method that the present embodiment provides, the partition code inquired about by being configured to subregion in query process, According to partition code to corresponding storage location, the inquiry of table is indexed according to subquery corresponding with partition code request, is obtained To index Key, so as to obtain and index the data Key of the corresponding relation of Key presence, and then number corresponding with data Key is obtained According to Value.In the present embodiment, when being inquired about based on partition code data, because concordance list enters in storage according to partition code Row storage, will carry out partitioned storage by continuous index data, and then large-scale inquiry mode is changed on a large scale Inquiry in multiple partition codes, and paging in partition code is realized by default record number and paging mark when subregion is inquired about Inquiry, and then the continuous-query of high-volume data is realized, and complicated processing is carried out without outside, improve search efficiency.
Embodiment five
As shown in fig. 7, it is the structural representation for the storage system that the embodiment of the present invention five provides.The storage system bag Include：Storage server 11, at least one the first database 12 for being used to store index data, and it is at least one for storing reality Second database 13 of border data.Storage server 11 is connected with the first database 12 and the second database 13 respectively.This reality Apply in example, alternatively, storage server 11 can be by index data and actual data storage in memory space.
In practical application, storage server 11, can be according to Key-Value forms, according to actual number when storing real data According to Regionalization basis, store data in the second database 13.Wherein, the generation of the data can be carried in the data of storage Time, data correspond to user's mark of user, and the information such as object identity of query object corresponding to data.It is for example, actual Data represent the service condition of a certain product, and the service condition can reflect that different user makes to the product in different time With.
In the present embodiment, after storage server 11 is by the second databases of data Cun Chudao 13 all in the period, Storage server 11 can get the generation time of all data in the period.
In the present embodiment, storage server 11 is after the generation time of pieces of data is got, according to the life of pieces of data Into the time, partition code corresponding to pieces of data is determined.In the present embodiment, on the generating process of partition code, reference can be made to above-mentioned reality The record of related content in example one is applied, here is omitted.
, can be according to the object carried in pieces of data after storage server 11 gets the partition code included by the period Mark, generation time and corresponding partition code, generate index data corresponding with pieces of data.
In order to realize secondary index, in storage server 11 after by the second databases of data Cun Chudao 13 of reality, Storage server 11 builds a concordance list using the index data of generation, then concordance list is stored, i.e., to index number According to being stored.Storage server 11 can be discrete by continuous index data according to partition code included in index data Store in multiple first databases 12.In the present embodiment, index data includes index Key and rope corresponding with index Key Draw Value.In order to ensure index Key uniqueness, index Key include data Key, then data Key and with the data Index prefix corresponding to Key forms an index Key.
Further, storage server 11 is in the partition code in index data, by the scattered storage of index data to more , can be with the corresponding relation between record partitioning code and the first database 12 after in individual first database 12.In the present embodiment, Partition code identical index datastore is in the first database of identical 12.
The storage system that the present embodiment provides can be used for the flow for performing the data query method shown in Fig. 2 and 3, and its is specific Operation principle repeats no more, and refers to the description of embodiment of the method.
The storage system provided in the present embodiment, the generation time of more datas by obtaining period memory storage, root According to the generation time of pieces of data, partition code corresponding to pieces of data, partition code and multiple time subregion phases of period are determined Corresponding, the partition code corresponding to different time subregion is different, according to partition code generation corresponding to pieces of data and pieces of data with Index data corresponding to pieces of data, wherein partition code are included in index data.The present embodiment increases when building index data Add a partition code, subregion is carried out to continuous index data according to the burst code, different partition codes causes under identical product Index data in continuous time section is dispersed on different machines so that and index data has preferable dispersion, and then It can solve the problem that existing concordance list has the problem of data focus because dispersion is poor.
Embodiment six
As shown in figure 8, it is the structural representation for the inquiry system that the embodiment of the present invention six provides.The inquiry system bag Include：Inquiry server 21, it is at least one be used for store index data the first database 22 and it is at least one be used for store reality Second database 23 of border data.Wherein, inquiry server 21 is connected with the first database 22 and the second database 23 respectively. In the present embodiment, alternatively, storage server can be by index data and actual data storage in memory space.
The index data generation method that the inquiry system that the present embodiment is provided is provided according to above-described embodiment one and two Index data is built, then using the index data structure concordance list of generation, concordance list is stored in the first database 22, with It is easy to user to carry out secondary index to data using concordance list.
, can be to inquiry server 21 when the real data stored during user attempts to the second database 23 is inquired about Input inquiry is asked, and inquiry server 21 can be inquired about data according to the inquiry request.Due in the first database 22 The index data of storage includes partition code, and in order to realize the inquiry to index data, inquiry server 21 first please from inquiry Query time scope is got in asking, to determine partition code according to the query time scope, inquires about server 21 by inquiry request Multiple subquery requests are decomposed into according to partition code.Partition code and the mistake of subquery request are determined on inquiry server 21 Journey, reference can be made in above-described embodiment one and two related content record, here is omitted.
In the present embodiment, different partition codes corresponds to the first different databases 22.Index data include index Key and Data Key, index Key include indexing prefix, wherein, index prefix includes partition code.Due in the first database 22 When storing index data, in order to realize the partitioned storage to index data, index data is stored according to partition code.To After the concordance list being made up of index data is stored into the first database 22, partition code and number of the server 21 according to record are inquired about According to storehouse or the corresponding relation of memory space, the first database 22 corresponding to each partition code is determined.
Inquiry server 21 can inquire and first corresponding to the partition code according to partition code in being asked to subquery Database 22, the inquiry of data is indexed according to subquery corresponding with each partition code request.
Finding each index number can be got with after the corresponding index data of subquery request, inquiring about server 21 According to the index Key included.
Because real data is stored in the way of Key-Value in the second database 23, server 21 is inquired about After data Key is got, inquired about according to data Key cans in the second database 23, to get and data Key The data Value of corresponding relation be present.
In the present embodiment, inquiry server 32 can carry out paging query, and detailed process can be found in phase in above-described embodiment The record held inside the Pass, here is omitted.
The inquiry system that the present embodiment provides can be used for the flow for performing the data query method shown in Fig. 5~6, and its is specific Operation principle repeats no more, and refers to the description of embodiment of the method.
The inquiry system that the present embodiment provides, by obtaining the query time scope in inquiry request, according to the inquiry Time range determines multiple partition codes, and inquiry request is decomposed into multiple subqueries according to partition code and asked, partition code and institute The multiple time subregions stated in the range of the query time of data to be checked are corresponding, and the partition code corresponding to different time subregion is not Together；According to pre-recorded partition code and the corresponding relation of database or memory space, the number corresponding to each partition code is determined According to storehouse or memory space, in the database or memory space corresponding to each partition code, according to corresponding with each partition code Subquery request carries out the inquiry of data.In the present embodiment, when being inquired about based on partition code data, because index data exists Stored during storage according to partition code, partitioned storage will have been carried out by continuous index data on a large scale, and then by big model The inquiry mode enclosed is changed into the inquiry in multiple partition codes, realizes the continuous-query of a wide range of high-volume data, and without outer Portion carries out complicated processing, improves search efficiency.
Embodiment seven
As shown in figure 9, it is the structural representation for the data storage and query system that the embodiment of the present invention seven provides.The number Include according to storage and inquiry system：Storage server 31, inquire about server 32, be at least one for storing the first of index data Database 33 and at least one the second database 34 for being used to store real data.Wherein, the first database 33 and the second data Storehouse 34 is connected with storage server 31, and the first database 33 and the second database 34 are connected with inquiry server 32.
Wherein, storage server 31 is used to generate the index data for including partition code, and by index data according to partition code Store in corresponding first database 33.The detailed process of index data is generated and stored on storage server 31, can be joined See the record of related content in above-described embodiment one and two, here is omitted.
Storage server 31 is additionally operable to the Regionalization basis according to real data, by actual data storage to corresponding second number According in storehouse 34.Wherein, real data can be according to uid or uid+product as Regionalization basis.
Server 32 is inquired about after the completion of index datastore, can be according to the inquiry request that user inputs to being stored in the Data in two databases 34 are inquired about.Server 32 is inquired about first according to inquiry request to the index in the first database 33 Data are inquired about, the process inquired about on inquiry server 32 index data in the first database 33, reference can be made to The record of related content in above-described embodiment three and four, here is omitted.After inquiry server 32 gets index data, The second database 34 can be inquired about according to data Key included in Key is indexed in index data, with get with Data Value corresponding to data Key, that is, it is stored in the second database 34, real data corresponding with inquiry request.
Herein it is to be appreciated that storage server 31 and inquiry server 32 can have a server to perform.
The present embodiment first generates the index data for building concordance list in the storing process of concordance list, in generation rope Argument according to when increase a partition code, subregion is carried out to continuous index data according to the burst code, different partition codes causes Index data in identical product section of lower continuous time is dispersed on different machines so that index data have preferably from Divergence, and then can solve the problem that existing concordance list has the problem of data focus because dispersion is poor.
Further, when being inquired about based on partition code data, because index data enters in storage according to partition code Row storage, will carry out partitioned storage by continuous index data, and then large-scale inquiry mode is changed on a large scale Inquiry in multiple partition codes, the continuous-query of a wide range of high-volume data is realized, and complicated processing is carried out without outside, carried High search efficiency.
Embodiment eight
As shown in Figure 10, the structural representation of its index data generating means provided for the embodiment of the present invention eight.The rope Drawing data generating device includes：Acquisition module 41, partition code generation module 42 and data generation module 43.
Wherein, acquisition module 41, the generation time of more datas for obtaining period memory storage.
Partition code generation module 42, for the generation time according to pieces of data, partition code corresponding to pieces of data is determined, The partition code is corresponding with multiple time subregions of the period, and the partition code corresponding to different time subregion is different；
Data generation module 43, for the partition code generation according to corresponding to the pieces of data and pieces of data and each bar number According to corresponding index data, wherein the partition code is included in the index data.
Further, the index data generating means in the present embodiment also include memory module 44, for according to the rope Partition code of the argument in, by the scattered storage of the index data into multiple databases or memory space, and described in record Partition code and the database or the corresponding relation of memory space, wherein, partition code identical index datastore is identical Database or memory space in.
Partition code generation module 42, specifically for being carried out with generation time of the default time step to the pieces of data Round, determine time stepping information, the partition code is generated according to the time stepping information.
Partition code generation module 42, calculated specifically for carrying out MD5 values for the time stepping information, generate MD5 values, The numerical value preset in the MD5 values on position is taken, as the partition code.
The storage system that the present embodiment provides can be used for the flow for performing the data query method shown in Fig. 2 and 3, and its is specific Operation principle repeats no more, and refers to the description of embodiment of the method.
The present embodiment, in the storing process of concordance list, the index data for building concordance list is first generated, in generation rope Argument according to when increase a partition code, subregion is carried out to continuous index data according to the burst code, different partition codes causes Index data in identical product section of lower continuous time is dispersed on different machines so that index data have preferably from Divergence, and then can solve the problem that existing concordance list has the problem of data focus because dispersion is poor.
Embodiment nine
As shown in figure 11, the structural representation of its data query arrangement provided for the embodiment of the present invention eight.The data are looked into Asking device includes：Acquisition module 51, generation module 52, determining module 53 and enquiry module 54.
Acquisition module 51, for obtaining the query time scope in inquiry request.
Generation module 52, for according to the query time scope, determining multiple partition codes, and the inquiry request is pressed Be decomposed into the request of multiple subqueries according to partition code, the partition code with it is multiple in the range of the query time of the data to be checked Time subregion is corresponding, and the partition code corresponding to different time subregion is different.
Determining module 53, for according to the corresponding of the pre-recorded partition code and the database or memory space Relation, determine database or memory space corresponding to each partition code.
Enquiry module 54, in the database or memory space corresponding to each partition code, according to each partition code The corresponding subquery request carries out the inquiry of data.
Further, generation module 52, specifically for the starting with default time step from the query time scope Time was rounded untill the termination time of the query time scope, multiple time stepping information was generated, according to the multiple Time stepping information generates multiple partition codes.
Generation module 52, calculated specifically for carrying out MD5 values for the multiple time stepping information, generate multiple MD5 Value, the numerical value preset in the multiple MD5 values on position is taken respectively, as corresponding more respectively with the multiple time stepping information Individual partition code.
A kind of alternatively frame mode of enquiry module 54 in the present embodiment, including：
Searching unit, in the database or memory space corresponding to each partition code, searching and each partition code pair The index data corresponding to subquery request answered.
Acquiring unit, the index Key included for getting each index data, wherein, the index Key includes Data Key.
Query unit, for inquiring about the data Value that corresponding relation be present with the data Key.
The inquiry system that the present embodiment provides can be used for the flow for performing the data query method shown in Fig. 5~6, and its is specific Operation principle repeats no more, and refers to the description of embodiment of the method.
In the present embodiment, when being inquired about based on partition code data, due to index data storage when according to partition code Stored, continuity index data will carry out partitioned storage on a large scale, and then large-scale inquiry mode is changed into Inquiry in multiple partition codes, the continuous-query of high-volume data is realized, and complicated processing is carried out without outside, raising is looked into Ask efficiency.
Embodiment ten
Above-described embodiment eight describes the built-in function and structure of index data generating means, as shown in figure 12, in practice, The index data generating means, including：Memory 61 and processor 62.
Memory 61, storage program can be configured as.
Memory 61 can realize by any kind of volatibility or non-volatile memory device or combinations thereof, such as Static RAM (SRAM), Electrically Erasable Read Only Memory (EEPROM), erasable programmable is read-only to be deposited Reservoir (EPROM), programmable read only memory (PROM), read-only storage (ROM), magnetic memory, flash memory, disk or CD.
Memory 61 and processor 62 can be connected with each other by bus and complete mutual communication.The bus can be with It is industry standard architecture (Industry Standard Architecture, referred to as ISA) bus, external equipment interconnection (Peripheral Component, referred to as PCI) bus or extended industry-standard architecture (Extended Industry Standard Architecture, referred to as EISA) bus etc..The bus can be divided into address bus, data/address bus, control Bus processed etc..For ease of representing, only represented in Figure 12 with a thick line, it is not intended that an only bus or a type of Bus.
Processor 62, for performing the program in memory 61, for：
Obtain the generation time of more datas of period memory storage；
According to the generation time of pieces of data, determine partition code corresponding to the pieces of data, the partition code with it is described Multiple time subregions of period are corresponding, and the partition code corresponding to different time subregion is different；
Index data corresponding with pieces of data is generated according to partition code corresponding to the pieces of data and pieces of data, its Described in partition code be included in the index data in.
Further, processor 62 is specifically used for：According to the partition code in the index data, by the index data point Storage is dissipated into multiple databases or memory space, and records the partition code and the database or pair of memory space It should be related to, wherein, partition code identical index datastore is in identical database or memory space.
Further, processor 62 is specifically used for：
Rounded with generation time of the default time step to the pieces of data, generate time stepping information；
The partition code is generated according to the time stepping information.
Further, processor 62 is specifically used for：
MD5 values are carried out to the time stepping information to calculate, and generate MD5 values；
The numerical value preset in the MD5 values on position is taken, as the partition code.
The present embodiment first generates the index data for building concordance list in the storing process of concordance list, in generation rope Argument according to when increase a partition code, subregion is carried out to continuous index data according to the burst code, different partition codes causes Index data in identical product section of lower continuous time is dispersed on different machines so that index data have preferably from Divergence, and then can solve the problem that existing concordance list has the problem of data focus because dispersion is poor.
Embodiment 11
Above-described embodiment nine describes the built-in function and structure of index data generating means, as shown in figure 13, in practice, The data query arrangement, including：Memory 71 and processor 72.
Memory 71, storage program can be configured as.
Memory 71 can realize by any kind of volatibility or non-volatile memory device or combinations thereof, such as Static RAM (SRAM), Electrically Erasable Read Only Memory (EEPROM), erasable programmable is read-only to be deposited Reservoir (EPROM), programmable read only memory (PROM), read-only storage (ROM), magnetic memory, flash memory, disk or CD.
Memory 71 and processor 72 can be connected with each other by bus and complete mutual communication.For ease of representing, Only represented in Figure 13 with a thick line, it is not intended that an only bus or a type of bus.
Processor 72, for performing the program in memory 71, for：
Obtain the query time scope in inquiry request；
According to the query time scope, multiple partition codes are determined, and the inquiry request is decomposed into according to partition code Multiple subquery requests, the partition code are relative with multiple time subregions in the range of the query time of the data to be checked Should, the partition code corresponding to different time subregion is different；
According to the pre-recorded partition code and the corresponding relation of the database or memory space, each subregion is determined Database or memory space corresponding to code；
In the database or memory space corresponding to each partition code, according to the subquery corresponding with each partition code Request carries out the inquiry of data.
Further, processor 71 is specifically used for：
Rounded with the initial time of default time step from the query time scope up to the query time scope The termination time untill, determine multiple time stepping information；
Multiple partition codes are generated according to the multiple time stepping information.
Further, processor 71 is specifically used for：
MD5 values are carried out for the multiple time stepping information to calculate, and generate multiple MD5 values；
The numerical value preset in the multiple MD5 values on position is taken respectively, as right respectively with the multiple time stepping information The multiple partition codes answered.
Further, processor 71 is specifically used for：
In the database or memory space corresponding to each partition code, the subquery corresponding with each partition code is searched The corresponding index data of request；
The index Key that each index data includes is got, wherein, the index Key includes data Key；
Inquire the data Value that corresponding relation be present with the data Key.
In the present embodiment, when being inquired about based on partition code data, due to index data storage when according to partition code Stored, continuity index data will carry out partitioned storage on a large scale, and then large-scale inquiry mode is changed into Inquiry in multiple partition codes, the continuous-query of high-volume data is realized, and complicated processing is carried out without outside, raising is looked into Ask efficiency.
One of ordinary skill in the art will appreciate that：Realizing all or part of step of above-mentioned each method embodiment can lead to The related hardware of programmed instruction is crossed to complete.Foregoing program can be stored in a computer read/write memory medium.The journey Sequence upon execution, execution the step of including above-mentioned each method embodiment；And foregoing storage medium includes：ROM, RAM, magnetic disc or Person's CD etc. is various can be with the medium of store program codes.
Finally it should be noted that：Various embodiments above is merely illustrative of the technical solution of the present invention, rather than its limitations；To the greatest extent The present invention is described in detail with reference to foregoing embodiments for pipe, it will be understood by those within the art that：Its according to The technical scheme described in foregoing embodiments can so be modified, either which part or all technical characteristic are entered Row equivalent substitution；And these modifications or replacement, the essence of appropriate technical solution is departed from various embodiments of the present invention technology The scope of scheme.

Claims (26)

  1. A kind of 1. generation method of index data, it is characterised in that including：
    Obtain the generation time of more datas of period memory storage；
    According to the generation time of pieces of data, partition code corresponding to the pieces of data, the partition code and the time are determined Multiple time subregions of section are corresponding, and the partition code corresponding to different time subregion is different；
    Index data corresponding with pieces of data, wherein institute are generated according to partition code corresponding to the pieces of data and pieces of data Partition code is stated included in the index data.
  2. 2. generation method according to claim 1, it is characterised in that also include：
    According to the partition code in the index data, multiple databases or memory space are arrived into the scattered storage of the index data In, and the partition code and the database or the corresponding relation of memory space are recorded, wherein, partition code identical index number According to being stored in identical database or memory space.
  3. 3. generation method according to claim 1, it is characterised in that according to the generation time of pieces of data, it is determined that described Partition code includes corresponding to pieces of data：
    Rounded with generation time of the default time step to the pieces of data, generate time stepping information；
    The partition code is generated according to the time stepping information.
  4. 4. generation method according to claim 3, it is characterised in that the temporal information generation after being rounded described in the basis The partition code includes：
    MD5 values are carried out for the time stepping information to calculate, and generate MD5 values；
    The numerical value preset in the MD5 values on position is taken, as the partition code.
  5. 5. generation method according to claim 1, it is characterised in that the index data include index Key and with the rope Draw index Value, the index Key corresponding to Key including data Key and it is corresponding with data Key index prefix, described point Area's code is included in the index prefix.
  6. 6. according to the method for claim 5, it is characterised in that the index prefix also includes：Inquiry pair corresponding to data The identification field of elephant, the generation temporal information of data.
  7. A kind of 7. querying method of data, it is characterised in that including：
    Obtain the query time scope in inquiry request；
    According to the query time scope, multiple partition codes are determined, and the inquiry request is decomposed into according to partition code multiple Subquery is asked, and the partition code is corresponding with multiple time subregions in the range of the query time of the data to be checked, no It is different with the partition code corresponding to time subregion；
    According to the pre-recorded partition code and the corresponding relation of the database or memory space, each partition code institute is determined Corresponding database or memory space；
    In the database or memory space corresponding to each partition code, asked according to the subquery corresponding with each partition code Carry out the inquiry of data.
  8. 8. querying method according to claim 7, it is characterised in that it is described according to the query time scope, it is determined that more Individual partition code includes：
    Rounded with the initial time of default time step from the query time scope up to the end of the query time scope Only untill the time, multiple time stepping information are determined；
    Multiple partition codes are generated according to the multiple time stepping information.
  9. 9. querying method according to claim 8, it is characterised in that described to be generated according to the multiple time stepping information Multiple partition codes include：
    MD5 values are carried out for the multiple time stepping information to calculate, and generate multiple MD5 values；
    The numerical value preset in the multiple MD5 values on position is taken respectively, as corresponding respectively with the multiple time stepping information Multiple partition codes.
  10. 10. querying method according to claim 9, it is characterised in that the database corresponding to each partition code or In person's memory space, the inquiry of data is carried out according to the subquery request corresponding with each partition code, including：
    In the database or memory space corresponding to each partition code, the subquery request corresponding with each partition code is searched Corresponding index data；
    The index Key that each index data includes is got, wherein, the index Key includes data Key；
    Inquire the data Value that corresponding relation be present with the data Key.
  11. A kind of 11. storage system, it is characterised in that including：
    Storage server, at least one the first database or the first memory space and at least one for being used to store index data Individual the second database or the second memory space for being used to store real data, wherein, the storage server and described first Database either the first memory space connection and second database or second memory space connection；
    The storage server, for by actual data storage to second database or the second memory space, Yi Jiyong In the generation time for more datas that period memory storage is obtained from second database or second memory space, root According to the generation time of pieces of data, partition code corresponding to the pieces of data is determined, and according to the pieces of data and each bar number Index data corresponding with pieces of data is generated according to corresponding partition code, wherein the partition code is included in the index data In, the partition code is corresponding with multiple time subregions of the period, and the partition code corresponding to different time subregion is different.
  12. 12. storage system according to claim 11, it is characterised in that also include：
    The storage server, the partition code being additionally operable in the index data, the index data is disperseed into storage and arrived In multiple first databases or first memory space, and record the partition code and first database or The corresponding relation of first memory space, wherein, partition code identical index datastore in the database of identical first or In the memory space of person first.
  13. A kind of 13. inquiry system, it is characterised in that including：
    Inquiry server, at least one the first database or the first memory space and at least one for being used to store index data Individual the second database or the second memory space for being used to store real data；Wherein, it is described inquiry server respectively with it is described Either first memory space and the connection of second database or second memory space of first database；
    The inquiry server, for obtaining the query time scope in inquiry request, according to the query time scope, generation Multiple partition codes, and the inquiry request is decomposed into multiple subqueries according to partition code and asked, according to pre-recorded Partition code and first database or the corresponding relation of first memory space, are determined described corresponding to each partition code First database or first memory space, and in first database corresponding to each partition code or described In two memory spaces, according to the subquery request corresponding with each partition code in first database or first storage Index data corresponding to space querying, and deposited according to the index data inquired in second database or second Store up space and carry out data query；
    Wherein, the partition code is corresponding with multiple time subregions in the range of the query time of the data to be checked, different Partition code corresponding to time subregion is different.
  14. 14. inquiry system according to claim 13, it is characterised in that the inquiry server, specifically for each subregion In first database or first memory space corresponding to code, the subquery corresponding with each partition code is searched The corresponding index data of request, gets the index Key that each index data includes, wherein, the index Key includes The data Value of corresponding relation with the data Key be present in data Key, and inquiry.
  15. A kind of 15. index data generating means, it is characterised in that including：
    Acquisition module, the generation time of more datas for obtaining period memory storage；
    Partition code generation module, for the generation time according to pieces of data, determine partition code corresponding to the pieces of data, institute Multiple time subregions that partition code is stated with the period are corresponding, and the partition code corresponding to different time subregion is different；
    Data generation module, it is corresponding with pieces of data for the partition code generation according to corresponding to the pieces of data and pieces of data Index data, wherein the partition code be included in the index data in.
  16. 16. device according to claim 15, it is characterised in that also include：
    Memory module, for the partition code in the index data, multiple data are arrived into the scattered storage of the index data In storehouse or memory space, and the partition code and the database or the corresponding relation of memory space are recorded, wherein, subregion Code identical index datastore is in identical database or memory space.
  17. 17. device according to claim 15, it is characterised in that the partition code generation module, specifically for default Generation time of the time step to the pieces of data round, time stepping information is determined, according to the time stepping Information generates the partition code.
  18. A kind of 18. data query arrangement, it is characterised in that including：
    Acquisition module, for obtaining the query time scope in inquiry request；
    Generation module, for according to the query time scope, determining multiple partition codes, and by the inquiry request according to subregion Code is decomposed into multiple subquery requests, the partition code and multiple times point in the range of the query time of the data to be checked Area is corresponding, and the partition code corresponding to different time subregion is different；
    Determining module, for the corresponding relation according to the pre-recorded partition code and the database or memory space, Determine the database or memory space corresponding to each partition code；
    Enquiry module, in the database or memory space corresponding to each partition code, according to corresponding with each partition code The subquery request carries out the inquiry of data.
  19. 19. device according to claim 18, it is characterised in that the generation module, specifically for the default time The initial time of step-length from the query time scope is rounded untill the termination time of the query time scope, it is determined that more Individual time stepping information, multiple partition codes are generated according to the multiple time stepping information.
  20. 20. device according to claim 19, it is characterised in that the enquiry module, including：
    Searching unit, in the database or memory space corresponding to each partition code, searching corresponding with each partition code The corresponding index data of the subquery request；
    Acquiring unit, the index Key included for getting each index data, wherein, the index Key includes data Key；
    Query unit, for inquiring about the data Value that corresponding relation be present with the data Key.
  21. A kind of 21. index data generating means, it is characterised in that including：Memory and processor；
    The memory, for storage program；
    The processor, for performing described program, for：
    Obtain the generation time of more datas of period memory storage；
    According to the generation time of pieces of data, partition code corresponding to the pieces of data, the partition code and the time are determined Multiple time subregions of section are corresponding, and the partition code corresponding to different time subregion is different；
    Index data corresponding with pieces of data, wherein institute are generated according to partition code corresponding to the pieces of data and pieces of data Partition code is stated included in the index data.
  22. 22. generating means according to claim 21, it is characterised in that the processor is specifically used for：
    According to the partition code in the index data, multiple databases or memory space are arrived into the scattered storage of the index data In, and the partition code and the database or the corresponding relation of memory space are recorded, wherein, partition code identical index number According to being stored in identical database or memory space.
  23. 23. generating means according to claim 22, it is characterised in that the processor is specifically used for：
    Rounded with generation time of the default time step to the pieces of data, determine time stepping information；
    The partition code is generated according to the time stepping information.
  24. A kind of 24. data query arrangement, it is characterised in that including：Memory and processor；
    The memory, for storage program；
    The processor, for performing described program, for：
    Obtain the query time scope in inquiry request；
    According to the query time scope, multiple partition codes are determined, and the inquiry request is decomposed into according to partition code multiple Subquery is asked, and the partition code is corresponding with multiple time subregions in the range of the query time of the data to be checked, no It is different with the partition code corresponding to time subregion；
    According to the pre-recorded partition code and the corresponding relation of the database or memory space, each partition code institute is determined Corresponding database or memory space；
    In the database or memory space corresponding to each partition code, asked according to the subquery corresponding with each partition code Carry out the inquiry of data.
  25. 25. device according to claim 24, it is characterised in that the processor is specifically used for：
    Rounded with the initial time of default time step from the query time scope up to the end of the query time scope Only untill the time, multiple time stepping information are determined；
    Multiple partition codes are generated according to the multiple time stepping information.
  26. 26. device according to claim 25, it is characterised in that the processor is specifically used for：
    In the database or memory space corresponding to each partition code, the subquery request corresponding with each partition code is searched Corresponding index data；
    The index Key that each index data includes is got, wherein, the index Key includes data Key；
    Inquire the data Value that corresponding relation be present with the data Key.
CN201610720018.8A 2016-08-24 2016-08-24 Index data generation and data query method and device, and storage and query system Active CN107783980B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610720018.8A CN107783980B (en) 2016-08-24 2016-08-24 Index data generation and data query method and device, and storage and query system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610720018.8A CN107783980B (en) 2016-08-24 2016-08-24 Index data generation and data query method and device, and storage and query system

Publications (2)

Publication Number Publication Date
CN107783980A true CN107783980A (en) 2018-03-09
CN107783980B CN107783980B (en) 2021-10-19

Family

ID=61393134

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610720018.8A Active CN107783980B (en) 2016-08-24 2016-08-24 Index data generation and data query method and device, and storage and query system

Country Status (1)

Country Link
CN (1) CN107783980B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108427748A (en) * 2018-03-12 2018-08-21 北京奇艺世纪科技有限公司 Distributed data base secondary index querying method, device and server
CN109815231A (en) * 2018-12-26 2019-05-28 亚信科技（中国）有限公司 Data fragmentation method, device, computer equipment and storage medium
CN110324380A (en) * 2018-03-30 2019-10-11 北京忆芯科技有限公司 Utilize the cloud computing and mist computing system of KV storage equipment
CN110334109A (en) * 2019-05-08 2019-10-15 重庆猪八戒知识产权服务有限公司 Relational database data query method, system, medium and electronic equipment
CN110888872A (en) * 2019-10-16 2020-03-17 浙江大搜车软件技术有限公司 Data storage method and device, computer equipment and storage medium
CN111008200A (en) * 2019-12-18 2020-04-14 北京数衍科技有限公司 Data query method and device and server
CN111104426A (en) * 2019-11-22 2020-05-05 深圳智链物联科技有限公司 Data query method and system
CN111159521A (en) * 2019-12-12 2020-05-15 中国平安财产保险股份有限公司 Data query method, system, computer device and storage medium
CN112288348A (en) * 2020-09-10 2021-01-29 浙江省疾病预防控制中心 Biological safety transfer supervision method and system
CN112579608A (en) * 2020-12-25 2021-03-30 平安银行股份有限公司 Case data query method, system, device and computer readable storage medium
CN112632129A (en) * 2020-12-31 2021-04-09 联想未来通信科技(重庆)有限公司 Code stream data management method, device and storage medium
CN113505172A (en) * 2021-07-02 2021-10-15 泰康保险集团股份有限公司 Data processing method and device, electronic equipment and readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101533406A (en) * 2009-04-10 2009-09-16 北京锐安科技有限公司 Mass data querying method
US8171110B1 (en) * 2009-12-11 2012-05-01 Go Daddy Operating Company, LLC Tools enabling a preferred placement service for domain registration websites
CN102968498A (en) * 2012-12-05 2013-03-13 华为技术有限公司 Method and device for processing data
CN104133867A (en) * 2014-07-18 2014-11-05 中国科学院计算技术研究所 DOT in-fragment secondary index method and DOT in-fragment secondary index system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101533406A (en) * 2009-04-10 2009-09-16 北京锐安科技有限公司 Mass data querying method
US8171110B1 (en) * 2009-12-11 2012-05-01 Go Daddy Operating Company, LLC Tools enabling a preferred placement service for domain registration websites
CN102968498A (en) * 2012-12-05 2013-03-13 华为技术有限公司 Method and device for processing data
CN104133867A (en) * 2014-07-18 2014-11-05 中国科学院计算技术研究所 DOT in-fragment secondary index method and DOT in-fragment secondary index system

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108427748A (en) * 2018-03-12 2018-08-21 北京奇艺世纪科技有限公司 Distributed data base secondary index querying method, device and server
CN110324380A (en) * 2018-03-30 2019-10-11 北京忆芯科技有限公司 Utilize the cloud computing and mist computing system of KV storage equipment
CN109815231A (en) * 2018-12-26 2019-05-28 亚信科技（中国）有限公司 Data fragmentation method, device, computer equipment and storage medium
CN110334109B (en) * 2019-05-08 2020-07-24 重庆猪八戒知识产权服务有限公司 Relational database data query method, system, medium and electronic device
CN110334109A (en) * 2019-05-08 2019-10-15 重庆猪八戒知识产权服务有限公司 Relational database data query method, system, medium and electronic equipment
CN110888872A (en) * 2019-10-16 2020-03-17 浙江大搜车软件技术有限公司 Data storage method and device, computer equipment and storage medium
CN111104426A (en) * 2019-11-22 2020-05-05 深圳智链物联科技有限公司 Data query method and system
CN111104426B8 (en) * 2019-11-22 2024-04-23 北京傲速科技有限公司 Data query method and system
CN111104426B (en) * 2019-11-22 2024-04-05 深圳智链物联科技有限公司 Data query method and system
CN111159521A (en) * 2019-12-12 2020-05-15 中国平安财产保险股份有限公司 Data query method, system, computer device and storage medium
CN111159521B (en) * 2019-12-12 2024-04-30 中国平安财产保险股份有限公司 Data query method, system, computer device and storage medium
CN111008200B (en) * 2019-12-18 2024-01-16 北京数衍科技有限公司 Data query method, device and server
CN111008200A (en) * 2019-12-18 2020-04-14 北京数衍科技有限公司 Data query method and device and server
CN112288348A (en) * 2020-09-10 2021-01-29 浙江省疾病预防控制中心 Biological safety transfer supervision method and system
CN112288348B (en) * 2020-09-10 2021-10-15 浙江省疾病预防控制中心 Biological safety transfer supervision method and system
CN112579608A (en) * 2020-12-25 2021-03-30 平安银行股份有限公司 Case data query method, system, device and computer readable storage medium
CN112632129B (en) * 2020-12-31 2023-11-21 联想未来通信科技(重庆)有限公司 Code stream data management method, device and storage medium
CN112632129A (en) * 2020-12-31 2021-04-09 联想未来通信科技(重庆)有限公司 Code stream data management method, device and storage medium
CN113505172B (en) * 2021-07-02 2023-10-24 泰康保险集团股份有限公司 Data processing method, device, electronic equipment and readable storage medium
CN113505172A (en) * 2021-07-02 2021-10-15 泰康保险集团股份有限公司 Data processing method and device, electronic equipment and readable storage medium

Also Published As

Publication number Publication date
CN107783980B (en) 2021-10-19

Similar Documents

Publication Publication Date Title
CN107783980A (en) Index data generates and data query method and device, storage and inquiry system
US7756889B2 (en) Partitioning of nested tables
US4468728A (en) Data structure and search method for a data base management system
EP1352339B1 (en) Method of querying a structure of compressed data
US7480662B2 (en) Fact table storage in a decision support system environment
CN103902702A (en) Data storage system and data storage method
CN104636349B (en) A kind of index data compression and the method and apparatus of index data search
US7225186B2 (en) Binary search tree system and method
CN102915382A (en) Method and device for carrying out data query on database based on indexes
CN108897761A (en) A kind of clustering storage method and device
CN105100050A (en) User permission management method and system
CN107577787B (en) Method and system for storing associated data information
CN110471923A (en) A kind of processing method and processing device of block chain transaction record
CN105991478A (en) Server resource distribution method and system
CN110597852A (en) Data processing method, device, terminal and storage medium
CN104346458A (en) Data storage method and device
CN104794130B (en) Relation query method and device between a kind of table
CN108920105B (en) Community structure-based graph data distributed storage method and device
CN111221840A (en) Data processing method and device, data caching method, storage medium and system
CN111930823A (en) Data query method and device, data center station and storage medium
CN108399175B (en) Data storage and query method and device
CN110716990A (en) Multi-data-source management system applied to data transaction
CN111427931A (en) Distributed query engine and method for querying relational database by using same
CN106547805A (en) The method and apparatus of optimization database index
CN116450607A (en) Data processing method, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant