CN107870924A - Inquire about the method and data query system of data - Google Patents

Inquire about the method and data query system of data Download PDF

Info

Publication number
CN107870924A
CN107870924A CN201610850631.1A CN201610850631A CN107870924A CN 107870924 A CN107870924 A CN 107870924A CN 201610850631 A CN201610850631 A CN 201610850631A CN 107870924 A CN107870924 A CN 107870924A
Authority
CN
China
Prior art keywords
data storage
data
intermediate result
storage cluster
query statement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610850631.1A
Other languages
Chinese (zh)
Inventor
党李飞
崔鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201610850631.1A priority Critical patent/CN107870924A/en
Publication of CN107870924A publication Critical patent/CN107870924A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

The invention provides a kind of method and data query system for inquiring about data, data query system includes client Client modules and multiple acts on behalf of Agent modules, multiple corresponding data storage clusters of each Agent modules acted on behalf of in Agent modules, method include：Client modules obtain serializing query statement set, and serializing query statement set includes at least two query statements；First Agent modules inquire about data from the database of the first data storage cluster, obtain the first Query Result according to serializing query statement set；First Query Result is returned to Client modules by the first Agent modules.The method of the inquiry data of the present invention can in reduction system invalid data transmission, the bandwidth availability ratio between raising data storage cluster, and the interaction times between Client modules and data storage cluster can be reduced, it is possible to increase inquiry response speed.

Description

Inquire about the method and data query system of data
Technical field
The application is related to calculating field, and more particularly, to a kind of method and data query system for inquiring about data.
Background technology
Large enterprise or large organization would generally be disposed in different geographical position multiple data centers (DataCenter, DC).For example, Hbase is exactly Distributed Storage cluster general in a kind of big data scene, different clusters can be deployed in In the multiple data centers being geographically spread out.Hbase different clusters would generally store different data, and data use key assignments (key-value) storage mode.
User is when using Hbase, it will usually is related to inquiring about the data in the database of multiple data centers.Use at present When family communicates with Hbase, HbaseClient is relied primarily on.One HbaseClient can only once connect a Hbase cluster. User accesses the data of multiple clusters if desired, can only be utilized by disposing corresponding application program in HbaseClient HbaseClient respectively with specific one or multiple cluster communications.HbaseClient initiates data to different clusters respectively Request, complete collecting and handling for data.
Specifically, such as Hbase includes two clusters, there is two tables in each cluster, respectively Table_1 and Table_2.Every table has identical table structure in different clusters, but every table has in different clusters Different data contents.When inquiring about data, HbaseClient is arrived first in each cluster according to query statement and is inquired about Table_ respectively 1, intermediate result content_1 therein is obtained, the content_1 that all clusters inquire is gathered.Then, HbaseClient inquires about Table_2 respectively further according to the content_1 after collecting into each cluster, obtains final inquiry As a result.
In this process, Hbase can be caused a large amount of when data center carries out the correlation inquiry between multilist for user The transmission of invalid data, the load of equipment in cluster is increased, and HbaseClient needs and Hbase different clusters enter Repeatedly interaction can just obtain final Query Result to row, cause inquiry response slow.
The content of the invention
The application provides a kind of method and data query system for inquiring about data, and hole is easily improved between data storage cluster Bandwidth availability ratio, it is possible to increase inquiry response speed.
First aspect, there is provided a kind of method for inquiring about data, methods described are performed by data query system, the data Inquiry system include client Client modules and it is multiple act on behalf of Agent modules, it is each in the multiple Agent modules The corresponding data storage cluster of Agent modules, wherein, the corresponding first data storage cluster of the first Agent modules, described first Agent modules are used to inquire about data from the database of the first data storage cluster, and methods described includes：The Client Module obtains serializing query statement set, and the serializing query statement set includes at least two query statements；Described One Agent modules inquire about number according to the serializing query statement set from the database of the first data storage cluster According to obtaining the first Query Result；First Query Result is returned to the Client modules by the first Agent modules.
The method of the inquiry data of first aspect, by increasing Agent modules, Client modules obtain serializing inquiry and referred to Order set, and Agent modules are handed down to, as performing data query in data storage cluster of the Agent modules corresponding to it, and Final Query Result is returned into Client modules, it is possible to reduce the transmission of invalid data in system, improve data storage cluster Between bandwidth availability ratio, and the interaction times between Client modules and data storage cluster can be reduced, it is possible to increase inquiry Response speed.
With reference in a first aspect, in the first possible implementation of first aspect, the serializing query statement collection Conjunction includes N number of query statement, and N number of query statement includes query statement 1 ..., query statement according to the logical order of inquiry I ..., query statement N, wherein, query statement i corresponding first intermediate result i, N are more than or equal to 2, the first Agent modules According to the serializing query statement set, data are inquired about from the database of the first data storage cluster, obtain first Query Result, including：The first Agent modules are according to the query statement 1, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first intermediate result 1；The first Agent modules are using the first intermediate result i-1 as the inquiry I querying condition is instructed, data is inquired about from the database of the first data storage cluster, obtains the first intermediate result i, its In, i value is 2 ..., N, when i is equal to N, using the first intermediate result N as first Query Result.The first may Implementation go for using the first intermediate result i-1 as during query statement i querying condition, the first intermediate result i It also is stored in the situation of same data storage cluster.
With reference in a first aspect, in second of possible implementation of first aspect, the serializing query statement collection Conjunction includes N number of query statement, and N number of query statement includes query statement 1 ..., query statement according to the logical order of inquiry I ..., query statement N, wherein, query statement i corresponding first intermediate result i, N are more than or equal to 2, the first Agent modules According to the serializing query statement set, data are inquired about from the database of the first data storage cluster, obtain first Query Result, including：The first Agent modules are according to the query statement 1, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first intermediate result 1；The first Agent modules are according to metadata information, it is determined that by among first As a result the data storage cluster where the intermediate result i obtained during querying conditions of the i-1 as the query statement i；When by Data storage cluster where the intermediate result i obtained during querying conditions of the one intermediate result i-1 as the query statement i is During the first data storage cluster, the first Agent modules are using the first intermediate result i-1 as the query statement i's Querying condition, data are inquired about from the database of the first data storage cluster, obtain the first intermediate result i, wherein, i's Value is 2 ..., N, when i is equal to N, using the first intermediate result N as first Query Result；When by the first intermediate result Data storage cluster where the intermediate result i obtained during querying conditions of the i-1 as query statement i is not first data During storage cluster, the first intermediate result i-1 is sent to corresponding to the second data storage cluster by the first Agent modules 2nd Agent modules.Second of possible implementation goes for using the first intermediate result i-1 as query statement i's During querying condition, the first intermediate result i is potentially stored in same data storage cluster, it is also possible to is stored in different data and deposits The situation of accumulation.
With reference to second of possible implementation of first aspect, in the third possible implementation of first aspect In, the first metadata information of data, described in database of the metadata information including the first data storage cluster First Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as being obtained during query statement i querying condition Data storage cluster where the intermediate result i arrived, including：The first Agent modules according to first metadata information, It is determined that using the first intermediate result i-1 as the data store set where the intermediate result i obtained during query statement i querying condition Group is not the first data storage cluster；The first intermediate result i-1 is sent to the second number by the first Agent modules According to the 2nd Agent modules corresponding to storage cluster, including：The first Agent modules send the first intermediate result i-1 Give the 2nd Agent modules corresponding to adjacent the second data storage cluster of the first data store set faciation.
With reference to the third possible implementation of first aspect, in the 4th kind of possible implementation of first aspect In, methods described also includes：The first Agent modules send configured information, the instruction letter to the 2nd Agent modules Cease for indicating to have inquired about in the first data storage cluster using the first intermediate result i-1 as query statement i's The intermediate result i obtained during querying condition, and using the first intermediate result i-1 as being obtained during query statement i querying condition Intermediate result i where data storage cluster be not the first data storage cluster.In the 4th kind of possibility of first aspect Implementation can prevent what the first intermediate result i-1 from repeating in Agent modules to be transmitted, or even there is endless loop Situation.
With reference to second of possible implementation of first aspect, in the 5th kind of possible implementation of first aspect In, the metadata information includes the first metadata information and the second metadata information, and first metadata information is described The metadata information of data in the database of first data storage cluster, second metadata information are except first data The metadata information of data, the first Agent module roots in the database of other data storage clusters beyond storage cluster According to metadata information, it is determined that using the first intermediate result i-1 as the intermediate result i institutes obtained during query statement i querying condition Data storage cluster, including：The first Agent modules are according to first metadata information and second metadata Information, it is determined that using the first intermediate result i-1 as the data where the intermediate result i obtained during query statement i querying condition Storage cluster is the second data storage cluster.
With reference to the various possible implementations of first aspect, in the 6th kind of possible implementation of first aspect, institute The first Agent modules are stated according to the serializing query statement set, are looked into from the database of the first data storage cluster Data are ask, obtain the first Query Result, in addition to：The first Agent modules are received in the 3rd Agent modules are sent the 3rd Between result i-1, querying condition of the first Agent modules using the 3rd intermediate result i-1 as the query statement i, Data are inquired about from the database of the first data storage cluster, obtain the first intermediate result i.
Second aspect, there is provided a kind of data query system, the data query system service including client Client Device and it is multiple act on behalf of Agent servers, wherein, Client servers include first processor and first memory, described first Memory is used for store instruction, and the first processor is used for the instruction for performing the first memory storage, and described first deposits Reservoir and the first processor are used to realizing Client modules, each Agent servers in the multiple Agent servers Including second processor and second memory, the second memory is used for store instruction, and the second processor is used to perform The instruction of the second memory storage, the second memory and the second processor are used to realize Agent modules, each The corresponding data storage cluster of Agent modules, corresponding first data storage of the first Agent modules of the first Agent servers Cluster, the Client modules are used to obtain serializing query statement set, and the serializing query statement set is included at least Two query statements；The first Agent modules are used for the serializing query statement obtained according to the Client modules Set, inquires about data from the database of the first data storage cluster, obtains the first Query Result；First Agent Module is additionally operable to first Query Result returning to the Client modules.
The third aspect, there is provided a kind of method for inquiring about data, methods described are performed by first agent Agent modules, the One Agent modules are a modules in data query system, the data query system include client Client modules and It is multiple to act on behalf of Agent modules, the corresponding data storage cluster of each Agent modules in the multiple Agent modules, its In, the corresponding first data storage cluster of the first Agent modules, the first Agent modules are used for from first data storage Data are inquired about in the database of cluster, methods described includes：The first Agent modules receive what the Client modules were sent Query statement set is serialized, the serializing query statement set includes at least two query statements；The first Agent moulds Root tuber inquires about data from the database of the first data storage cluster, obtains the according to the serializing query statement set One Query Result；First Query Result is returned to the Client modules by the first Agent modules.
With reference to the third aspect, in the first possible implementation of the third aspect, the serializing query statement collection Conjunction includes N number of query statement, and N number of query statement includes query statement 1 ..., query statement according to the logical order of inquiry I ..., query statement N, wherein, query statement i corresponding first intermediate result i, N are more than or equal to 2, the first Agent modules According to the serializing query statement set, data are inquired about from the database of the first data storage cluster, obtain first Query Result, including：The first Agent modules are according to the query statement 1, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first intermediate result 1；The first Agent modules are using the first intermediate result i-1 as the inquiry I querying condition is instructed, data is inquired about from the database of the first data storage cluster, obtains the first intermediate result i, its In, i value is 2 ..., N, when i is equal to N, using the first intermediate result N as first Query Result.
With reference to the third aspect, in second of possible implementation of the third aspect, the serializing query statement collection Conjunction includes N number of query statement, and N number of query statement includes query statement 1 ..., query statement according to the logical order of inquiry I ..., query statement N, wherein, query statement i corresponding first intermediate result i, N are more than or equal to 2, the first Agent modules According to the serializing query statement set, data are inquired about from the database of the first data storage cluster, obtain first Query Result, including：The first Agent modules are according to the query statement 1, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first intermediate result 1；The first Agent modules are according to metadata information, it is determined that by among first As a result the data storage cluster where the intermediate result i obtained during querying conditions of the i-1 as the query statement i；When by Data storage cluster where the intermediate result i obtained during querying conditions of the one intermediate result i-1 as the query statement i is During the first data storage cluster, the first Agent modules are using the first intermediate result i-1 as the query statement i's Querying condition, data are inquired about from the database of the first data storage cluster, obtain the first intermediate result i, wherein, i's Value is 2 ..., N, when i is equal to N, using the first intermediate result N as first Query Result；When by the first intermediate result Data storage cluster where the intermediate result i obtained during querying conditions of the i-1 as query statement i is not first data During storage cluster, the first intermediate result i-1 is sent to corresponding to the second data storage cluster by the first Agent modules 2nd Agent modules.
With reference to second of possible implementation of the third aspect, in the third possible implementation of the third aspect In, the first metadata information of data, described in database of the metadata information including the first data storage cluster First Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as being obtained during query statement i querying condition Data storage cluster where the intermediate result i arrived, including：The first Agent modules according to first metadata information, It is determined that using the first intermediate result i-1 as the data store set where the intermediate result i obtained during query statement i querying condition Group is not the first data storage cluster；The first intermediate result i-1 is sent to the second number by the first Agent modules According to the 2nd Agent modules corresponding to storage cluster, including：The first Agent modules send the first intermediate result i-1 Give the 2nd Agent modules corresponding to adjacent the second data storage cluster of the first data store set faciation.
With reference to the third possible implementation of the third aspect, in the 4th kind of possible implementation of the third aspect In, methods described also includes：The first Agent modules send configured information, the instruction letter to the 2nd Agent modules Cease for indicating to have inquired about in the first data storage cluster using the first intermediate result i-1 as query statement i's The intermediate result i obtained during querying condition, and using the first intermediate result i-1 as being obtained during query statement i querying condition Intermediate result i where data storage cluster be not the first data storage cluster.
With reference to second of possible implementation of the third aspect, in the 5th kind of possible implementation of the third aspect In, the metadata information includes the first metadata information and the second metadata information, and first metadata information is described The metadata information of data in the database of first data storage cluster, second metadata information are except first data The metadata information of data, the first Agent module roots in the database of other data storage clusters beyond storage cluster According to metadata information, it is determined that using the first intermediate result i-1 as the intermediate result i institutes obtained during query statement i querying condition Data storage cluster, including：The first Agent modules are according to first metadata information and second metadata Information, it is determined that using the first intermediate result i-1 as the data where the intermediate result i obtained during query statement i querying condition Storage cluster is the second data storage cluster.
With reference to the various possible implementations of the third aspect, in the 6th kind of possible implementation of the third aspect, institute The first Agent modules are stated according to the serializing query statement set, are looked into from the database of the first data storage cluster Data are ask, obtain the first Query Result, in addition to：The first Agent modules are received in the 3rd Agent modules are sent the 3rd Between result i-1, querying condition of the first Agent modules using the 3rd intermediate result i-1 as the query statement i, Data are inquired about from the database of the first data storage cluster, obtain the first intermediate result i.
The method and the third aspect of the data query system of second aspect and the inquiry data of the third aspect are each possible The beneficial effect that implementation can obtain is corresponding with the method for the inquiry data of first aspect, repeats no more.
Fourth aspect, there is provided a kind of storage medium, the program code stored in the storage medium can be real when being performed The method for the inquiry data that the various implementations of the existing third aspect or the third aspect provide.
5th aspect, there is provided a kind of computer program product, the computer program product can be a software installation Bag, when the software installation coating computer is run, the various implementations offer of the execution third aspect or the third aspect is looked into The method for asking data.
Brief description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be in embodiment or description of the prior art The required accompanying drawing used is briefly described, it should be apparent that, drawings in the following description are only some realities of the present invention Example is applied, for those of ordinary skill in the art, without having to pay creative labor, can also be attached according to these Figure obtains other accompanying drawings.
Fig. 1 is a kind of indicative flowchart for the method for inquiring about data.
Fig. 2 is the schematic block diagram of the data query system of one embodiment of the invention.
Fig. 3 is the indicative flowchart of the method for the inquiry data of one embodiment of the invention.
Fig. 4 be one embodiment of the invention data storage cluster in table content schematic diagram.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation describes, it is clear that described embodiment is part of the embodiment of the present invention, rather than whole embodiments.Based on this hair Embodiment in bright, the every other implementation that those of ordinary skill in the art are obtained under the premise of creative work is not made Example, belongs to the scope of protection of the invention.
Various embodiments of the present invention are applied to the data query system with Distributed Storage cluster, are particularly suitable for use in Hbase.The data query system of the embodiment of the present invention can include multiple data-storage clusters, wherein, each data-storage collection Group is specifically realized by an at least data storage device.
In various embodiments of the present invention, the X in X intermediate result Y is the sign to certain Agent module, and Y is that certain inquiry is referred to The sign of order.X intermediate result Y represents that X Agent modules are inquired about according to query statement Y, resulting intermediate result.
In existing data query technique, user is when using Hbase, it will usually is related to inquiring about multiple data centers Database in data.When user communicates with Hbase at present, HbaseClient is relied primarily on.One HbaseClient mono- It is secondary to connect a Hbase cluster.User accesses the data of multiple clusters if desired, can only be by HbaseClient The corresponding application program of middle deployment, using HbaseClient respectively with specific one or multiple cluster communications. HbaseClient initiates request of data to different cluster respectively, completes collecting and handling for data.
Specifically, such as Hbase includes two clusters, the first data storage cluster and the second data storage cluster.Such as The flow chart of inquiry data shown in Fig. 1, there are two tables in each cluster, table name is respectively Table_1 and Table_2.Every Table has identical table structure in different clusters, but every table has in different data in different clusters Hold.For sake of simplicity, each table only depicts a row in Fig. 1.It is understood that there is index relative between index and key, i.e., In row in Table_2 in addition to key is arranged, an also row major key, the content of major key is the content in Table_1.Wherein it is possible to There is following corresponding relation：Index1 corresponds to key1, and index2 corresponds to key2, and it is corresponding that index3 corresponds to key3 and index4 key4.Herein in follow-up description, all tables include above-mentioned similar implication, and here is omitted.
For a series of query statements of user, query statement 1：The content inquired about according to querying condition in Table_1, tool Can be indexList=query Table_1 for body；Query statement 2：After obtaining the content of query statement 1, by these Hold and go to inquire about Table_2 again as querying condition, obtain final Query Result, can be specifically keyList=query Table_2by indexList。
(A) to (D) of the idiographic flow of existing inquiry data as shown in Figure 1.
When inquiring about data, HbaseClient first deposits number to the first data storage cluster and data respectively according to query statement 1 Table_1 is inquired about respectively in cluster 2, such as (A) in Fig. 1.
Such as Fig. 1 (B), deposited from the first data storage cluster and data intermediate result content_ is obtained in manifold group 2 respectively 1.Wherein, intermediate result content_1 corresponding to the first data storage cluster includes index1 and index2；Second data storage Intermediate result content_1 corresponding to cluster includes index3 and index4.HbaseClient inquires all clusters Content_1 gathers, and obtains index1-index4.
Then, such as Fig. 1 (C), HbaseClient is further according to the content_1 (i.e. index1-index4) after collecting to One data storage cluster and data are deposited in manifold group 2 inquires about Table_2 respectively.
Such as Fig. 1 (D), obtain and return to final Query Result key1-key4.
In the example of fig. 1, in intermediate result index1-index4, in practice for inquiry Table_2, the first data Index1 and index2 are only needed in storage cluster, DC2 only needs index3 and index4.But HbaseClient collects centre As a result index1-index4 can be all separately sent to two data storage clusters after, during actual transmissions, causes invalid data Transmission.In addition, HbaseClient needs and Hbase different clusters carry out repeatedly interaction and can just obtain final inquiry knot Fruit, cause inquiry response slow.
Based on the above situation, the embodiments of the invention provide a kind of method for inquiring about data, methods described is by data query System performs, i.e., data query system is used to inquire about data from data storage cluster, and Fig. 2 schematically illustrates data query System and 3 data storage clusters, the first data storage cluster, the second data storage cluster and the 3rd data storage cluster. The data query system includes client (Client) module and multiple agency's (Agent) modules.I.e. relative to existing number According to inquiry system, the data query system of the embodiment of the present invention has additionally introduced Agent modules.Agent modules are entered as user Agency of the row across data center (i.e. across data storage cluster) distributed query.It is the multiple act on behalf of it is each in Agent modules The corresponding data storage cluster of Agent modules, wherein, the corresponding first data storage cluster of the first Agent modules, described first Agent modules are used to inquire about data from the database of the first data storage cluster.
The method of the inquiry data of the embodiment of the present invention can include：The Client modules obtain serializing query statement Set, the serializing query statement set include at least two query statements；The first Agent modules are according to the sequence Change query statement set, inquire about data from the database of the first data storage cluster, obtain the first Query Result；It is described First Query Result is returned to the Client modules by the first Agent modules.
It should be understood that the first Agent modules can be any one Agent module in multiple Agent modules.
It should also be understood that any number of Agent modules and data storage cluster can be included in data query system, it is unlimited The number shown in Fig. 2.
The method of the inquiry data of the embodiment of the present invention, by increasing Agent modules, Client modules obtain serializing and looked into Instruction set is ask, and is handed down to Agent modules, is looked into as performing data in data storage cluster of the Agent modules corresponding to it Ask, and final Query Result is returned into Client modules, it is possible to reduce the transmission of invalid data in system, improve data storage Bandwidth availability ratio between cluster, and the interaction times between Client modules and data storage cluster can be reduced, it is possible to increase Inquiry response speed.
Specifically, Agent modules carry out interior optimization to a plurality of query statement in serializing query statement set, complete Into specific query logic.In distributed multi-table query, for a data storage cluster, looked into equivalent to by a plurality of Instruction folding is ask, mass carries out data query, so as to reduce the transmission of invalid data.
In one embodiment of the present of invention, the serializing query statement set can include N number of query statement, the N Individual query statement can include query statement 1 ..., query statement i ..., query statement N according to the logical order of inquiry.Wherein, Query statement i corresponding first intermediate result i, N are more than or equal to 2.For any one Agent mould in multiple Agent modules For block, such as the first Agent modules, the first Agent modules are according to the serializing query statement set, from described the Data are inquired about in the database of one data storage cluster, obtain the first Query Result, can be included：The first Agent modules According to the query statement 1, data are inquired about from the database of the first data storage cluster, obtain the first intermediate result 1； Querying condition of the first Agent modules using the first intermediate result i-1 as the query statement i, from first data Data are inquired about in the database of storage cluster, obtain the first intermediate result i, wherein, i value is 2 ..., N, when i is equal to N, Using the first intermediate result N as first Query Result.
Illustrate the method for the inquiry data of the embodiment of the present invention with a specific example with reference to Fig. 3.
1.Client modules obtain serializing query statement set caused by user inputs or program operation.Still with Fig. 1 Exemplified by corresponding query statement, serializing query statement set includes 2 query statements.Query statement 1：IndexList= query Table_1；Query statement 2：KeyList=query Table_2byindexList.
Serializing query statement set is sent to each Agent modules in all Agent modules by 2.Client modules. Assuming that user currently has two data storage clusters, the first data storage cluster and the second data storage cluster.First data are deposited Accumulation and the second data storage cluster correspond to the first Agent modules and the 2nd Agent modules respectively.Each data storage cluster In table content it is as shown in Figure 3.
(A) of step 1 and step 2 idiographic flow as shown in Figure 3.
3. each Agent modules are obtained after serializing query statement set, processing is as follows.
A) such as Fig. 3 (B), query statement 1, indexList=query is first carried out in the data storage cluster of local Table_1, and obtain intermediate result 1.For the first Agent modules corresponding to the first data storage cluster, can now obtain First intermediate result 1, i.e. indexList=(index1, index2), for second corresponding to the second data storage cluster Agent modules, it can now obtain the second intermediate result 1, i.e. indexList=(index3, index4).
B) as Fig. 3 (C), two Agent modules perform query statement 2, keyList=queryTable_2by respectively indexList.In the present example, associated data involved in Table_1 and Table_2 are stored entirely in same number According in storage cluster.That is the index1 in the first intermediate result 1, the value of the key corresponding to index2 are all deposited in the first data In the Table_2 of accumulation；Index3 in second intermediate result 1, the key corresponding to index4 value are all in the second data In the Table_2 of storage cluster.In other words, all data are all deposited in the form of major key in local Table_1 indexList In the Table_2 in local.Therefore, the inquiry of query statement 2 is in each self-corresponding data store set group of Agent modules Carry out.For the first Agent modules, it utilizes the first intermediate result 1 (index1, index2) in the first data storage cluster As querying condition, inquiry Table_2 obtains the first intermediate result 2 (key1, key2)；For the 2nd Agent modules, it is Querying condition is used as using the second intermediate result 1 (index3, index4) in two data storage clusters, inquiry Table_2 is obtained Second intermediate result 2 (key3, key4).Here, the first intermediate result 2 (key1, key2) is the first Query Result, in second Between result 2 (key3, key4) be the second Query Result.
In addition, in query process, Agent modules can be believed by the metadata inquired about in local data storage cluster Breath, draws and whether there is the first intermediate result i-1 as the intermediate result i obtained during the querying condition of the query statement i In the data storage cluster of local.When in the absence of when, can report an error or return Query Result for sky.
4. such as Fig. 3 (D), the first Agent modules and the 2nd Agent modules separately return finally to Client modules The first Query Result and the second Query Result, the first Query Result and the second Query Result sum are total Query Result, This poll-final.
In this example embodiment, for whole query process, Client modules only need to interact one respectively with each Agent modules It is secondary to complete entirely to inquire about.Due to the design of data structure, each Agent modules are inquired about in corresponding data storage cluster The intermediate result arrived, it is follow-up only to be handled in the data storage cluster of local, it is not necessary to be transferred to other data store sets Group.Certainly, in the embodiment of the present invention, each Agent modules can receive the intermediate result that other Agent modules are sent, Using these intermediate results as querying condition, follow-up inquiry is carried out, the embodiment of the present invention is not construed as limiting to this.
The embodiment of the present invention goes for Data Structure Design：Involved by a series of table of Different Logic orders Related data is stored entirely in the situation of same data storage cluster.In other words, the embodiment of the present invention go for by During querying conditions of the first intermediate result i-1 as query statement i, the first intermediate result i also is stored in same data storage The situation of cluster.In this case, when the first Agent modules can not inquire corresponding result in the first data storage cluster When, it may be possible to really there is no in data storage cluster in the data for meeting query statement, or data storage cluster due to There is the inconsistent situation of data in exception.
In an alternative embodiment of the invention, the serializing query statement set includes N number of query statement, described N number of Query statement includes query statement 1 ... according to the logical order of inquiry, query statement i ..., query statement N, wherein, inquiry refers to The corresponding first intermediate result i of i, N is made to be more than or equal to 2, the first Agent modules are according to the serializing query statement collection Close, inquire about data from the database of the first data storage cluster, obtain the first Query Result, including：Described first Agent modules inquire about data from the database of the first data storage cluster, obtain first according to the query statement 1 Intermediate result 1；The first Agent modules are according to metadata information, it is determined that referring to the first intermediate result i-1 as the inquiry Data storage cluster where the intermediate result i obtained during the querying condition for making i；When using the first intermediate result i-1 as described in Data storage cluster where the intermediate result i obtained during query statement i querying condition is the first data storage cluster When, querying condition of the first Agent modules using the first intermediate result i-1 as the query statement i, from the described first number According to data are inquired about in the database of storage cluster, the first intermediate result i is obtained, wherein, i value is 2 ..., N, when i is equal to N When, using the first intermediate result N as first Query Result；When the inquiry using the first intermediate result i-1 as query statement i When data storage cluster where the intermediate result i obtained during condition is not the first data storage cluster, described first The first intermediate result i-1 is sent to the 2nd Agent modules corresponding to the second data storage cluster by Agent modules.
It is suitable for Data Structure Design relative to above an example, this example：A series of table of Different Logic orders In involved related data be potentially stored in same data storage cluster, it is also possible to be stored in different data storage clusters Situation.In other words, the embodiment of the present invention goes for the inquiry bar using the first intermediate result i-1 as query statement i During part, the first intermediate result i is potentially stored in same data storage cluster, it is also possible to is stored in different data storage clusters Situation.
Such case is directed to, the interacting message between Agent modules as shown in Figure 2 can be increased.That is, Agent moulds When block performs query statement, according to metadata information, it is determined that using the first intermediate result i-1 as the query statement i's Data storage cluster where the intermediate result i obtained during querying condition.Refer to when using the first intermediate result i-1 as the inquiry The intermediate result i obtained during the querying condition for making i is still carried out follow-up in the first data storage cluster according to the example of top Inquiry.When using the first intermediate result i-1 as the intermediate result i obtained during the querying condition of the query statement i not During one data storage cluster, and in two kinds of situation.
The first situation be the metadata information including the first data storage cluster database in data the One metadata information, the first Agent modules are according to metadata information, it is determined that referring to using the first intermediate result i-1 as inquiry Data storage cluster where the intermediate result i obtained during the querying condition for making i, including：The first Agent modules are according to institute The first metadata information is stated, it is determined that using the first intermediate result i-1 as the intermediate result obtained during query statement i querying condition Data storage cluster where i is not the first data storage cluster；The first Agent modules tie the described first centre Fruit i-1 is sent to the 2nd Agent modules corresponding to the second data storage cluster, including：The first Agent modules are by described The second data storage cluster that one intermediate result i-1 is sent to the first data store set faciation is adjacent is corresponding described 2nd Agent modules.
Wherein, the first metadata information can be stored in the first data storage cluster, can also be stored in the first Agent In the caching of module.Here, can after the data of each table in the excessively corresponding data storage cluster of each Agent resume modules With metadata information be stored in the caching of oneself.
It should be understood that the second data storage cluster can be the data storage closed on the first data storage cluster geographical position Cluster or the nearer data storage cluster of communication distance, the embodiment of the present invention are not construed as limiting to this.
Certainly, in addition to above-mentioned scene, though be stored with the first data storage cluster except the first data storage cluster with The metadata information of data in the database of outer other data storage clusters, but not can determine that in first according to these information Between querying conditions of the result i-1 as query statement i when obtained intermediate result i where data storage cluster when, described One Agent modules the first intermediate result i-1 can also be sent to the first data store set faciation neighbour described in The 2nd Agent modules corresponding to second data storage cluster.
In the first scenario, in order to prevent the first intermediate result i-1 in Agent modules repeat be transmitted, very To there is the situation of endless loop.Methods described can also include：The first Agent modules are sent out to the 2nd Agent modules Configured information is sent, the configured information is used to indicate to have inquired about in the first data storage cluster to tie the first centre The intermediate result i obtained during querying conditions of the fruit i-1 as query statement i, and refer to using the first intermediate result i-1 as inquiry Data storage cluster where the intermediate result i obtained during the querying condition for making i is not the first data storage cluster.That is, Other Agent modules are informed, this Agent modules have treated first intermediate result i-1, and specific illustrate will be It is given below.
Second of situation is that the metadata information includes the first metadata information and the second metadata information, described first Metadata information for the first data storage cluster database in data metadata information, second metadata information It is described for the metadata information of data in the database of other data storage clusters in addition to the first data storage cluster First Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as being obtained during query statement i querying condition Data storage cluster where the intermediate result i arrived, including：The first Agent modules are according to first metadata information With second metadata information, it is determined that using the first intermediate result i-1 as the centre obtained during query statement i querying condition As a result the data storage cluster where i is the second data storage cluster.
In the latter case, the first Agent modules, can be true according to the first metadata information and the second metadata information Determine using the first intermediate result i-1 as the data storage cluster where the intermediate result i obtained during query statement i querying condition When, the first intermediate result i-1 is sent to Agent modules corresponding to the data storage cluster determined.Wherein, as described above, First metadata information can be stored in the first data storage cluster, can also be stored in the caching of the first Agent modules. Second metadata information can be then stored in the caching of the first Agent modules in the following manner.Each Agent resume modules After crossing the data of each table in corresponding data storage cluster, can with metadata information be stored in the caching of oneself. The data in the database of each self-corresponding data store set group can regularly be shared between Agent modules mutually by heartbeat Metadata information.Each Agent modules receive the data in the database of data storage cluster corresponding to other Agent modules Metadata information after, these metadata informations can be cached.
In various embodiments of the present invention, metadata information is can to represent the data area that data storage cluster is stored Information.
Illustrate the method for the inquiry data of the embodiment of the present invention with a still specific example below.
1.Client modules obtain serializing query statement set caused by user inputs or program operation.Still with Fig. 1 Exemplified by corresponding query statement, serializing query statement set includes 2 query statements.Query statement 1：IndexList= query Table_1；Query statement 2：KeyList=query Table_2byindexList.
Serializing query statement set is sent to each Agent modules in all Agent modules by 2.Client modules. Assuming that user currently has three data storage clusters, the first data storage cluster, the second data storage cluster and the 3rd data are deposited Accumulation.First data storage cluster, the second data storage cluster and the 3rd data storage cluster correspond to the first Agent moulds respectively Block, the 2nd Agent modules and the 3rd Agent modules.The content of table in each data storage cluster is as shown in Figure 4.
3. each Agent modules are obtained after serializing query statement set, processing is as follows.
A) query statement 1, indexList=queryTable_1 is first carried out in the data storage cluster of local, and Obtain intermediate result 1.For the first Agent modules corresponding to the first data storage cluster, can now obtain tying among first Fruit 1, i.e. indexList=(index1, index2, index10, index12), for corresponding to the second data storage cluster Two Agent modules, it can now obtain the second intermediate result 1, i.e. indexList=(index5, index6), for the 3rd number According to the 3rd Agent modules corresponding to storage cluster, it can now obtain the 3rd intermediate result 1, i.e. indexList=(index3, index4,index16)。
B) three Agent modules perform query statement 2, keyList=query Table_2byindexList respectively.Look into When inquiry, for second of the situation provided above, intermediate result 1 can be allocated as three classes and handle respectively.
I. if by query metadata information, intermediate result 2 is drawn in data storage cluster corresponding to itself, then directly The querying condition using intermediate result 1 as query statement 2 is connect, is inquired about in data storage cluster corresponding to itself, and in caching Between result 2 in case returning to Client modules.
Ii. if by query metadata information, intermediate result 2 is obtained not in data storage cluster corresponding to itself In, and when can obtain the long-range data storage cluster belonging to it, then intermediate result 1 is sent to long-range data storage cluster Corresponding Agent modules.By Agent modules corresponding to long-range data storage cluster, query statement is used as using intermediate result 1 2 querying condition, inquired about in long-range data storage cluster, and cache intermediate result 2 in case returning to Client modules.
When due to exception or other reasonses, when long-range data storage cluster does not inquire intermediate result 2, long-range number It can consider Query Result for sky according to Agent modules corresponding to storage cluster.Or corresponding to the long-range data storage cluster Agent modules can continue intermediate result 1 to be sent to other Agent modules, and specific processing may refer to the i-th ii classes. Or intermediate result 1 can be sent to specific Agent modules by Agent modules corresponding to the long-range data storage cluster, When the specific Agent modules are still inquired about less than intermediate result 2, then it is assumed that Query Result is sky, to the place of the intermediate result 1 Reason terminates.
Iii. if passing through query metadata information, it is impossible to when obtaining the data storage cluster belonging to intermediate result 2, Intermediate result 1 can be sent to Agent modules corresponding to adjacent data storage cluster by Agent modules.By adjacent data Agent modules continue with according to above-mentioned logic corresponding to storage cluster.If it is still inquiry in adjacent data storage cluster To intermediate result 2, then Agent modules corresponding to adjacent data storage cluster continue intermediate result 1 being sent to other Agent Module.
In order to prevent what intermediate result 1 from repeating in Agent modules to be transmitted, or even there is the situation of endless loop, Which data store set is Agent modules can carry intermediate result 1 by when intermediate result 1 is sent into other Agent modules The information that the Agent resume modules of group are crossed, and each Agent modules are it is known that being affixed one's name in the middle part of all data storage clusters Agent modules.So when Agent resume modules intermediate result 1, other all Agent modules are found all After treated, it is known that it is that this is that last time is handled, if still looked into data storage cluster corresponding to this Agent module Ask less than intermediate result 2, then abandon the intermediate result 1.Intermediate result 1 can be one in multiple index.Here, Information that entrained intermediate result 1 is crossed by the Agent resume modules of which data storage cluster, it is as middle above to describe Configured information.
In this example, querying condition of each Agent modules using intermediate result 1 as query statement 2, continue subsequent treatment Logic it is as follows.
First Agent modules：
(index1, index2) in first intermediate result 1：Directly in the first data storage cluster among inquiry first As a result 2, and cached the first intermediate result 2 inquired as the first final Query Result, in case returning to Client moulds Block.
(index10) in first intermediate result 1：The 3rd Agent modules corresponding to the 3rd data storage cluster are sent to, Subsequent query is completed by the 3rd Agent modules.
(index12) in first intermediate result 1：The 2nd Agent modules corresponding to the second data storage cluster are sent to, Subsequent query is completed by the 2nd Agent modules.
2nd Agent modules：
(index5, index6) in second intermediate result 1：Directly in the second data storage cluster among inquiry second As a result 2, and cached the second intermediate result 2 inquired as the second final Query Result, in case returning to Client moulds Block.
3rd Agent modules：
(index3, index4) in 3rd intermediate result 1：Directly in the 3rd data storage cluster among inquiry the 3rd As a result 2, and cached the 3rd intermediate result 2 inquired as the 3rd final Query Result, in case returning to Client moulds Block.
(index16) in 3rd intermediate result 1：The first Agent modules corresponding to the first data storage cluster are sent to, Subsequent query is completed by the first Agent modules.Specifically, the first Agent modules are received in the 3rd Agent modules are sent the 3rd Between (index16) in result 1, the querying condition using (index16) in the 3rd intermediate result 1 as the query statement 2, Data are inquired about from the database of the first data storage cluster, obtained result is as the first intermediate result 2.
Correspondingly, the first Agent modules are according to the serializing query statement set, from first data storage Data are inquired about in the database of cluster, obtain the first Query Result, can also be included：The first Agent modules receive the 3rd The 3rd intermediate result i-1 that Agent modules are sent, the first Agent modules are using the 3rd intermediate result i-1 as described in Query statement i querying condition, data are inquired about from the database of the first data storage cluster, are obtained among described first As a result i.
In the embodiment of the present invention, the first intermediate result 2 (key1, key2, key16) is the first Query Result, in second Between result 2 (key5, key6, key12) be the second Query Result, the 3rd intermediate result 2 (key3, key4, key10) is Three Query Results.
4. the first Agent modules, the 2nd Agent modules and the 3rd Agent modules are separately returned to Client modules Return final the first Query Result, the second Query Result and the 3rd Query Result, the first Query Result, the second Query Result and the Three Query Result sums are total Query Result, this poll-final.
The method of the inquiry data of the embodiment of the present invention, can be according to the position attribution of data dynamically by intermediate result point Local data storage cluster or the data storage cluster of distal end are dealt into, and provides the querying flow of efficiently and accurately.
As shown in Fig. 2 from the perspective of entity apparatus, Client modules can be deployed in Client servers, led to First processor and the first memory of Client servers are crossed to realize the function of Client modules.Wherein, described first deposit Reservoir is used for store instruction, and the first processor is used for the instruction for performing the first memory storage.Agent modules can be with It is deployed in Agent servers, Agent modules is realized by the second processor and second memory of Agent servers Function.Wherein, the second memory is used for store instruction, and the second processor is used to perform the second memory storage Instruction.
Correspondingly, the data query system of the embodiment of the present invention can include client Client servers and multiple agencies Agent servers, wherein, Client servers include first processor and first memory, and first memory refers to for storage Order, first processor are used for the instruction for performing first memory storage, and first memory and first processor are used to realize Client modules, each Agent servers in multiple Agent servers include second processor and second memory, and second Memory is used for store instruction, and second processor is used for the instruction for performing second memory storage, at second memory and second Reason device is used to realize Agent modules, the corresponding data storage cluster of each Agent modules, and the first of the first Agent servers The corresponding first data storage cluster of Agent modules.The Client modules are used to obtain serializing query statement set, the sequence The set of rowization query statement includes at least two query statements；The first Agent modules are used for according to the Client modules The serializing query statement set obtained, data are inquired about from the database of the first data storage cluster, obtain the One Query Result；The first Agent modules are additionally operable to first Query Result returning to the Client modules.
The data query system of the embodiment of the present invention, Client modules obtain serializing query statement set, and are handed down to Agent modules, as performing data query in data storage cluster of the Agent modules corresponding to it, and by final inquiry knot Fruit returns to Client modules, it is possible to reduce the transmission of invalid data in system, the bandwidth availability ratio between data storage cluster is improved, And the interaction times between Client modules and data storage cluster can be reduced, it is possible to increase inquiry response speed.
Alternatively, as one embodiment, the serializing query statement set includes N number of query statement, described N number of to look into Ask instruction includes query statement 1 ... according to the logical order of inquiry, query statement i ..., query statement N, wherein, query statement I corresponding first intermediate result i, N are more than or equal to 2, and the first Agent modules are specifically used for：The first Agent module roots According to the query statement 1, data are inquired about from the database of the first data storage cluster, obtain the first intermediate result 1；Institute Querying condition of the first Agent modules using the first intermediate result i-1 as the query statement i is stated, is deposited from first data Data are inquired about in the database of accumulation, obtain the first intermediate result i, wherein, i value is 2 ..., N, will when i is equal to N First intermediate result N is as first Query Result.
Alternatively, N number of query statement, the N are included as another embodiment, the serializing query statement set Individual query statement includes query statement 1 ... according to the logical order of inquiry, query statement i ..., query statement N, wherein, inquiry I corresponding first intermediate result i, N is instructed to be more than or equal to 2, the first Agent modules are specifically used for：The first Agent moulds Root tuber inquires about data from the database of the first data storage cluster, obtains the first intermediate result according to the query statement 1 1；The first Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 looking into as the query statement i Data storage cluster where the intermediate result i obtained during inquiry condition；When using the first intermediate result i-1 as the query statement When data storage cluster where the intermediate result i obtained during i querying condition is the first data storage cluster, described Querying condition of the one Agent modules using the first intermediate result i-1 as the query statement i, from first data store set Data are inquired about in the database of group, obtain the first intermediate result i, wherein, i value is 2 ..., N, when i is equal to N, by first Intermediate result N is as first Query Result；When using the first intermediate result i-1 as being obtained during query statement i querying condition When data storage cluster where the intermediate result i arrived is not the first data storage cluster, the first Agent modules will The first intermediate result i-1 is sent to the 2nd Agent modules corresponding to the second data storage cluster.
In a kind of scheme of the present embodiment, the metadata information can include the number of the first data storage cluster According to the first metadata information of data in storehouse, the first Agent modules are specifically used for：The first Agent modules are according to institute The first metadata information is stated, it is determined that using the first intermediate result i-1 as the intermediate result obtained during query statement i querying condition Data storage cluster where i is not the first data storage cluster；The first Agent modules tie the described first centre Fruit i-1 is sent to the second data storage cluster corresponding described second adjacent with the first data store set faciation Agent modules.
Wherein, the first Agent modules can be also used for：The first Agent modules are to the 2nd Agent modules Configured information is sent, the configured information is used to indicate to inquire about among by first in the first data storage cluster As a result the intermediate result i obtained during querying conditions of the i-1 as query statement i, and using the first intermediate result i-1 as inquiry Data storage cluster where the intermediate result i obtained during the querying condition for instructing i is not the first data storage cluster.
In another scheme of the present embodiment, the metadata information can include the first metadata information and second yuan Data message, first metadata information for the first data storage cluster database in data metadata information, Second metadata information is number in the database of other data storage clusters in addition to the first data storage cluster According to metadata information, the first Agent modules are specifically used for：The first Agent modules are according to first metadata Information and second metadata information, it is determined that using the first intermediate result i-1 as being obtained during query statement i querying condition Data storage cluster where intermediate result i is the second data storage cluster.
Alternatively, the first Agent modules are additionally operable to：The first Agent modules receive the 3rd Agent modules and sent The 3rd intermediate result i-1, the first Agent modules are using the 3rd intermediate result i-1 the looking into as the query statement i Inquiry condition, data are inquired about from the database of the first data storage cluster, obtain the first intermediate result i.
It should be understood that in various embodiments of the present invention, Agent modules and Client modules can be realized by hardware, can also Corresponding software is performed by hardware to realize.Each Agent servers can be with corresponding data storage cluster physically or ground Disposed in reason together, can also separate and dispose.Agent servers can be an equipment in data storage cluster, should Agent servers can be communicated with the miscellaneous equipment in data storage cluster or data storage cluster beyond equipment, The embodiment of the present invention is not construed as limiting to this.
Those of ordinary skill in the art are it is to be appreciated that the list of each example described with reference to the embodiments described herein Member and algorithm steps, it can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually Performed with hardware or software mode, application-specific and design constraint depending on technical scheme.Professional and technical personnel Described function can be realized using distinct methods to each specific application, but this realization is it is not considered that exceed The scope of the present invention.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, the corresponding process in preceding method embodiment is may be referred to, will not be repeated here.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, can be with Realize by another way.For example, device embodiment described above is only schematical, for example, the unit Division, only a kind of division of logic function, can there is other dividing mode, such as multiple units or component when actually realizing Another system can be combined or be desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or The mutual coupling discussed or direct-coupling or communication connection can be the indirect couplings by some interfaces, device or unit Close or communicate to connect, can be electrical, mechanical or other forms.
The unit illustrated as separating component can be or may not be physically separate, show as unit The part shown can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple On NE.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs 's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, can also That unit is individually physically present, can also two or more units it is integrated in a unit.
If the function is realized in the form of SFU software functional unit and is used as independent production marketing or in use, can be with It is stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words The part to be contributed to prior art or the part of the technical scheme can be embodied in the form of software product, the meter Calculation machine software product is stored in a storage medium, including some instructions are causing a computer equipment (can be People's computer, server, or network equipment etc.) perform all or part of step of each embodiment methods described of the present invention. And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (Read-Only Memory, ROM), arbitrary access are deposited Reservoir (Random Access Memory, RAM), magnetic disc or CD etc. are various can be with the medium of store program codes.
The foregoing is only a specific embodiment of the invention, but protection scope of the present invention is not limited thereto, any Those familiar with the art the invention discloses technical scope in, change or replacement can be readily occurred in, should all be contained Cover within protection scope of the present invention.Therefore, protection scope of the present invention should be defined by scope of the claims.

Claims (14)

  1. A kind of 1. method for inquiring about data, it is characterised in that methods described is performed by data query system, the data query system System include client Client modules and it is multiple act on behalf of Agent modules, each Agent modules in the multiple Agent modules A corresponding data storage cluster, wherein, the corresponding first data storage cluster of the first Agent modules, the first Agent modules For inquiring about data from the database of the first data storage cluster, methods described includes：
    The Client modules obtain serializing query statement set, and the serializing query statement set is looked into including at least two Ask instruction；
    The first Agent modules are according to the serializing query statement set, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first Query Result；
    First Query Result is returned to the Client modules by the first Agent modules.
  2. 2. according to the method for claim 1, it is characterised in that the serializing query statement set includes N number of inquiry and referred to Order, N number of query statement include query statement 1 ... according to the logical order of inquiry, query statement i ..., query statement N, Wherein, query statement i corresponding first intermediate result i, N is more than or equal to 2,
    The first Agent modules are according to the serializing query statement set, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first Query Result, including：
    The first Agent modules inquire about number according to the query statement 1 from the database of the first data storage cluster According to obtaining the first intermediate result 1；
    Querying condition of the first Agent modules using the first intermediate result i-1 as the query statement i, from described first Data are inquired about in the database of data storage cluster, obtain the first intermediate result i, wherein, i value is 2 ..., N, when i is equal to During N, using the first intermediate result N as first Query Result.
  3. 3. according to the method for claim 1, it is characterised in that the serializing query statement set includes N number of inquiry and referred to Order, N number of query statement include query statement 1 ... according to the logical order of inquiry, query statement i ..., query statement N, Wherein, query statement i corresponding first intermediate result i, N is more than or equal to 2,
    The first Agent modules are according to the serializing query statement set, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first Query Result, including：
    The first Agent modules inquire about number according to the query statement 1 from the database of the first data storage cluster According to obtaining the first intermediate result 1；
    The first Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as the query statement i's Data storage cluster where the intermediate result i obtained during querying condition；
    When using the first intermediate result i-1 as the data where the intermediate result i obtained during the querying condition of the query statement i When storage cluster is the first data storage cluster, the first Agent modules look into the first intermediate result i-1 as described in Instruction i querying condition is ask, data is inquired about from the database of the first data storage cluster, obtains the first intermediate result i, Wherein, i value is 2 ..., N, when i is equal to N, using the first intermediate result N as first Query Result；
    When using the first intermediate result i-1 as the data storage where the intermediate result i obtained during query statement i querying condition When cluster is not the first data storage cluster, the first intermediate result i-1 is sent to by the first Agent modules 2nd Agent modules corresponding to two data storage clusters.
  4. 4. according to the method for claim 3, it is characterised in that the metadata information includes first data store set First metadata information of data in the database of group,
    The first Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as query statement i inquiry Data storage cluster where the intermediate result i obtained during condition, including：
    The first Agent modules are according to first metadata information, it is determined that using the first intermediate result i-1 as query statement Data storage cluster where the intermediate result i obtained during i querying condition is not the first data storage cluster；
    The first intermediate result i-1 is sent to second corresponding to the second data storage cluster by the first Agent modules Agent modules, including：
    The first intermediate result i-1 is sent to the first data store set faciation neighbour's by the first Agent modules The 2nd Agent modules corresponding to the second data storage cluster.
  5. 5. according to the method for claim 4, it is characterised in that methods described also includes：
    The first Agent modules send configured information to the 2nd Agent modules, and the configured information is used to indicate Inquired about in the first data storage cluster using the first intermediate result i-1 as being obtained during query statement i querying condition Intermediate result i, and using the first intermediate result i-1 as where the intermediate result i that is obtained during query statement i querying condition Data storage cluster be not the first data storage cluster.
  6. 6. according to the method for claim 3, it is characterised in that the metadata information includes the first metadata information and the Binary data information, first metadata information are believed for the metadata of data in the database of the first data storage cluster Breath, second metadata information are in the database of other data storage clusters in addition to the first data storage cluster The metadata information of data,
    The first Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as query statement i inquiry Data storage cluster where the intermediate result i obtained during condition, including：
    The first Agent modules are according to first metadata information and second metadata information, it is determined that by first Between querying conditions of the result i-1 as query statement i when obtained intermediate result i where data storage cluster be described second Data storage cluster.
  7. 7. the method according to any one of claim 2 to 6, it is characterised in that the first Agent modules are according to Query statement set is serialized, data is inquired about from the database of the first data storage cluster, obtains the first Query Result, Also include：
    The first Agent modules receive the 3rd intermediate result i-1 that the 3rd Agent modules are sent, the first Agent modules Querying condition using the 3rd intermediate result i-1 as the query statement i, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first intermediate result i.
  8. 8. a kind of data query system, it is characterised in that the data query system includes client Client servers and more It is individual to act on behalf of Agent servers, wherein, Client servers include first processor and first memory, the first memory For store instruction, the first processor is used for the instruction for performing first memory storage, the first memory and The first processor is used to realize Client modules, and each Agent servers in the multiple Agent servers include the Two processors and second memory, the second memory are used for store instruction, and the second processor is used to performing described the The instruction of two memory storages, the second memory and the second processor are used to realize Agent modules, each Agent The corresponding data storage cluster of module, the corresponding first data storage cluster of the first Agent modules of the first Agent servers,
    The Client modules are used to obtain serializing query statement set, and the serializing query statement set includes at least two Individual query statement；
    The first Agent modules are used for the serializing query statement set obtained according to the Client modules, from institute State in the database of the first data storage cluster and inquire about data, obtain the first Query Result；
    The first Agent modules are additionally operable to first Query Result returning to the Client modules.
  9. 9. data query system according to claim 8, it is characterised in that the serializing query statement set includes N Individual query statement, N number of query statement include query statement 1 ... according to the logical order of inquiry, query statement i ..., looked into Command N is ask, wherein, query statement i corresponding first intermediate result i, N are more than or equal to 2,
    The first Agent modules are specifically used for：
    The first Agent modules inquire about number according to the query statement 1 from the database of the first data storage cluster According to obtaining the first intermediate result 1；
    Querying condition of the first Agent modules using the first intermediate result i-1 as the query statement i, from described first Data are inquired about in the database of data storage cluster, obtain the first intermediate result i, wherein, i value is 2 ..., N, when i is equal to During N, using the first intermediate result N as first Query Result.
  10. 10. data query system according to claim 8, it is characterised in that the serializing query statement set includes N Individual query statement, N number of query statement include query statement 1 ... according to the logical order of inquiry, query statement i ..., looked into Command N is ask, wherein, query statement i corresponding first intermediate result i, N are more than or equal to 2,
    The first Agent modules are specifically used for：
    The first Agent modules inquire about number according to the query statement 1 from the database of the first data storage cluster According to obtaining the first intermediate result 1；
    The first Agent modules are according to metadata information, it is determined that using the first intermediate result i-1 as the query statement i's Data storage cluster where the intermediate result i obtained during querying condition；
    When using the first intermediate result i-1 as the data where the intermediate result i obtained during the querying condition of the query statement i When storage cluster is the first data storage cluster, the first Agent modules look into the first intermediate result i-1 as described in Instruction i querying condition is ask, data is inquired about from the database of the first data storage cluster, obtains the first intermediate result i, Wherein, i value is 2 ..., N, when i is equal to N, using the first intermediate result N as first Query Result；
    When using the first intermediate result i-1 as the data storage where the intermediate result i obtained during query statement i querying condition When cluster is not the first data storage cluster, the first intermediate result i-1 is sent to by the first Agent modules 2nd Agent modules corresponding to two data storage clusters.
  11. 11. data query system according to claim 10, it is characterised in that the metadata information includes described first First metadata information of data in the database of data storage cluster,
    The first Agent modules are specifically used for：
    The first Agent modules are according to first metadata information, it is determined that using the first intermediate result i-1 as query statement Data storage cluster where the intermediate result i obtained during i querying condition is not the first data storage cluster；
    The first intermediate result i-1 is sent to the first data store set faciation neighbour's by the first Agent modules The 2nd Agent modules corresponding to the second data storage cluster.
  12. 12. data query system according to claim 11, it is characterised in that the first Agent modules are additionally operable to：
    The first Agent modules send configured information to the 2nd Agent modules, and the configured information is used to indicate Inquired about in the first data storage cluster using the first intermediate result i-1 as being obtained during query statement i querying condition Intermediate result i, and using the first intermediate result i-1 as where the intermediate result i that is obtained during query statement i querying condition Data storage cluster be not the first data storage cluster.
  13. 13. data query system according to claim 10, it is characterised in that the metadata information includes first yuan of number It is believed that breath and the second metadata information, first metadata information is data in the database of the first data storage cluster Metadata information, second metadata information is other data storage clusters in addition to the first data storage cluster Database in data metadata information,
    The first Agent modules are specifically used for：
    The first Agent modules are according to first metadata information and second metadata information, it is determined that by first Between querying conditions of the result i-1 as query statement i when obtained intermediate result i where data storage cluster be described second Data storage cluster.
  14. 14. the data query system according to any one of claim 9 to 13, it is characterised in that the first Agent moulds Block is additionally operable to：
    The first Agent modules receive the 3rd intermediate result i-1 that the 3rd Agent modules are sent, the first Agent modules Querying condition using the 3rd intermediate result i-1 as the query statement i, from the data of the first data storage cluster Data are inquired about in storehouse, obtain the first intermediate result i.
CN201610850631.1A 2016-09-26 2016-09-26 Inquire about the method and data query system of data Pending CN107870924A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610850631.1A CN107870924A (en) 2016-09-26 2016-09-26 Inquire about the method and data query system of data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610850631.1A CN107870924A (en) 2016-09-26 2016-09-26 Inquire about the method and data query system of data

Publications (1)

Publication Number Publication Date
CN107870924A true CN107870924A (en) 2018-04-03

Family

ID=61750966

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610850631.1A Pending CN107870924A (en) 2016-09-26 2016-09-26 Inquire about the method and data query system of data

Country Status (1)

Country Link
CN (1) CN107870924A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114860971A (en) * 2022-07-07 2022-08-05 墨研计算科学（南京）有限公司 Test data query method, terminal, server and system
CN115604146A (en) * 2022-11-30 2023-01-13 广东睿江云计算股份有限公司（Cn） Method and system for continuously acquiring K8s cluster condition

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5881232A (en) * 1996-07-23 1999-03-09 International Business Machines Corporation Generic SQL query agent
US6604135B1 (en) * 1995-06-07 2003-08-05 International Business Machines Corporation WWW client server dynamic interactive system method
CN101334773A (en) * 2007-06-28 2008-12-31 联想(北京)有限公司 Method for filtrating search engine searching result
CN102254031A (en) * 2011-08-03 2011-11-23 无锡浙潮科技有限公司 Batch processing request-based Microsoft SQL server database cluster
CN103116655A (en) * 2013-03-06 2013-05-22 亿赞普(北京)科技有限公司 Clustered data query method, client side and system
CN103136364A (en) * 2013-03-14 2013-06-05 曙光信息产业(北京)有限公司 Cluster database system and data query processing method thereof
CN103617231A (en) * 2013-11-26 2014-03-05 国家电网公司 Large data management system
CN104077294A (en) * 2013-03-27 2014-10-01 腾讯科技（深圳）有限公司 Information recommendation method, information recommendation device and information resource recommendation system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6604135B1 (en) * 1995-06-07 2003-08-05 International Business Machines Corporation WWW client server dynamic interactive system method
US5881232A (en) * 1996-07-23 1999-03-09 International Business Machines Corporation Generic SQL query agent
CN101334773A (en) * 2007-06-28 2008-12-31 联想(北京)有限公司 Method for filtrating search engine searching result
CN102254031A (en) * 2011-08-03 2011-11-23 无锡浙潮科技有限公司 Batch processing request-based Microsoft SQL server database cluster
CN103116655A (en) * 2013-03-06 2013-05-22 亿赞普(北京)科技有限公司 Clustered data query method, client side and system
CN103136364A (en) * 2013-03-14 2013-06-05 曙光信息产业(北京)有限公司 Cluster database system and data query processing method thereof
CN104077294A (en) * 2013-03-27 2014-10-01 腾讯科技（深圳）有限公司 Information recommendation method, information recommendation device and information resource recommendation system
CN103617231A (en) * 2013-11-26 2014-03-05 国家电网公司 Large data management system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114860971A (en) * 2022-07-07 2022-08-05 墨研计算科学（南京）有限公司 Test data query method, terminal, server and system
CN115604146A (en) * 2022-11-30 2023-01-13 广东睿江云计算股份有限公司（Cn） Method and system for continuously acquiring K8s cluster condition

Similar Documents

Publication Publication Date Title
CN110399373A (en) A kind of block chain account book storage system, storage querying method and delet method
CN105446991B (en) Date storage method, querying method and equipment
CN107679192A (en) More cluster synergistic data processing method, system, storage medium and equipment
CN1794729B (en) Data arrangement management method, data arrangement management system and data arrangement management device
CN103365929B (en) The management method of a kind of data base connection and system
CN103309738B (en) User job dispatching method and device
CN101901242A (en) Federated configuration data management
CN109978546A (en) A kind of alliance's block chain framework and its classification storage and transaction method for punching
CN107291928A (en) A kind of daily record storage system and method
CN108694195A (en) A kind of management method and system of Distributed Data Warehouse
CN104346458B (en) Date storage method and storage device
CN104885078A (en) Method for two-stage query optimization in massively parallel processing database clusters
CN107247758B (en) Data processing method based on bank core system
CN102075402A (en) Virtual network mapping processing method and system
CN104317957B (en) A kind of open platform of report form processing, system and report processing method
CN107016069A (en) Towards the metadata interchange system of intelligent transportation
CN109145188A (en) For searching for the method, equipment and computer readable storage medium of block chain data
CN101944113A (en) Cloud computing system-based data acquisition system
CN107547606A (en) Data processing method, cluster manager dual system, explorer, data handling system
CN106815258A (en) A kind of date storage method and coordinator node
CN105471893B (en) A kind of distributed equivalent data flow connection method
CN111984718A (en) Information management platform of government affair information system based on data sharing exchange
CN107870924A (en) Inquire about the method and data query system of data
Gong et al. VEGA infrastructure for resource discovery in grids
CN103377236B (en) A kind of Connection inquiring method and system for distributed data base

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180403

RJ01 Rejection of invention patent application after publication