CN106202102B

CN106202102B - Batch data querying method and device

Info

Publication number: CN106202102B
Application number: CN201510226374.XA
Authority: CN
Inventors: 李丰; 张赟; 王蕾; 冯晓兵
Original assignee: Huawei Technologies Co Ltd; Institute of Computing Technology of CAS
Current assignee: Huawei Technologies Co Ltd; Institute of Computing Technology of CAS
Priority date: 2015-05-06
Filing date: 2015-05-06
Publication date: 2019-04-05
Anticipated expiration: 2035-05-06
Also published as: WO2016177027A1; US10678789B2; EP3282370A1; EP3282370A4; CN106202102A; US20180060392A1

Abstract

The embodiment of the present invention provides a kind of batch data querying method and device, data query server connect according in N query statement operator and operand determine N query statement operand symbol logo, symbol logo includes the version number of operand, then the dependence between N query statement is determined according to the version number of the operand of N query statement, so according between N query statement dependence and the preset principle of optimality to N query statement inquire between optimize.Since the symbol logo of the operand of N query statement is fixed, it will not change with the difference of the query statement of input, therefore, the symbol logo for the symbolic number that the embodiment of the present invention determines can be applicable in all input sets, and any part of any query statement is not needed to execute and without the data access and update status in monitoring and inquiry sentence implementation procedure, the efficiency optimized between inquiry is improved, and reduces the expense optimized between inquiry.

Description

Batch data querying method and device

Technical field

The present embodiments relate to field of computer technology more particularly to a kind of batch data querying methods and device.

Background technique

Current representative big data inquiry system (such as: Hive, Shark, Impala) using single query statement as The basic unit of parsing and optimization.One key performance of big data inquiry system is search efficiency.However, in data warehouse Under the batch query scene of (Data Warehouse), it is traditional with single query statement be parsing and optimization basic unit place Reason mode there is a problem of optimizing chance deficiency.It is data warehouse batch with optimize the composition sharp contrast of chance deficiency in inquiry Optimize chance between the inquiry abundant showed in inquiry application scenarios, optimizes between chance, that is, a plurality of query statement between inquiry Existing optimization chance.

Under batch query application scenarios, in the prior art, by query statement implementation procedure to data record Update status is monitored in real time and is fed back, and by first carrying out the modes such as the part of functions in query statement in advance, dynamic Ground obtain it is each inquire the data record that specifically needs to access, with this determine between data record operated by multiple queries whether In the presence of conflict or intersection, and some dynamic optimizations are executed based on above-mentioned analysis.But it is looked by monitoring data record or execution part Dynamic data dependence relevant to a certain group of input can only be collected by asking function, based on performed by the dependence of these dynamic datas Optimization can only be adapted to a certain group and specifically input, once input changes, it is necessary to re-execute analysis and optimization.

Summary of the invention

The embodiment of the present invention provides a kind of batch data querying method and device, improves the efficiency optimized between inquiry, and And reduce the overhead optimized between inquiry.

First aspect present invention provides a kind of batch data querying method, comprising:

Receive N pending query statement, wherein the N is the positive integer not less than 2；

According in the N query statement operator and operand determine the N query statement operand symbol Number mark, wherein the operator is used to indicate operation to be performed, and the operand is used to indicate the N query statement In operator data to be operated storage location, the symbol logo includes the version number of operand, also, is referred to identical The operand of data version number having the same, the operand for referring to different data have different version numbers, the operator Include at least: creation operator destroys operator, scan operation symbol and file definite value operator；

According to the version number of the operand of the determining N query statement determine between the N query statement according to The relationship of relying；

According between the N query statement dependence and the preset principle of optimality to the N query statement Optimize between being inquired；

Query statement after executing optimization is to obtain the query result of the N query statement.

With reference to first aspect, in the first possible implementation of the first aspect, described that language is inquired according to the N item Operator and operand in sentence determine that the symbol logo of the operand of the N query statement includes:

Obtain the corresponding N logical query plan tree of the N query statement, wherein a query statement is one corresponding Logical query plan tree；

Symbol logo is added for the operand of the first kind operator in the N logical query plan tree, wherein described First kind operator includes: creation operator, destroys operator, scan operation symbol and file definite value operator；

Each query plan tree in the N logical query plan tree is performed the following operations respectively:

According to the first generic operation in the topological order of the first logical query plan tree, the first logical query plan tree The symbol logo of the operand of symbol and preset addition rule are the second generic operation in the first logical query plan tree The operand of symbol adds symbol logo, wherein the first logical query plan tree is in the N logical query plan tree Any one logical query plan tree, the second generic operation symbol is other operators in addition to the first kind operator.

The possible implementation of with reference to first aspect the first, in second of possible implementation of first aspect In, the root node of the first logical query plan tree includes file definite value operator, the first logical query plan tree Leaf node includes file scan operator, the internal node of the first logical query plan tree include the second generic operation symbol, It creates operator and destroys operator, wherein the internal node is other nodes in addition to leaf node and root node；Institute Stating addition rule includes: that each of the first logical query plan tree the second generic operation is accorded with, and is performed the following operations:

It is described if the operand of the first operator is identical as the operand of left child nodes of first operator The operand of first operator adds symbol identical with the symbol logo of operand of left child nodes of first operator Number mark, first operator be second generic operation symbol in any one operator；

If the operand of first operator is identical as the operand of right child nodes of first operator, for The operand addition of first operator is identical as the symbol logo of operand of right child nodes of first operator Symbol logo.

With reference to first aspect, the first of first aspect is any one of to second of possible implementation, first In the third possible implementation of aspect, the principle of optimality includes at least one of following rules rule:

Delete the query statement that with the first query statement there is same operation number version number and same operation to accord with, wherein institute Stating the first query statement is any one query statement in the N query statement；

The search order for having between the query statement of stream dependence is kept, and will there are stream the multiple of dependence to look into It askes sentence and is optimized for a new query statement, wherein the stream dependence refers to query statement in preceding execution Version number's phase of the version number of the operand of file definite value operator and the operand of another query statement in rear execution Together；With

Merge with same operation symbol and operand has the query statement of overlapping.

The possible implementation of with reference to first aspect the first, in the 4th kind of possible implementation of first aspect In, the symbol logo of the operand in the N query statement further include: dsc data mark；

The operator and operand according in the N query statement determines the operand of the N query statement Symbol logo further include:

Count the number that the operand of each scan operation symbol in the N logical query plan tree is cited；

Whether the number for judging that the operand of each scan operation symbol in the N logical query plan tree is cited is big In dsc data threshold value；

The number being cited for the operand that scan operation in the N logical query plan tree accords with is greater than the dsc data The operand addition dsc data mark of threshold value, the dsc data mark is for indicating pointed by the operand with dsc data mark Data be dsc data；

The method also includes:

During executing the query statement after the optimization, concurrently executes and identified comprising the dsc data and stream is not present Query statement after the optimization of dependence and output dependence.

With reference to first aspect the first or the 4th kind of possible implementation, in the 5th kind of possible reality of first aspect In existing mode, the symbol logo of the operand in the N query statement further include: starting enlivens position and termination enlivens position It sets, the operator and operand according in the N query statement determines the symbol of the operand of the N query statement Mark further include:

For first operand, the mark that is accorded with according to the scan operation for quoting the first operand for the first time and described sweep The serial number for retouching the logical query plan tree where operator determines that the starting of the first operand enlivens position, wherein described First operand is any one operand in the operand in the N query statement；

According to patrolling where the mark of the destruction operator for destroying the first operand and the destruction operator The serial number for collecting query plan tree determines that the termination of the first operand enlivens position；

The method also includes:

During executing the query statement after the optimization, position release is enlivened according to the termination of the first operand The memory space of the data of the first operand instruction.

The 5th kind of possible implementation with reference to first aspect, in the 6th kind of possible implementation of first aspect In, the operator and operand according in the N query statement determines the symbol of the operand of the N query statement Mark further include:

For second operand, according to the mark for the first file definite value operator for quoting the second operand for the first time Determine that the starting of the second operand is living with the serial number of the logical query plan tree where the first file definite value operator Jump position, wherein the first file definite value operator is used to write data to the storage location that the second operand refers to, institute Second operand is stated as any one operand in the operand in the N query statement；

According to the mark for the second file definite value operator for quoting the second operand and the second file definite value behaviour The serial number of logical query plan tree where according with determines that the termination of the second operand enlivens position, wherein described second File definite value operator is used to rewrite the data that the second operand of the first file definite value operator operation is directed toward；

The method also includes:

During executing the query statement after the optimization, position release is enlivened according to the termination of the second operand The memory space of the data of the second operand instruction.

Second aspect of the present invention provides a kind of data query server, comprising:

Receiving module, for receiving N pending query statement, wherein the N is the positive integer not less than 2；

Identify determining module, for according in the N query statement operator and operand determine that the N item is inquired The symbol logo of the operand of sentence, wherein the operator is used to indicate operation to be performed, and the operand is used to indicate The storage location of operator in N query statement data to be operated, the symbol logo includes the version of operand Number, also, the operand version number having the same of identical data is referred to, the operand for referring to different data has different versions This number, the operator includes at least: creation operator destroys operator, scan operation symbol and file definite value operator；

Relationship determination module, the operand of the N query statement for being determined according to the mark determining module Version number determines the dependence between the N query statement；

Optimization module, for according between the N query statement dependence and the preset principle of optimality to institute It states between N query statement is inquired and optimizes；

Enquiry module, for executing the query statement after optimizing to obtain the query result of the N query statement.

In conjunction with second aspect, in the first possible implementation of the second aspect, the mark determining module is specific For:

In conjunction with the first possible implementation of second aspect, in second of possible implementation of second aspect In, the root node of the first logical query plan tree includes file definite value operator, the first logical query plan tree Leaf node includes file scan operator, the internal node of the first logical query plan tree include the second generic operation symbol, It creates operator and destroys operator, wherein the internal node is other nodes in addition to leaf node and root node；Institute Stating addition rule includes: that each of the first logical query plan tree the second generic operation is accorded with, and is performed the following operations:

In conjunction with second aspect, second aspect the first any one of to second of possible implementation, second In the third possible implementation of aspect, the principle of optimality includes at least one of following rules rule:

In conjunction with the first possible implementation of second aspect, in the 4th kind of possible implementation of second aspect In, the symbol logo of the operand in the N query statement further include: dsc data mark, the mark determining module are also used In:

The enquiry module is also used to:

In conjunction with the first or the 4th kind of possible implementation of second aspect, in the 5th kind of possible reality of second aspect In existing mode, the symbol logo of the operand in the N query statement further include: starting enlivens position and termination enlivens position It sets, the mark determining module is also used to:

The enquiry module is also used to:

In conjunction with the 5th kind of possible implementation of second aspect, in the 6th kind of possible implementation of second aspect In, the mark determining module is also used to:

The enquiry module is also used to:

Batch data querying method and device provided in an embodiment of the present invention, data query server were connected according to N item Operator and operand in query statement determine that the symbol logo of the operand of N query statement, symbol logo include operation Then several version numbers determines the dependence between N query statement according to the version number of the operand of N query statement, And then according between N query statement dependence and the preset principle of optimality to N query statement inquire between it is excellent Change.Due to the symbol logo of the operand of N query statement be it is fixed, will not become with the difference of the query statement of input Change, therefore, the present embodiment is proposed that a kind of static analysis and maintenance technology between data flow relation inquiry, i.e., it is a kind of with Input data is unrelated, does not need any part for executing any query statement and without in monitoring and inquiry sentence implementation procedure Data access and update status, improve the efficiency that optimizes between inquiry, and reduce the expense optimized between inquiry.

Detailed description of the invention

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described.

Fig. 1 is the flow chart for the batch data querying method that the embodiment of the present invention one provides；

Fig. 2 is the flow chart of the symbol logo of the operand of determining N query statement provided by Embodiment 2 of the present invention；

Fig. 3 is the structural schematic diagram for the data query server that the embodiment of the present invention three provides；

Fig. 4 is the structural schematic diagram for the data query server that the embodiment of the present invention four provides.

Specific embodiment

In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is A part of the embodiment of the present invention, instead of all the embodiments.

The method of various embodiments of the present invention is mainly used under the scene of batch data inquiry.Batch data inquiry is usually adopted With customer end/server mode, multiple data query servers, storage system and a large amount of client are generally included in database End, storage system may include one or more storage equipment.During a batch query, multiple client all may be to Data query server sends query statement, when data query server determines that the quantity of the query statement received reaches default Quantity, then batch query is carried out to a plurality of query statement, alternatively, data query server is to receiving in preset time period All query statements carry out batch query.One common scene of batch data inquiry is data warehouse.Data warehouse is decision The structural data environment of support system and on-line analysis application data source.Data warehouse is mainly studied and is solved from database The problem of obtaining information.Data warehouse is characterized in that subject-oriented, integration, stability and time variation, in data warehouse There are a large amount of batch data enquiry machine meeting, there are a large amount of data in data warehouse, these data are generally stored inside distribution In storage system.

Fig. 1 is the flow chart for the batch data querying method that the embodiment of the present invention one provides, method provided in this embodiment It can be executed by data query server, as shown in Figure 1, method provided in this embodiment may comprise steps of:

Step 101 receives N pending query statement, and N is the positive integer more than or equal to 2.

Step 102, according in N query statement operator and operand determine N query statement operand symbol Number mark.

It include multiple operators in every query statement, each operator quotes one or more operands, wherein operation Symbol is used to indicate operation to be performed, and common operator includes: scan operation symbol (Scan Operator), file definite value behaviour It accords with (Filesink Operator), creation operator, destroy operator, sorting operation symbol (Sort Operator), selection Operator (Select Operator), aggregation operator accord with (Aggregate Operator), product operator (Product Operator) and attended operation accords with (Join Operator) etc..

In the present embodiment, operand does not imply that specific data record, but is used to indicate operator and waits operating Data storage location, i.e. operand is corresponding with storage location.Operand can be a variable or expression formula, operand institute The storage location of instruction can be a tables of data, a subregion in tables of data or field in tables of data etc..This reality It applies in example, the symbol logo of operand includes the version number of operand, also, the operand for referring to identical data is having the same Version number, the operand for referring to different data have different version numbers.In the database, two operators possess behaviour of the same name It counts (such as: a), does not represent the two and centainly operate identical data；Similarly, two operators possess operand not of the same name (such as: a and b) does not represent data operated by the two yet and is different.Therefore, the operand of N query statement is being determined When symbol logo, only cannot determine whether two operands are identical according to the title of two operands, and should be grasped according to two Count reference data two operands of whether identical determination it is whether identical, wherein operand refer to data, that is, operand institute The data stored in the storage location of instruction.

Data query server can inquire language according to N item when determining the symbol logo of operand of N query statement The sequencing that sequence and every query statement internal operators between sentence execute determines all behaviour of N query statement The symbol logo counted.In a kind of implementation, data query server first obtains the corresponding N logic of N query statement and looks into Ask plan tree, wherein the corresponding logical query plan tree of a query statement, each node of every logical query plan tree For an operator.Then, symbol logo is added for the operand of the first kind operator in N logical query plan tree, In, first kind operator includes: creation operator, destroys operator, scan operation symbol and file definite value operator.It is being N It to be also N logical query plan before the operand addition symbol logo of first kind operator in logical query plan tree The operand of the first kind operator of tree generates symbol logo, and the symbol logo of generation includes the version number of operand.

It, will also basis after adding symbol logo for the operand of the first kind operator of N logical query plan tree The symbol logo of the operand of first kind operator is that the operand of the second generic operation symbol adds symbol logo.Specifically, for N Each query plan tree in logical query plan tree performs the following operations respectively:

According to the first kind operator in the topological order of the first logical query plan tree, the first logical query plan tree The symbol logo of operand and preset addition rule are the operation of the second generic operation symbol in the first logical query plan tree Number addition symbol logo.Wherein, the first logical query plan tree is any one Boolean query in N logical query plan tree Plan tree, the second generic operation symbol are other operators in addition to the first kind operator.In the present embodiment, the first logic is looked into The node of inquiry plan tree includes: root node, leaf node and internal node, and the top for being in logical query plan tree (does not have Parents) node be root node, be in the node of the lowermost end (without child) of logical query plan tree for leaf node, inside Node is the node that existing parents also have child.The topological order of first logical query plan tree refers to from leaf node to root section The sequence of point.

In the present embodiment, the root node of the first logical query plan tree includes file definite value operator, the first Boolean query The leaf node of plan tree includes file scan operator, and the internal node of the first logical query plan tree includes the second generic operation Symbol, creation operator and destruction operator.Wherein, internal node is other nodes in addition to leaf node and root node.If First logical query plan tree is binary tree, then the internal node of the first logical query plan tree have left child nodes with One right child nodes.Above-mentioned addition rule includes: that each of the first logical query plan tree the second generic operation is accorded with, It performs the following operations: being if the operand of the first operator is identical as the operand of the left child nodes of the first operator The operand of one operator adds symbol mark identical with the symbol logo of the operand of the left child nodes of the first operator Know,；If the operand of the first operator is identical as the operand of right child nodes of the first operator, for the first operator Operand adds symbol logo identical with the symbol logo of operand of right child nodes of the first operator.Wherein, first Operator is any one operator in the second generic operation symbol.

In the present embodiment, the left child nodes of the first operator and right child nodes may be first kind operator, can also It can be the second generic operation symbol, specifically, when the left child nodes of the first operator and right child nodes are leaf node, first The left child nodes of operator and right child nodes are first kind operators.Left child nodes and right child when the first operator When node is internal node, the left child nodes of the first operator and right child nodes are the second generic operation symbols.For example, it is assumed that the One logical query plan tree shares 4 layers, and the node that first layer includes is root node, and the node that the second layer and third layer include is interior Portion's node, the 4th layer of node for including are leaf node.In the behaviour of the second generic operation symbol for the first logical query plan tree It counts when adding symbol logo, is first the operation on the internal node of third layer according to the operand on the 4th layer of leaf node Number addition symbol logo, the 4th layer of leaf node are first kind operator, and the internal node of third layer is the second generic operation symbol, Symbol logo is added according to the operand that the operand of first kind operator is the second generic operation symbol.In owning for third layer It is the second layer according to the operand of all internal nodes of third after the operand of internal node has all added symbol logo All internal nodes add symbol logos, and second layer internal node is that the second generic operation accords with, at this moment can be according to the of third layer The symbol logo of the operand of two generic operations symbol is that the operand that the second generic operation of the second layer accords with adds symbol logo.

Step 103, according to the version number of the operand of N determining query statement determine between N query statement according to The relationship of relying.

Dependence between N sentence may include: stream dependence, output dependence, operator overlapping relation, Operand overlapping relation.Wherein, stream dependence refers to the behaviour in the file definite value operator of a query statement of preceding execution The version number counted is identical as the version number of operand that the scan operation of another query statement in rear execution accords with.Output according to Bad relationship refers in the version number of the operand of the file definite value operator of a query statement of preceding execution by rear execution The file definite value institute definite value of another query statement, i.e., in the file definite value operator of rear execution to the file definite value in preceding execution The operand of operator is rewritten.Operator overlapping relation refers to that the quantity of all kinds of operators of two query statements is homogeneous Together, operand overlapping relation refers to that the version number of all or part of operand of two query statements is identical.

Step 104, according between N query statement dependence and the preset principle of optimality to N query statement Optimize between being inquired.

Optimization is integrally to optimize for more logical query plan trees, that is, there are logics for optimization chance between inquiry Between query plan tree, rather than optimized for single logical query plan tree.The principle of optimality includes in following rules At least one rule: (1) deleting the query statement that with the first query statement there is same operation number version number and same operation to accord with, Wherein, the first query statement is any one query statement in N query statement；(2) keep that there is looking into for stream dependence The search order between sentence is ask, and will there is the multiple queries sentence of stream dependence to be optimized for a new query statement； (3) merge with same operation symbol and operand has the query statement of overlapping.

The principle of optimality (1) is directed to optimize between the inquiry with the query statement of operator overlapping relation, will be with first The query statement that there is query statement same operation number version number and same operation to accord with is known as the second query statement, looks into due to first Asking sentence and the second query statement, there is same operation number version number and same operation to accord with, therefore, the first query statement and second The query result of query statement is identical, can delete the second query statement.If all by the first query statement and the second query statement It is parsed into logical query plan subtree, then the corresponding first Boolean query tree of the first query statement and the second query statement are corresponding Second logical query plan tree has common query subtree, i.e. the first query sub tree of the first logical query plan tree is patrolled with second The tree construction for collecting the second query sub tree of query plan tree is identical, and the operand of each operator of the first query sub tree The version number of Data Identification and the operand of each operator of the second query sub tree is all the same.By to common query subtree Optimization chance optimizes, and avoids duplicate the inquiring repeatedly to same query result, reduces opening for data base querying Pin, improves the efficiency of data base querying.

The principle of optimality (2) is directed to optimize between the inquiry of the query statement with stream dependence, can will hold preceding Subsequent being directly connected to of the forerunner of capable file definite value operator and the scan operation symbol in rear execution, and delete in rear execution Scan operation symbol, in this way in preceding execution file definite value operator after having executed, directly by the output knot of file definite value operator Fruit is handled as the input that scan operation accords with, and does not need first to deposit the output result write-in distribution of file definite value operator Then storage system is read out again, to reduce the read-write expense to distributed memory system, improve search efficiency.

The principle of optimality (3) is directed to optimize between the inquiry with the query statement of operand overlapping relation, if first looks into The first operator for asking sentence is identical as the second operator of the second query statement, and the part operation number of the first operator Version number is identical as the version number of part operation number of the second operator, then subsequent in query process, inquires for first First operator and second operator are merged into same when sentence and the second query statement generate physics query tree Business, operand and nonoverlapping operation tree to overlapping are inquired respectively, due to merging for the same task, only to overlapping Operand carry out one query, to reduce the scan operation expense to overlapped data.

Step 105 executes the query statement after optimizing to obtain the query result of N query statement.

In the present embodiment, data query server connected according in N query statement operator and operand determine N The symbol logo of the operand of query statement, symbol logo includes the version number of operand, then according to N query statement The version number of operand determines the dependence between N query statement, and then according to the dependence between N query statement And the preset principle of optimality to N query statement inquire between optimize.Due to the symbol mark of the operand of N query statement Knowledge be it is fixed, will not change with the difference of the query statement of input, therefore, the present embodiment is proposed that a kind of to looking into The static analysis and maintenance technology of data flow relation, i.e., a kind of unrelated with input data between inquiry, does not need to execute any inquiry Any part of sentence and without the data access and update status in monitoring and inquiry sentence implementation procedure improves between inquiry The efficiency of optimization, and reduce the expense optimized between inquiry.

On the basis of example 1, optionally, the symbol logo of the operand in N query statement can also include: Dsc data mark, then according in N query statement operator and operand determine N query statement operand symbol mark Know further include: the number that the operand of each scan operation symbol in N logical query plan tree of statistics is cited；Judge N Whether the number that the operand of each scan operation symbol in logical query plan tree is cited is greater than dsc data threshold value；It is N The operand that the number that the operand that scan operation accords in logical query plan tree is cited is greater than dsc data threshold value adds hot number According to mark, dsc data mark is for indicating that data pointed by the operand with dsc data mark are dsc data.After correspondingly, Continue execute optimization after query statement during, can concurrently execute comprising dsc data identify and there is no stream dependence and Query statement after exporting the optimization of dependence.Alternatively, under the premise of stream between not changing inquiry relies on and output relies on, it is right The corresponding operator of operand comprising dsc data mark reorders, and continuously performs them, to improve to dsc data Access efficiency.

Optionally, the symbol logo of the operand in N query statement can also include: that starting enlivens position and terminates work Jump position, then according in N query statement operator and operand determine N query statement operand symbol logo also It include: that institute is accorded with according to the mark of the scan operation symbol of first time reference first operand and scan operation for first operand The serial number of logical query plan tree determine that the starting of first operand enlivens position.Wherein, first operand is the inquiry of N item Any one operand in operand in sentence.According to the mark and pin of the destruction operator for destroying first operand The serial number of the logical query plan tree ruined where operator determines that the termination of first operand enlivens position.Correspondingly, it is subsequent During query statement after executing optimization, release first operand instruction in position can be enlivened according to the termination of first operand Data memory space.Specifically, data query server, which enlivens position according to the termination of first operand, can determine One operand is inactive after termination enlivens position, should as early as possible will be occupied by it for inactive operand Memory space discharged, for other operands occupancy, so as to improve the utilization rate of memory space.Data query clothes Business device can also enliven position according to the starting of first operand and what termination enlivened that position determines first operand enlivens section, For enlivening the operand of section shorter (for example, only in some inquiry actively), if the operand is stored in distribution In storage system, then by optimization, the storage location of the operand can be changed to data query clothes by distributed memory system The local disk or memory of business device, to reduce solidification and access expense.

It, in embodiments of the present invention can be in addition, for any one operand in the operand in N query statement Referred to as second operand.For second operand, according to the first file definite value operator of first time reference second operand The serial number of logical query plan tree where mark and the first file definite value operator determines that the starting of second operand enlivens position It sets.Wherein, the first file definite value operator is used to write data to the storage location that second operand refers to.It is grasped according to reference second The serial number of logical query plan tree where the mark for the second file definite value operator counted and the second file definite value operator Determine that the termination of second operand enlivens position, wherein the second file definite value operator is for rewriting the operation of the first file definite value The data that the second operand of symbol operation is directed toward.Correspondingly, it is subsequent during executing the query statement after optimization, according to second The termination of operand enlivens the memory space of the data of position release second operand instruction.It should be noted that in the present embodiment Sequence of the mark for each operator mentioned for identification operation symbol, is specifically as follows the identity number of operator (Identity, abbreviation ID).

On the basis of the above embodiment 1, the embodiment of the present invention two specifically will for example illustrate how to determine by one The symbol logo of the operand of N query statement, Fig. 2 are the operation of determining N query statement provided by Embodiment 2 of the present invention The flow chart of several symbol logos, as shown in Fig. 2, method provided in this embodiment may comprise steps of:

Step 201 creates symbol logo table, and every data record in the symbol logo table includes following field: version Number, reference, definite value, active, inactive and path.

In the present embodiment, a symbol logo table is created first, which is used to safeguard the behaviour of N query statement The symbol logo counted.Table-like structure is not the unique forms for being stored symbols mark, can also be deposited using chained list, hash table etc. Storage form stored symbols mark.

Every data record includes following field in the symbol logo table: version number, reference, definite value, it is active, inactive and Path, wherein version number field records the version number of corresponding operand for storing data, and reference field is for storing reference The logical query plan tree of the mark and scan operation symbol place of the scan operation symbol of the corresponding operand of the data record Serial number, valued field are used to store the mark and this document of the file definite value operator of the corresponding operand of the definite value data record The serial number of logical query plan tree where definite value operator, path field is for storing the corresponding operand of the data record Storage location enlivens field for storing the mark and the scanning behaviour that quote the scan operation symbol of the corresponding operand of the data record The serial number of logical query plan tree where according with, or the file definite value for storing the corresponding operand of the definite value data record The mark of operator and the serial number of the logical query plan tree where this document definite value operator, inactive field is for storing pin The mark for ruining the destruction operator of the corresponding operand of the data record and the logical query plan tree where the destruction operator Serial number, or for storing the mark and this document definite value of nullifying the file definite value operator of the corresponding operand of the data record The serial number of logical query plan tree where operator.

Step 202, the value for initializing version number field.

For example, the value of version number field is initialized as -1, one data of every increase is recorded in the symbol logo table later, The value of version number field adds 1.

Step 203, according to the sequence of N logical query plan tree, successively traverse every logical query plan tree, be every The operand of creation operator, scan operation symbol, file definite value operator and destruction operator in logical query plan tree is raw At corresponding data record, wherein the sequence of N logical query plan tree is consistent with the input sequence of N query statement.

It can be specifically the creation operator in every logical query plan tree, scan operation by following the following two kinds mode Symbol, file definite value operator and the corresponding data record of operand generation for destroying operator, first way are as follows:

When traversing i-th logical query plan tree, if in i-th logical query plan tree including creation operator, Data record is created for the operand of the creation operator of i-th logical query plan tree, by i-th logical query plan The storage of the operand of the creation operator of the version number of the operand of the creation operator of tree and i-th logical query plan tree The corresponding relationship of position is saved in mapping table.In the present embodiment, mapping table is used to save the version number of operand With the corresponding relationship of the storage location of operand, i is the number of N logical query plan tree, and the initial value of i is the value of 1, i Are as follows: it is more than or equal to 1, and is less than or equal to the integer of N.For the operand creation of the creation operator of i-th logical query plan tree One data record specifically: add 1 to obtain i-th Boolean query the current value of the version number field in the symbol logo table The version number for planning the operand of the creation operator of tree, by the operand of the creation operator of i-th logical query plan tree Version number be added to i-th logical query plan tree creation operator the corresponding data record of operand version number's word Duan Zhong, other fields of the corresponding data record of operand of the creation operator of i-th logical query plan tree are sky.

If including destroying operator in i-th logical query plan tree, according to the destruction of i-th logical query plan tree The storage location of the operand of operator searches the destruction operator of i-th logical query plan tree from the mapping table Operand the corresponding version number of storage location.If finding the operation of the destruction operator of i-th logical query plan tree The corresponding version number of several storage locations, then according to the storage of the operand of the destruction operator of i-th logical query plan tree The operand pair of the destruction operator of i-th logical query plan tree is searched from the symbol logo table by the corresponding version number in position The data record answered, by the destruction operator of the serial number i of i-th logical query plan tree and i-th logical query plan tree Mark, is added to the inactive field of the corresponding data record of operand of the destruction operator of i-th logical query plan tree In.

If including that scan operation accords in i-th logical query plan tree, according to the scanning of i-th logical query plan tree The storage location of the operand of operator searches the scan operation symbol of i-th logical query plan tree from mapping table The corresponding version number of the storage location of operand.

If finding the corresponding version of storage location of the operand of the scan operation symbol of i-th logical query plan tree Number, then according to the corresponding version number of storage location of the operand of the scan operation of i-th logical query plan tree symbol from the symbol The corresponding data record of operand that the scan operation of i-th logical query plan tree accords with is searched in number mark table.It is finding After the corresponding data record of operand of the scan operation symbol of i-th logical query plan tree, i-th logical query plan is judged The corresponding data record of operand of the file scan operator of tree enlivens whether field is empty.If i-th Boolean query meter The value for enlivening field for drawing the corresponding data record of operand of the file scan operator of tree is sky, then looks into i-th logic The mark for asking the serial number i of plan tree and the scan operation symbol of i-th logical query plan tree, is added to i-th Boolean query meter It draws the reference field of the corresponding data record of operand of the scan operation symbol of tree and enlivens in field.If i-th Boolean query That plans the corresponding data record of operand of the file scan operator of tree enlivens field not and is empty, then looks into i-th logic The mark for asking the serial number i of plan tree and the scan operation symbol of i-th logical query plan tree, is added to i-th Boolean query meter In the reference field for drawing the corresponding data record of operand of the scan operation symbol of tree.If not finding i-th Boolean query The corresponding version number of storage location of the operand of the scan operation symbol of plan tree, then sweeping for i-th logical query plan tree The operand for retouching operator creates data record, by the operand of the scan operation symbol of i-th logical query plan tree The corresponding relationship of the storage location of the operand of the scan operation symbol of version number and i-th logical query plan tree is saved in this and reflects It penetrates in relation table, and by the scan operation of the serial number i of i-th logical query plan tree and i-th logical query plan tree symbol Mark is added to the reference field of newly created data record and enlivens in field.

In the present embodiment, if the scan operation symbol of i-th logical query plan tree is found in the mapping table The corresponding version number of the storage location of operand, the then operand that scan operation accords in i-1 logical query plan tree before illustrating It is identical as the operand of the scan operation of i-th logical query plan tree symbol, therefore, generated for the identical operand Version number.If not finding the operand of the scan operation symbol of i-th logical query plan tree in the mapping table The corresponding version number of storage location then illustrates that the operand of the scan operation symbol of i-th logical query plan tree occurs for the first time, There are no the operand build versions number of the scan operation symbol for i-th logical query plan tree.

If including file definite value operator in i logical query plan tree, according to the text of i-th logical query plan tree The storage location of the operand of part definite value operator searches the file of i-th logical query plan tree from the mapping table The corresponding version number of the storage location of the operand of definite value operator.If the file for finding i-th logical query plan tree is fixed The corresponding version number of storage location of the operand of Value Operations symbol, then grasp according to the file definite value of i-th logical query plan tree The text of i-th logical query plan tree is searched from the symbol logo table by the corresponding version number of storage location for making the operand accorded with The corresponding data record of operand of part definite value operator, in the file definite value operation for finding i-th logical query plan tree After the corresponding data record of the operand of symbol, judge that the operand of the file operator of i-th logical query plan tree is corresponding Whether the value of the valued field of data record is empty.

If the valued field of the corresponding data record of operand of the file definite value operator of i-th logical query plan tree For sky, then first by the file definite value operator of the serial number i of i-th logical query plan tree and i-th logical query plan tree Mark, be added to the corresponding data record of operand of the file definite value operator of i-th logical query plan tree It is that the operand of the file definite value operator of i-th logical query plan tree creates data note then in inactive field Record, and by the version number of the operand of the file definite value operator of i-th logical query plan tree and i-th logical query plan The corresponding relationship of the storage location of the operand of the file definite value operator of tree is saved in the mapping table, and i-th is patrolled The mark for collecting the serial number i of query plan tree and the file definite value operator of i-th logical query plan tree, is added to newly created It the valued field of data record and enlivens in field.

It in the first way, is successively in the following order every logic when traversing every logical query plan tree The operand build version number of query plan tree: creation operator destroys operator, referencing operation symbol and the operation of file definite value Symbol.It, can when for the operand build version of every logical query plan tree in other possible implementations of the invention With the sequence not accorded with according to aforesaid operations.It should be noted that if being in other orders the behaviour of every logical query plan tree When build version of counting, when creating data record every time, will first it inquire in mapping table, if mapping table In do not find the corresponding version number of operand of current operator, then for current operator operand generate it is new Data record.

In the first way, it is only necessary to can be completed by once traversing to the operand of N logical query plan tree The generation of version number.In second in mode, need by repeatedly traversing the operation that could be completed to N logical query plan tree The generation of several version numbers, traversal is only an operand build version number for a kind of particular operators every time, in second in mode, It being also required to first establish the symbol logo table, each field of symbol logo table is also identical with embodiment one, unlike, the present embodiment In, mapping table is for saving as the Boolean query where the version number of the storage location of operand to operand and operand The corresponding relationship for the binary group that the serial number of plan tree is constituted.The second way is as follows:

Firstly, successively being patrolled N according to N logical query plan tree of order traversal of N logical query plan tree The operand for collecting each file definite value operator of query plan tree creates data record, operates to each file definite value When the operand of symbol creates data record, the value after the current value of version number field adds 1 in the symbol logo table is used as ought be above The version number of the operand of current file definite value operator is added to current number by the version number of the operand of part definite value operator According to the version number field of record, by the logic where the mark of current file definite value operator and the current file definite value operator The serial number of query plan tree is added to the valued field of the current data record and enlivens in field.Then, by the current file The storage location of the operand of definite value operator to the current file definite value operator operand version number and should be above The corresponding relationship for the binary group that the serial number of logical query plan tree where part definite value operator is constituted is saved in mapping table In.

Secondly, N logical query plan tree is traversed according to second of sequence of N logical query plan tree, successively to N The destruction operator of logical query plan tree is handled as follows: (1) according to the storage position of the current operand for destroying operator Lookup mapping table is set, the corresponding all versions by operand of storage location for currently destroying the operand of operator are obtained Number and logical query plan tree serial number composition binary group set.(2) logical query plan is selected from the binary group set The serial number of tree is less than all alternating logic query plan trees of the current serial number for destroying the logical query plan tree where operator, The maximum logical query plan tree of serial number is determined from alternating logic query plan tree, finds this most from the binary group set Binary group where small serial number obtains the version number in the binary group where the smallest sequence number.(3) according to where the smallest sequence number Binary group in version number search the symbol logo table, the version number found in the binary group where the maximum serial number is corresponding Data record, by the current mark for destroying operator and the current serial number for destroying the logical query plan tree where operator, It is added in the inactive field of the corresponding data record of version number in the binary group where the smallest sequence number.If do not looked for To the version number in the binary group where the maximum serial number, then jumps to next destruction operator and execute above-mentioned processing.

Later, N logical query plan tree is traversed according to the sequence third time of N logical query plan tree, successively to N Each scan operation symbol of logical query plan tree executes following processing: (1) depositing according to the operand of current scan operation symbol Storage space is set, and mapping table is searched, and the storage location for obtaining the operand of current scan operation symbol is corresponding all by operand Version number and logical query plan tree serial number composition binary group set.(2) logic is selected from the binary group set to look into All alternating logics for asking the logical query plan tree that plan tree serial number is greater than or equal to where current scan operation symbol inquire meter Tree is drawn, the smallest logical query plan tree of serial number is determined from alternating logic query plan tree, is looked for from the binary group set Binary group to where the smallest sequence number obtains the version number in the binary group where the smallest sequence number.(3) the smallest sequence number institute Binary group in version number search the symbol logo table, the version number found in the binary group where the smallest sequence number is corresponding Data record, by current scan operation symbol mark and the current scan operation symbol where logical query plan tree sequence Number, it is added in the reference field of the corresponding data record of version number in the binary group where the maximum serial number.It (4) if should The field of enlivening of the corresponding data record of the version number in binary group where smallest sequence number is sky, then accords with current scan operation Mark and the current scan operation symbol where logical query plan tree serial number, the binary being added to where the smallest sequence number The corresponding data record of version number in group is enlivened in field.If in the binary group where not finding the smallest sequence number Version number then newly creates data record in the symbol logo table, adds 1 to obtain currently the maximum value of version number field The version number of the operand of current scan operation symbol is added to newly created data by the version number of the operand of scan operation symbol In the version number field of record, the storage location of the operand of current scan operation symbol is added to newly created data record In path field, by the serial number of the mark of current scan operation symbol and the logical query plan tree at the place of current scan operation symbol It is added to the reference field of newly created data record and enlivens in field.

Finally, the N logical query plan tree of backward the 4th time traversal of the sequence according to N logical query plan tree, according to Secondary each creation operator to N logical query plan tree executes following processing: (1) according to the operation of current creation operator Several storage locations, search mapping table, obtain currently create operator operand storage location it is corresponding it is all by The binary group set of the serial number composition of the version number and logical query plan tree of operand.(2) it is selected from the binary group set The serial number of logical query plan tree is more than or equal to all standby of the serial number of the logical query plan tree where current creation operator Select logical query plan tree, the smallest logical query plan tree of serial number determined from alternating logic query plan tree, from this two The binary group where the smallest sequence number is found in tuple-set, obtains the version number in the binary group where the smallest sequence number.(3) The symbol logo table is searched by the version number in binary group where the smallest sequence number, finds the binary where the smallest sequence number The corresponding data record of version number in group judges the corresponding data record of version number in the binary group where the smallest sequence number Path field whether be it is empty, if it is empty, then the storage location of the operand of current creation operator is added to the most foreword In the path field of the corresponding data record of version number in binary group where number, if not empty, then next wound is jumped to It builds operator and executes method as described above.

It is successively in the following order every logic when traversing every logical query plan tree in the second way The operand build version number of query plan tree: operator, referencing operation symbol, file definite value operator and creation operation are destroyed Symbol.In the second way, in N logical query plan tree of the 4th traversal, according to the sequence of N logical query plan tree Backward traverses N logical query plan tree.To N logical query plan of order traversal using N logical query plan tree Tree, then when according with build version for referencing operation, if not finding the version number of the condition of satisfaction in mapping table, Then the operand for current referencing operation symbol generates new data record.In the present embodiment, by for every logical query plan The operand build version number of all operators in tree, and the version number of generation is added in logical query plan tree, with Convenient for subsequent in optimization procedures, N logical query plan tree is optimized according to the version number of operand.

On the basis of example 2, data query device is counting each scanning behaviour in N logical query plan tree When making the number that the operand that accords with is cited, what can be recorded according to every data in symbol logo table enlivens field and non-live The value for the field that jumps, the starting of all operands enlivens position in statistical symbol mark table and termination enlivens position.Specifically, first First judge current data record enlivens whether field and the value of inactive field are empty, if current data record enlivens word The value of section and inactive field is sky, then illustrating that current data records corresponding operand in entire batch query process In, it is all inactive, current data can be recorded and be deleted from symbol logo table.If current data record enlivens field and non- It is not empty for enlivening the value of field, then according to the mark for enlivening the operator in field and the operator of current data record The serial number of affiliated logical query plan tree determines that current data records the starting of corresponding operand and enlivens position, according to current The serial number of logical query plan tree belonging to the mark and the operator of operator in the inactive field of data record determines The termination that current data records corresponding operand enlivens position, only one in field of enlivening of current data record operates Symbol, the operator may be scan operation symbol or file definite value operator, also only have in the inactive field of current data record One operator, the operator may be destruction operator or file definite value operator.

Fig. 3 is the structural schematic diagram for the data query server that the embodiment of the present invention three provides, as shown in figure 3, this implementation The data query server that example provides includes: receiving module 11, mark determining module 12, relationship determination module 13, optimization module 14 and enquiry module 15.

Wherein, receiving module 11, for receiving N pending query statement, wherein the N is just whole not less than 2 Number；

Identify determining module 12, for according in the N query statement operator and operand determine that the N item is looked into Ask the symbol logo of the operand of sentence, wherein the operator is used to indicate operation to be performed, and the operand is for referring to Show the storage location of operator in N query statement data to be operated, the symbol logo includes the version of operand This number, also, the operand version number having the same of identical data is referred to, the operand of different data is referred to different Version number, the operator include at least: creation operator destroys operator, scan operation symbol and file definite value operator；

Relationship determination module 13, the operation of the N query statement for being determined according to the mark determining module 12 Several version numbers determines the dependence between the N query statement；

Optimization module 14, for according to the dependence and the preset principle of optimality pair between the N query statement The N query statement optimizes between being inquired；

Enquiry module 15, for executing the query statement after optimizing to obtain the query result of the N query statement.

Optionally, the mark determining module 12 is specifically used for: obtaining the corresponding N logic of the N query statement and looks into Ask plan tree, wherein the corresponding logical query plan tree of a query statement；For in the N logical query plan tree The operand of first kind operator adds symbol logo, wherein the first kind operator includes: creation operator, destroys behaviour It accords with, scan operation symbol and file definite value operator；For each query plan tree in the N logical query plan tree It performs the following operations respectively: according in the topological order of the first logical query plan tree, the first logical query plan tree The symbol logo of the operand of first kind operator and preset addition rule are in the first logical query plan tree The operand of second generic operation symbol adds symbol logo, wherein the first logical query plan tree is the N Boolean query Any one logical query plan tree in plan tree, the second generic operation symbol are its in addition to the first kind operator His operator.

In the present embodiment, the root node of the first logical query plan tree includes file definite value operator, and described first The leaf node of logical query plan tree includes file scan operator, the internal node packet of the first logical query plan tree Include the second generic operation symbol, creation operator and destroy operator, wherein the internal node be except leaf node and root node it Other outer nodes；The addition rule includes: for each of the first logical query plan tree the second generic operation Symbol, performs the following operations:

It is described if the operand of the first operator is identical as the operand of left child nodes of first operator The operand of first operator adds symbol identical with the symbol logo of operand of left child nodes of first operator Number mark, first operator be second generic operation symbol in any one operator；If first operator Operand is identical as the operand of right child nodes of first operator, then adds for the operand of first operator Symbol logo identical with the symbol logo of operand of right child nodes of first operator.

Optionally, the principle of optimality includes at least one of following rules rule: deleting and has with the first query statement There is the query statement of same operation number version number and same operation symbol, wherein first query statement is that the N item inquires language Any one query statement in sentence；The search order for having between the query statement of stream dependence is kept, and there will be stream The multiple queries sentence of dependence is optimized for a new query statement, wherein the stream dependence refers in preceding execution A query statement file definite value operator operand version number in another query statement of rear execution The version number of operand is identical；Merge with same operation symbol and operand has the query statement of overlapping.

Optionally, the symbol logo of the operand in the N query statement further include: dsc data mark, the mark Determining module 12 is also used to: counting what the operand that each scan operation in the N logical query plan tree accords with was cited Number；Whether the number for judging that the operand of each scan operation symbol in the N logical query plan tree is cited is greater than Dsc data threshold value；The number being cited for the operand that scan operation in the N logical query plan tree accords with is greater than the heat The operand addition dsc data mark of data threshold, the dsc data mark is for indicating the operand institute with dsc data mark The data of direction are dsc data.Correspondingly, the enquiry module 15 is also used to: the query statement process after executing the optimization In, concurrently executing includes dsc data mark and there is no the inquiry languages after the optimization of stream dependence and output dependence Sentence.

Optionally, the symbol logo of the operand in the N query statement further include: starting enlivens position and terminates and lives Jump position, and the mark determining module 12 is also used to: first operand is directed to, according to quoting the first operand for the first time The mark of scan operation symbol and the serial number of the logical query plan tree where scan operation symbol determine the first operand Starting enliven position, wherein the first operand be the N query statement in operand in any one operation Number；Boolean query where the mark of the destruction operator for destroying the first operand and the destruction operator The serial number of plan tree determines that the termination of the first operand enlivens position.Correspondingly, the enquiry module 15 is also used to: holding During query statement after the row optimization, release first operation in position is enlivened according to the termination of the first operand The memory space of the data of number instruction.

Optionally, the mark determining module 12 is also used to: second operand is directed to, according to quoting described second for the first time Logical query plan tree where the mark of first file definite value operator of operand and the first file definite value operator Serial number determine that the starting of the second operand enlivens position, wherein the first file definite value operator is used for described The storage location that second operand refers to writes data, and the second operand is in the operand in the N query statement Any one operand；According to the mark for the second file definite value operator for quoting the second operand and second file The serial number of logical query plan tree where definite value operator determines that the termination of the second operand enlivens position, wherein institute State what the second file definite value operator was directed toward for rewriting the second operand of the first file definite value operator operation Data.The enquiry module 15 is also used to: during executing the query statement after the optimization, according to the second operand Termination enliven the memory space that position discharges the data of second operand instruction.

Data query server provided in this embodiment, the method that can be used for executing embodiment one and embodiment two, specifically Implementation is similar with technical effect, and which is not described herein again.

Fig. 4 is the structural schematic diagram for the data query server that the embodiment of the present invention four provides, as shown in figure 4, this implementation The data query server 200 that example provides includes: processor 21, memory 22, communication interface 23 and system bus 24, described to deposit Reservoir 22 and the communication interface 23 are connected and communicated by the system bus 24 with the processor 21；The memory 22, for storing computer executed instructions；The communication interface 23 is used for and other equipment are communicated, the processor 21, For running the computer executed instructions, execution following methods:

Optionally, the processor 21 according in the N query statement operator and operand determine that the N item is looked into The symbol logo of the operand of sentence is ask, specifically:

Optionally, the symbol logo of the operand in the N query statement further include: dsc data mark；The processing Device 21 is also used to: counting the number that the operand of each scan operation symbol in the N logical query plan tree is cited；Sentence Whether the number that is cited of operand of each scan operation symbol in the N logical query plan tree of breaking is greater than dsc data threshold Value；The number being cited for the operand that scan operation in the N logical query plan tree accords with is greater than the dsc data threshold value Operand addition dsc data mark, number pointed by operand of the dsc data mark for indicating to identify with dsc data According to for dsc data.It is subsequent during executing the query statement after the optimization, concurrently execute comprising dsc data mark and There is no the query statements after the optimization of stream dependence and output dependence.

Optionally, the symbol logo of the operand in the N query statement further include: starting enlivens position and terminates and lives Jump position, and the processor 31 is also used to: for first operand, being grasped according to the scanning for quoting the first operand for the first time The serial number for making the logical query plan tree where the mark accorded with and scan operation symbol determines the starting of the first operand Enliven position, wherein the first operand is any one operand in the operand in the N query statement；Root According to the logical query plan where the mark and the destruction operator of the destruction operator for destroying the first operand The serial number of tree determines that the termination of the first operand enlivens position.The subsequent query statement process after executing the optimization In, the memory space that position discharges the data of the first operand instruction is enlivened according to the termination of the first operand.

The processor 31 is also used to: second operand is directed to, according to quoting the first of the second operand for the first time The serial number of logical query plan tree where the mark of file definite value operator and the first file definite value operator determines institute The starting for stating second operand enlivens position, wherein the first file definite value operator to the second operand for referring to The storage location in generation writes data, and the second operand is any one operation in the operand in the N query statement Number；According to the mark for the second file definite value operator for quoting the second operand and the second file definite value operator institute The serial number of logical query plan tree determine that the termination of the second operand enlivens position, wherein second file is fixed The data that Value Operations symbol is directed toward for rewriting the second operand of the first file definite value operator operation.It is subsequent to hold During query statement after the row optimization, release second operation in position is enlivened according to the termination of the second operand The memory space of the data of number instruction.

The embodiment of the present invention also provides a kind of computer program product of data processing, the meter including storing program code Calculation machine readable storage medium storing program for executing, the instruction that said program code includes is for executing side described in any one aforementioned embodiment of the method Method process.It will appreciated by the skilled person that storage medium above-mentioned include: USB flash disk, mobile hard disk, magnetic disk, CD, Random access memory (Random-Access Memory, RAM), solid state hard disk (Solid State Disk, SSD) or non-volatile Various non-transitory (the non-that can store program code such as property memory (non-volatile memory) Transitory) machine readable media.

It should be noted that embodiment provided herein is only schematical.Those skilled in the art can To be well understood, for convenience of description and succinctly, in the above-described embodiments, all respectively there is side to the description of each embodiment It is heavy, there is no the part being described in detail in some embodiment, reference can be made to the related descriptions of other embodiments.In the embodiment of the present invention, power The feature disclosed in benefit requirement and attached drawing can be individually present or combine presence.In embodiments of the present invention with hardware shape The feature of formula description can be executed by software, and vice versa.It is not limited here.

Claims

1. a kind of batch data querying method characterized by comprising

According in the N query statement operator and operand determine the N query statement operand symbol mark Know, wherein the operator is used to indicate operation to be performed, and the operand is used to indicate in the N query statement The storage location of operator data to be operated, the symbol logo include the version number of operand, also, refer to identical data Operand version number having the same, refer to different data operand have different version numbers, the operator is at least Include: creation operator, destroy operator, scan operation symbol and file definite value operator；

Determine that the dependence between the N query statement is closed according to the version number of the operand of the determining N query statement System；

According between the N query statement dependence and the preset principle of optimality N query statement is carried out Optimize between inquiry；

Query statement after executing optimization is to obtain the query result of the N query statement；

Wherein, the operator and operand according in the N query statement determines the operand of the N query statement Symbol logo include:

According between the N query statement sequence and every query statement internal operators execute it is successive suitable Sequence determines the symbol logo of all operands of the N query statement.

2. the method according to claim 1, wherein the sequence according between the N query statement, with And the sequencing that every query statement internal operators execute, determine all operands of the N query statement Symbol logo, comprising:

Obtain the corresponding N logical query plan tree of the N query statement, wherein the corresponding logic of a query statement Query plan tree；

Symbol logo is added for the operand of the first kind operator in the N logical query plan tree, wherein described first Generic operation symbol includes: creation operator, destroys operator, scan operation symbol and file definite value operator；

According to the first kind operator in the topological order of the first logical query plan tree, the first logical query plan tree The symbol logo of operand and preset addition rule are the second generic operation symbol in the first logical query plan tree Operand adds symbol logo, wherein the first logical query plan tree is any in the N logical query plan tree One logical query plan tree, the second generic operation symbol is other operators in addition to the first kind operator.

3. according to the method described in claim 2, it is characterized by: the root node of the first logical query plan tree includes text Part definite value operator, the leaf node of the first logical query plan tree include file scan operator, first logic The internal node of query plan tree includes the second generic operation symbol, creation operator and destroys operator, wherein the internal node For other nodes in addition to leaf node and root node；

The addition rule includes: that each of the first logical query plan tree the second generic operation is accorded with, and is executed such as Lower operation:

It is described first if the operand of the first operator is identical as the operand of left child nodes of first operator The operand of operator adds symbol mark identical with the symbol logo of operand of left child nodes of first operator Know, first operator is any one operator in second generic operation symbol；

It is described if the operand of first operator is identical as the operand of right child nodes of first operator The operand of first operator adds symbol identical with the symbol logo of operand of right child nodes of first operator Number mark.

4. method according to any one of claim 1-3, which is characterized in that the principle of optimality includes in following rules At least one rule:

Delete the query statement that with the first query statement there is same operation number version number and same operation to accord with, wherein described the One query statement is any one query statement in the N query statement；

Keep the search order for having between the query statement of stream dependence, and the multiple queries language that will there is stream dependence Sentence is optimized for a new query statement, wherein the stream dependence refers to the file in a query statement of preceding execution The version number of the operand of definite value operator is identical as the version number of operand of another query statement in rear execution；With

5. the method according to claim 1, wherein the symbol logo of the operand in the N query statement Further include: dsc data mark；

The operator and operand according in the N query statement determines the symbol of the operand of the N query statement Number mark further include:

Whether the number for judging that the operand of each scan operation symbol in the N logical query plan tree is cited is greater than heat Data threshold；

The number being cited for the operand that scan operation in the N logical query plan tree accords with is greater than the dsc data threshold value Operand addition dsc data mark, number pointed by operand of the dsc data mark for indicating to identify with dsc data According to for dsc data；

The method also includes:

During executing the query statement after the optimization, concurrently executes comprising dsc data mark and there is no streams to rely on Query statement after the optimization of relationship and output dependence.

6. according to the method described in claim 1, it is characterized by: the symbol logo of the operand in the N query statement Further include: starting enlivens position and termination enlivens position；

For first operand, the mark and the scanning accorded with according to the scan operation for quoting the first operand for the first time is grasped The serial number of logical query plan tree where according with determines that the starting of the first operand enlivens position, wherein described first Operand is any one operand in the operand in the N query statement；

Logic where the mark of the destruction operator for destroying the first operand and the destruction operator is looked into The serial number of inquiry plan tree determines that the termination of the first operand enlivens position；

The method also includes:

During executing the query statement after the optimization, position release is enlivened according to the termination of the first operand described in The memory space of the data of first operand instruction.

7. according to the method described in claim 6, it is characterized in that, the operator according in the N query statement and Operand determines the symbol logo of the operand of the N query statement further include:

For second operand, according to the mark for the first file definite value operator for quoting the second operand for the first time and institute The serial number for stating the logical query plan tree where the first file definite value operator determines that the starting of the second operand enlivens position It sets, wherein the first file definite value operator is used to write data to the storage location that the second operand refers to, and described the Two operands are any one operand in the operand in the N query statement；

According to the mark and the second file definite value operator of the second file definite value operator for quoting the second operand The serial number of the logical query plan tree at place determines that the termination of the second operand enlivens position, wherein second file Definite value operator is used to rewrite the data that the second operand of the first file definite value operator operation is directed toward；

The method also includes:

During executing the query statement after the optimization, position release is enlivened according to the termination of the second operand described in The memory space of the data of second operand instruction.

8. a kind of data query server characterized by comprising

Identify determining module, for according in the N query statement operator and operand determine the N query statement Operand symbol logo, wherein the operator is used to indicate operation to be performed, and the operand is used to indicate described The storage location of operator in N query statement data to be operated, the symbol logo includes the version number of operand, and And the operand version number having the same for referring to identical data, the operand for referring to different data have different version numbers, The operator includes at least: creation operator destroys operator, scan operation symbol and file definite value operator；

Relationship determination module, the version of the operand of the N query statement for being determined according to the mark determining module Number determine the dependence between the N query statement；

Optimization module, for according between the N query statement dependence and the preset principle of optimality to the N item Query statement optimizes between being inquired；

Enquiry module, for executing the query statement after optimizing to obtain the query result of the N query statement；

Wherein, the mark determining module, specifically for according between the N query statement sequence and every described in The sequencing that query statement internal operators execute, determines the symbol logo of all operands of the N query statement.

9. server according to claim 8, which is characterized in that the mark determining module is specifically used for:

10. server according to claim 9, it is characterised in that: the root node packet of the first logical query plan tree Including file definite value operator, the leaf node of the first logical query plan tree includes file scan operator, and described first The internal node of logical query plan tree includes the second generic operation symbol, creation operator and destroys operator, wherein the inside Node is other nodes in addition to leaf node and root node；

11. the server according to any one of claim 8-10, which is characterized in that the principle of optimality includes following rule At least one of then rule:

12. server according to claim 8, which is characterized in that the symbol mark of the operand in the N query statement Know further include: dsc data mark, the mark determining module are also used to:

The enquiry module is also used to:

13. server according to claim 8, which is characterized in that the symbol mark of the operand in the N query statement Know further include: starting enlivens position and termination enlivens position, and the mark determining module is also used to:

The enquiry module is also used to:

14. server according to claim 13, which is characterized in that the mark determining module is also used to:

The enquiry module is also used to: