WO2018058671A1 - 执行多表连接操作的控制方法及对应装置 - Google Patents

执行多表连接操作的控制方法及对应装置 Download PDF

Info

Publication number
WO2018058671A1
WO2018058671A1 PCT/CN2016/101366 CN2016101366W WO2018058671A1 WO 2018058671 A1 WO2018058671 A1 WO 2018058671A1 CN 2016101366 W CN2016101366 W CN 2016101366W WO 2018058671 A1 WO2018058671 A1 WO 2018058671A1
Authority
WO
WIPO (PCT)
Prior art keywords
field
field group
group
execution
sub
Prior art date
Application number
PCT/CN2016/101366
Other languages
English (en)
French (fr)
Inventor
刘文洁
魏建生
张晓飞
陈雷
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201680089781.3A priority Critical patent/CN109791543B/zh
Priority to PCT/CN2016/101366 priority patent/WO2018058671A1/zh
Publication of WO2018058671A1 publication Critical patent/WO2018058671A1/zh
Priority to US16/370,343 priority patent/US11301470B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24542—Plan optimisation
    • G06F16/24544—Join order optimisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24537—Query rewriting; Transformation of operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G06F16/24558—Binary matching operations
    • G06F16/2456—Join operations

Definitions

  • the present invention relates to the field of computer technology, and more particularly to a database multi-table join query technology.
  • a relational database is a database created on the basis of a relational model.
  • a relational model is a two-dimensional table model used to organize data into multiple tables consisting of rows and columns.
  • the computer memory is addressed in the order of one-dimensional structure, so you can use the method of inputting each row of data in the table one by one (that is, the row storage), or importing the data of each column in the table in batches (that is, the column) Save).
  • join operations can be performed on the table based on the relational algebra, for example, theta connection is performed on the two tables, that is, the fields in the two tables satisfying the theta condition are merged.
  • the storage system and the database query engine are generally independent of each other, and the database query engine cannot sense the organization of data in the storage system.
  • the database query engine cannot sense the organization of data in the storage system.
  • it is usually necessary to first read all the fields involved in the query statement in the table and calculate the Cartesian product, and then filter out the connection result according to the query condition.
  • the Cartesian product calculation for all the fields involved in the query in the table will cause the data operation to expand dramatically, thereby greatly increasing the computational overhead, memory overhead, and network between nodes. Transmission overhead, severely degrading performance.
  • the present invention provides a control method and a corresponding device for performing a multi-table connection operation, to solve the problem of calculating a Cartesian for all fields involved in a query in a table when performing the multi-table theta connection operation in the prior art.
  • the amount of data calculation caused by the product is violently expanded, and the calculation overhead, memory overhead, and network transmission overhead between nodes are large, and the performance is low.
  • a first aspect of the present invention provides a control method for performing a multi-table connection operation, comprising the steps of:
  • the query statement is parsed, and the query condition, the table name, and the field corresponding to the table corresponding to the table name in the query statement are obtained.
  • the Cartesian product calculation causes the data operation amount to exceed the preset threshold.
  • the second type of table is a table other than the first type of table in the table corresponding to the table name.
  • the field group is used to implement the theta connection operation step by step in the form of multiple field groups, which can reduce the amount of data calculated by the Cartesian product in one connection operation, and greatly reduce the network transmission overhead, calculation overhead and memory overhead. Improve the efficiency of execution.
  • a second aspect of the present invention discloses an optimizer comprising: a memory for storing a program and data generated during program running; and a processor for performing the following functions by running a program in the memory:
  • Parsing the query statement obtaining a query condition, a table name, and a field corresponding to the table corresponding to the table name in the query statement; and decomposing each field in the first type table in a field involved in the query statement Obtaining a plurality of first field groups of each table in the first class table; and adopting a field in which each table in the second class table is involved in the query statement to form a second field group Obtaining a second field group of each table in the second type of table; generating an execution plan according to the query statement, the first field group, and the second field group.
  • the query condition indicates that the table participating in the ta connection operation, and the field involved in the query statement satisfies the first preset condition is a first type table, where the first preset condition is used to indicate that the table is in the
  • the Cartesian product calculation causes the data operation amount to exceed the preset threshold
  • the second type table is the table corresponding to the table name except the first type table. a table other than the same
  • the execution plan is configured to control reading data corresponding to the fields in the first field group and the second field group, and perform a connection operation that satisfies an execution condition on the read data according to the execution step, The result of the query that meets the requirements of the query statement.
  • the first preset condition includes:
  • the number of fields involved in the query statement exceeds a first predetermined threshold.
  • the storage overhead of the fields involved in the query statement exceeds the first preset space threshold.
  • the table refers to the number of fields involved in the theta connection operation in the query statement exceeding a second predetermined threshold. or,
  • the storage overhead of the field involved in the join operation of the table in the query statement exceeds the second preset space threshold.
  • the field involved in each query in the first class table is decomposed into a plurality of first field groups of each table in the first class table. ,include:
  • the field and the primary key participating in the node connection operation in the table respectively constitute a first sub-class field group, and the table is involved in the query statement but does not participate in the node connection operation.
  • the fields and primary keys form a second subclass of field groups;
  • the generating an execution plan according to the first field group and the second field group of the query statement including:
  • the field and the primary key participating in the node connection operation in the table constitute a first sub-class field group, including:
  • the generating an execution plan according to the query statement, the first type field group, the second type field group, and the third type field group including:
  • the first type of field group, the second type of field group, and the third type of field group are used as the field group to be constructed included in the execution plan.
  • theta connection execution condition and the original equivalent connection execution condition in the execution condition are generated.
  • the primary key of the first type of table is used as the equivalent connection parameter, and the condition for performing the equivalence connection on the intermediate result generated by the query condition for the first type field group and the second type field group is performed. , as a derived equivalence join condition in the execution condition.
  • the execution order of the execution condition is taken as the execution step.
  • the method further includes:
  • the first sub-class field group is decomposed according to a predetermined rule to obtain multiple fields.
  • the preset rule includes: decomposing a field in the first type field group that is connected with the same table into the field group, and decomposing the common field between any two field groups to satisfy the first
  • the third preset condition is used to determine the association between the two field groups formed by the decomposition.
  • the plurality of field groups obtained by the decomposition do not all include a primary key, adding a primary key in the field group not including the primary key, and returning execution according to the query statement, the first subclass field group, and the second sub
  • the class field group and the second field group generate steps of an execution plan.
  • the plurality of field groups obtained by the decomposition all include a primary key, returning execution is performed according to the query statement, the first sub-class field group, the second sub-class field group, and the second field group, and executing execution The steps of the plan.
  • the according to the query statement, the first sub-class field group, the The second sub-field group and the second field group, before generating the execution plan further include:
  • the first sub-class field group is decomposed to obtain a plurality of field groups.
  • the plurality of field groups obtained by decomposing the first sub-category field group satisfying the second preset condition do not include the primary key, adding a primary key to the field group not including the primary key, and returning to the execution basis
  • the query statement, the first sub-class field group, the second sub-class field group, and the second field group are configured to generate an execution plan.
  • the plurality of field groups obtained by decomposing the first sub-category field group satisfying the second preset condition all include a primary key, returning execution according to the query statement, the first sub-class field group, The second sub-category field group and the second field group are configured to generate an execution plan.
  • the second preset condition includes:
  • the number of the fields in the first sub-category field group exceeds a third preset threshold.
  • the storage overhead of the field in the first sub-class field group exceeds a third preset space threshold.
  • a third aspect of the present invention provides a control method for performing a multi-table connection operation, comprising the steps of:
  • An execution plan is received, the execution plan including a first field group and a second field group to be constructed, an execution step, and an execution condition.
  • the read data is grouped according to the first field group and the second field group to be constructed to form field group data.
  • the query results are mapped to a sequence of output fields.
  • the read data is grouped to form field group data, and the field group data is connected, and the The form of multiple field groups completes the node connection operation step by step, which can reduce the amount of data calculated by Cartesian product in one connection operation, greatly reduce network transmission overhead, computational overhead and memory overhead, and improve execution efficiency.
  • a second aspect of the invention discloses a server cluster comprising n servers, n being greater than or equal to 1, each of the servers comprising a processor and a memory; wherein:
  • the processor of the n servers constitutes a distributed computing platform, configured to receive an execution plan, where the execution plan includes a first field group and a second field group to be constructed, an execution step, and an execution condition; reading the to-be-built Data corresponding to the fields in the first field group and the second field group; grouping the read data according to the first field group and the second field group to be constructed to form field group data;
  • the field group data performs a connection operation that satisfies the execution condition, and obtains a query result; maps the query result into an output field sequence; and sends the output field sequence.
  • the memory of the n servers constitutes a distributed storage system for storing data corresponding to the fields.
  • the method further includes:
  • the execution condition includes a single table filter condition
  • filtering is performed on the read data according to the single table filter condition.
  • the grouping the data according to the first field group and the second field group to be constructed to form field group data including:
  • the filtered data is grouped according to the first field group and the second field group to be constructed to form field group data.
  • the grouping the read data according to the field group to be constructed, and after forming the field group data further includes:
  • the execution condition includes a native equivalence connection execution condition, and according to the execution step, the equivalence connection is performed on the field group data according to the native equivalence connection execution condition.
  • the step of connecting the field group data according to the performing step to obtain a query result that meets the execution condition including:
  • the field group data after performing the equivalence connection is connected, and the query result that meets the execution condition is obtained.
  • FIG. 1 is a schematic structural diagram of a database system of a distributed computing framework
  • FIG. 2 is a flowchart of a method for controlling a multi-table connection operation according to an embodiment of the present invention
  • FIG. 3 is a flowchart of a method for controlling a multi-table connection operation according to another embodiment of the present invention.
  • FIG. 4 is a flowchart of a method for controlling a multi-table connection operation according to another embodiment of the present invention.
  • FIG. 5 is a flowchart of a method for controlling a multi-table connection according to another embodiment of the present invention.
  • FIG. 6 is a diagram showing execution conditions of a field group according to another embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of an optimizer according to another embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a server cluster according to another embodiment of the present invention.
  • the library query engine 101 and the n servers 102 are a database system of a distributed computing framework, including data, according to an embodiment of the present invention
  • the library query engine 101 and the n servers 102 wherein the database query engine 101 includes an optimizer 11, each of which includes an executor 12 and a storage node 13, and the executors in the n servers 102 constitute a distributed computing platform, n
  • the storage nodes in server 102 form a distributed storage system.
  • the database query engine 101 receives the query statement input by the user, the optimizer 11 parses the query statement, and generates an execution plan; the database query engine 101 sends the execution plan to the n servers 102.
  • the distributed computing platform composed of the executors in the n servers 102 receives the execution plan, reads the storage nodes in the n servers 102 to form the field data stored in the distributed storage system according to the execution plan, processes the read field data and generates
  • the result of the query is mapped and the query result is mapped to a sequence of output fields, which is sent to the database query engine 101.
  • the control method for performing the multi-table connection operation is to solve the data calculation amount caused by calculating the Cartesian product of all the fields involved in the query in the table when performing the multi-table theta connection operation in the prior art. Severe expansion, calculation of overhead, memory overhead and network transmission overhead between nodes, low performance.
  • a control method for performing a multi-table connection operation disclosed in an embodiment of the present invention is applied to an optimizer 11, and the control method includes:
  • the database query engine 101 inputs a query statement, and after receiving the query statement, the database query engine 101 forwards the query to the optimizer 11.
  • the optimizer 11 receives the query statement, parses the query statement to obtain the query condition, the table name, and the field corresponding to the table corresponding to the table name in the query statement; wherein the query condition can be divided into: theta connection condition, the equivalent connection condition, and the single Table filter condition; theta join condition refers to performing theta join operation on the fields in the two tables.
  • the equivalence join condition refers to performing equivalence join operation on the fields in the two tables, and the single table filter condition refers to filtering according to a field in a certain table. Condition filter data.
  • the primary key of the customer table is c_custkey
  • the primary key of thenation table is n_nationkey
  • the primary key of the region table is r_regionkey.
  • the optimizer 11 receives the above query statement, and the query conditions obtained after parsing include:
  • connection conditions c_nationkey ⁇ n_nationkey, c_custkey ⁇ o_custkey, and n_regionkey ⁇ r_regionkey;
  • the table names in the query include: customer, nation, orders, and region; where:
  • the fields involved in the customer table include: c_custkey, c_name, and c_nationkey; the fields involved in thenation table include: n_nationkey, n_regionkey, and n_name; the fields involved in the orders table include: o_custkey and o_orderdate; the fields involved in the region table include: r_regionkey and r_name.
  • ⁇ refers to theta connection
  • connection operators that can be included include: >, ⁇ , ⁇ , ⁇ , etc.; in this example, no specific operator is written, and only ⁇ is used instead. .
  • the table corresponding to all the table names obtained in step S201 is respectively identified according to whether it is determined whether to participate in the theta connection operation and whether the field involved in the query statement satisfies the first preset condition.
  • the first preset condition is set as follows: when the field involved in the query statement performs the theta connection operation in the judgment table, whether the calculation process of the Cartesian product causes the data operation amount to exceed the preset threshold. It should be noted that if the calculation of the Cartesian product execution process when the field involved in the query in the query performs the theta connection operation, the data operation amount exceeds the preset threshold, indicating that the data operation amount is drastically expanded. In order to solve the problem of a huge amount of data computation, it is necessary to classify the table as a table that needs to be decomposed.
  • the first preset condition may be: the number of fields involved in the query statement exceeds a first preset threshold; or the storage overhead of the field involved in the query may exceed the first pre-preparation
  • the spatial threshold is set; the number of fields involved in the join operation of the table in the query statement may exceed a second preset threshold; or the table involves participation in the talian in the query statement.
  • the storage overhead of the field of the operation exceeds the second preset space threshold.
  • the storage overhead of the field involved in the query in the first preset condition is greater than the preset space threshold, and the three fields c_custkey, c_name, and c_nationkey in the customer table, and nation are set.
  • the storage overhead of the n_nationkey, n_regionkey, and n_name fields in the table exceeds the preset space threshold.
  • the storage overhead of the fields involved in the query table and the region table in the query statement does not exceed the preset space threshold.
  • step S201 the query condition obtained in step S201 is known:
  • the customer table and the orders table have a connection operation based on the field c_custkey ⁇ o_custkey, and the storage overheads of the c_custkey, c_name, and c_nationkey fields in the customer table exceed the preset space threshold, and therefore, the customer table is the first type table;
  • the nation table and the region table have a connection operation based on the field n_regionkey ⁇ r_regionkey, and the storage overheads of the n_nationkey, n_regionkey, and n_name fields in the nation table exceed the preset space threshold. Therefore, the nation table is the first class table.
  • the orders table and the region table are the second type of tables.
  • step S203 is performed on the first type table identified in this step; and step S206 is performed on the second type table identified in this step.
  • the field participating in the theta connection operation in the table constitutes a first type of field group, and the field involved in the query statement but not participating in the theta connection operation field constitutes the second type field. group;
  • the field involved in the query statement participates in the theta connection operation, and the field involved in the query statement satisfies the first preset condition, indicating that the field of the table is
  • the fields involved in the query statement of the first class table are decomposed, and the number of each table in the first class table is obtained.
  • the field group implements the node connection operation step by step in the form of multiple field groups, which can reduce the amount of data calculated by Cartesian product in one connection operation, and greatly reduce network transmission overhead, calculation overhead and memory overhead. Improve the efficiency of execution.
  • the fields of the connection operation constitute the first type of field group
  • the fields in the table that are involved in the query statement but are not involved in the theta connection operation constitute a second type of field group, and may also have other decomposition methods, as long as the The number of fields that actually participate in the theta connection operation can be low.
  • the fields participating in the theta connection operation and the partial fields in the fields not involved in the theta connection operation but involved in the query statement constitute a field group, and the remaining fields constitute a field group.
  • the first type of field group in this step may be referred to as a first sub-class field group
  • the second type field group may also be referred to as a second sub-class field group.
  • the first type of field group may be: all the fields participating in the theta connection operation in the table constitute a field group; the second type of field group may be understood as involving the table in the query statement, But a field group consisting of all the fields that are not involved in the theta connection operation.
  • the manner of forming the first type of field group may also be: a plurality of field groups formed by the fields participating in the theta connection operation in the table.
  • the fields participating in the theta connection operation in the first type of table are decomposed according to a predetermined rule to obtain a plurality of first type field groups; wherein, the predetermined rule is: the field connected to the same table by theta is decomposed into a field group, And the common field between any two different field groups satisfies the third preset condition.
  • the predetermined rule is: the field connected to the same table by theta is decomposed into a field group, And the common field between any two different field groups satisfies the third preset condition.
  • the first type of field group formed by decomposing the fields in the customer table includes: (c_custkey, c_nationkey), and the second type of field group formed by decomposing the fields in the customer table includes: (c_name).
  • the first type of field group formed by decomposing the fields in thenation table includes: (n_regionkey, n_nationkey), and the second type of field group formed by decomposing the fields in thenation table includes: (n_name).
  • steps S205 and S207 If the first type of field group or the second type of field group does not include the primary key, then steps S205 and S207;
  • step S207 is performed;
  • the primary key is a non-empty field. If the first type of field group or the second type of field group does not include a primary key, a primary key is added, and the purpose is to perform the theta connection operation on the first type field group and the second type field group. After the result, you can use the primary key to perform the equivalent connection to get the complete query result. Moreover, the primary key of the table to which the field group belongs is added to the field group that does not include the primary key in the first type field group and the second type field group.
  • steps S203-S205 are to implement a field and a primary key participating in the node connection operation in each table in the first class table to form a first class field group; each of the first class table is in the query statement.
  • the fields and primary keys involved in but not participating in the theta connection operation constitute the second type of field
  • the first type of field group (c_custkey, c_nationkey) and (n_regionkey, n_nationkey) both contain a primary key
  • the second type of field group (c_name) and (n_name) do not contain a primary key
  • the fields involved in the query in the table form a field group, which may be referred to as a third type field group (also referred to as a second field group);
  • the orders table and the region table are tables of the obtained table except the first type of table
  • the third type of field group formed by the fields involved in the query statement in the orders table is: (o_custkey, o_orderdate);
  • the third type of field group formed by the field of the region table in the query statement is: (r_regionkey, r_name).
  • S207 Generate an execution plan according to the query statement, the first type field group, the second type field group, and the third type field group.
  • the execution plan includes a first type field group to be constructed, a second type field group and a third type field group, an execution step and an execution condition, and the generated execution plan is used to control reading the first type field group and the second class.
  • the execution conditions include theta connection execution condition and the equivalence connection execution condition, and the equivalence connection execution condition can be further divided into a native equivalence connection execution condition and a derived equivalence connection execution condition; wherein, theta connection execution condition The connection condition is the same as the theta in the query statement; the execution condition of the native equivalence join is the same as the equivalence join condition in the query statement; the derivation equivalence join execution condition generally takes the primary key of the first type of table as the equivalence join parameter, for the first The first type of field group and the second type of field group decomposed by the class table perform an equivalence connection by performing an intermediate result generated by the query condition.
  • an implementation manner of generating an execution plan according to the query statement, the first type field group, the second type field group, and the third type field group may include:
  • the first type field group, the second type field group, and the third type field group are used as a field group to be constructed included in the execution plan;
  • the primary key of the first type of table is connected as an equal value.
  • the execution order of the execution condition is taken as the execution step.
  • a control method for performing a multi-table connection operation disclosed in another embodiment of the present invention includes steps S301-S307, wherein the contents of steps S301-S307 can be referred to the embodiment corresponding to FIG. Steps S201-S207 are not described here.
  • the method further includes:
  • the second preset condition and the first preset condition are set in the same principle, and are also used to determine whether the field of the first type of field group performs the theta connection operation, whether the calculation process of the Cartesian product leads to the data operation amount exceeding
  • the second preset condition may also be set from the number of fields and the field storage overhead, for example, the number of fields in the first type of field group exceeds a third preset threshold, or the first type of field
  • the storage overhead of the fields in the group exceeds the third preset space threshold.
  • the third preset threshold corresponding to the number of fields in the second preset condition and the third preset space threshold corresponding to the field storage overhead may be the same as the first preset condition, or may be smaller than the first preset condition. The value in .
  • step S307 is performed
  • step S307 can be directly performed.
  • step S309 is performed, and the first type field group that satisfies the second preset condition is decomposed to obtain multiple Field group
  • Steps reduce the number of fields involved in the theta connection operation, reducing the amount of data operations in the Cartesian product execution process.
  • step S309 there may be multiple ways to decompose the fields in the first type of field group, for example, the equalization mode, that is, the fields in the first type of field group are equally divided to form a plurality of field groups; or The method of forming a field group by a field with a large storage overhead and a field with a small storage overhead; even a random decomposition or the like can be adopted.
  • the equalization mode that is, the fields in the first type of field group are equally divided to form a plurality of field groups
  • the decomposition of the fields in the first type of field group can be understood as a single decomposition, that is, only one decomposition, forming two field groups; it can also be understood as multiple decomposition, and the number of times of multiple decompositions can be set as a deadline.
  • steps S308 and S309 are performed on the decomposed field group until the field in the decomposed field group does not satisfy the second preset condition.
  • step S309 if multiple fields in the first type field group participate in the same theta connection operation, such fields are not decomposed, for example: theta connection operation t1.a1+ T1.a2>t2.b, where the a1 field and the a2 field of the t1 table participate in the same theta connection operation—t1.a1+t1.a2>t2.b, at this time, the a1 field and the a2 field of the t1 table are not Do decomposition.
  • step S311 is performed, and the primary key is added in the field group not including the primary key;
  • the primary key of the table to which the field group belongs needs to be added to the field group.
  • the purpose of adding the primary key to the field group that does not include the primary key is the same as that of step S205 in the above embodiment, and details are not described herein again.
  • step S307 is performed.
  • step S307 is specifically: according to the query statement, the field group after the first type field group decomposition, the second type field group, and the third type field group, Generate an execution plan.
  • a control method for performing a multi-table connection operation disclosed in another embodiment of the present invention includes steps S401-S407, wherein the content of steps S401-S407 can be referred to the embodiment corresponding to FIG. Steps S201-S207 are not described here.
  • the method further includes:
  • step S407 If it is determined that the field in the first type of table field group is not used to participate in the theta connection of the plurality of tables, step S407 is performed;
  • step S407 can be performed.
  • step S409 is performed;
  • the first type of field group of a table in the first type of table is used to participate in the theta connection of multiple tables, it indicates that the fields in the first type of field group can be further decomposed, and then participate in the theta. The number of fields when connecting operations.
  • the predetermined rule for decomposing the first type of field group is that the field connected to the same table is decomposed into a field group, and the common field between any two different field groups satisfies the third preset condition.
  • the third preset condition is used to determine the association between the two field groups formed by the decomposition. If the correlation between the two is high, the two field groups should not be decomposed and should be merged into one field group, if both The relevance is low, indicating that the two field groups have less influence on each other, and theta connection can be performed for two independent field groups.
  • the common field of the two field groups is used to judge the association between the two field groups.
  • the third preset condition may be that the ratio of the number of the common fields to the number of all the fields in the two field groups is lower than the specified value, and the ratio of the space cost of the shared field to the space cost of all the fields of the two field groups may be Below the specified value. It should also be noted that duplicate fields are not included in all fields of the two field groups.
  • the a1 field, the a2 field, and the a3 field of the t1 table and the field in the t2 table have theta connection
  • the a1 field, the a2 field, the a4 field, and the a5 field of the t1 table have theta connection with the field in the t3 table.
  • the a1 field, the a2 field, the a3 field, the a4 field, and the a5 field in the t1 table constitute a first type of field group, and the a1 field and the a2 field are decomposed when the common field a1 field and the a2 field satisfy the third preset condition.
  • the first type of field group (c_custkey, c_nationkey)
  • c_custkey is used for theta connection with the o_custkey field in the orders table
  • c_nationkey is used for theta connection with the n_nationkey field in thenation table
  • the first type of field group ( C_custkey, c_nationkey) satisfies step S408, needs to be decomposed, and the decomposed field groups are: (c_custkey) and (c_nationkey).
  • step S411 is performed, and the primary key is added to the field group;
  • the primary key of the table to which the field group belongs needs to be added to the field group.
  • the purpose of adding the primary key to the field group that does not include the primary key is the same as that of step S205 in the above embodiment, and details are not described herein again.
  • the field group (c_nationkey) obtained by decomposing the first type field group (c_custkey, c_nationkey) in step S409 has no primary key, and then adds a primary key to form a field group (c_nationkey, c_custkey).
  • step S407 is performed.
  • step S407 generates an execution plan according to the query statement, the first type field group, the second type field group, and the third type field group, specifically: An execution plan is generated according to the query statement, the field group after the first type of field group decomposition, the second type field group, and the third type field group.
  • step S308 to step S311 in FIG. 3 may be further performed, that is, determining whether to participate in the theta connection of multiple tables to determine whether to decompose the first Before a type of field group, it may first determine whether the first type of field group is decomposed by determining whether the field satisfies the second preset condition.
  • step S308 to step S311 in FIG. 3 may be performed, that is, whether the first field is determined by determining whether the field satisfies the second preset condition, and then determining whether to participate in multiple The way the table is connected to determine whether to decompose the first type of field group.
  • Another embodiment of the present invention further discloses a control method for performing multi-table connection.
  • the application is applied to a server cluster composed of n servers, where n is greater than or equal to 1.
  • the control method disclosed in this embodiment include:
  • the execution plan includes a first field group and a second field group to be constructed, an execution step, and an execution condition; wherein the execution condition includes: theta connection execution condition and the equivalent connection execution condition, and the execution step and the execution condition requirement, Please refer to the content of step S207 in the embodiment corresponding to FIG. 2, and details are not described herein again.
  • the first field group in the execution plan may be: the first type field group and the second type field group in step S203 in the method embodiment corresponding to FIG. 2; or may be in the manner corresponding to FIG.
  • the plurality of field groups obtained by the step S309 in the embodiment of FIG. 4 may also be a plurality of field groups obtained by the step S409 in the embodiment of FIG.
  • the second field group in the execution plan refers to: the third type field group in step S206 corresponding to the method embodiment in FIG. 2 described above.
  • the data corresponding to the fields in the first field group and the second field group to be constructed may be stored in n storage nodes in the n servers, and received by the distributed computing platform composed of the executors in the n servers 102. Execution plan, reading data corresponding to the fields in the field group to be constructed from the distributed storage system composed of n storage nodes.
  • the field group to be constructed is a field group obtained by the optimizer 11. After the data corresponding to the fields in the field group is read, the read data is combined according to the field group display manner to form the field group data.
  • the field group data is composed: r_regionkey, r_name.
  • a Cartesian product is generated for cp1:c_custkey and the field group: o_custkey, o_orderdate, and data r1 satisfying the connection condition of c_custkey ⁇ o_custkey is obtained;
  • the field group cp2:c_custkey,c_nationkey And np2:n_nationkey, n_regionkey do Cartesian product, get the data r2 satisfying c_nationkey ⁇ n_nationkey;
  • n_regionkey ⁇ r_regionkey perform Cartesian product on the field group np2:n_nationkey, n_regionkey and field group: r_regionkey, r_name, and get the connection condition satisfying n_regionkey ⁇ r_regionkey The data r3.
  • the intermediate result obtained in step S504 is also a field group with data.
  • the primary key field shared by the field group is an equivalent connection parameter.
  • the equivalence connection is performed on r1 and cp3:c_custkey,c_name to obtain r4; the connection condition of the primary key n_nationkey is equivalently connected, and r2 Perform equivalence with r3 to get r5. Finally, perform the equivalence connection on r4 and r5 with the connection condition of the primary key c_custkey as the equivalence connection, and get r6.
  • the equivalence connection execution condition may include a native equivalence join execution condition and a derived equivalence join execution condition; the native equivalence join execution condition refers to an equivalent between fields included in the query condition of the query statement.
  • the execution condition formed by the connection, the derived equivalence connection execution condition refers to the execution condition of the equivalence connection based on the common primary key field in order to obtain the query result.
  • this step may be: first to the field group data in the field group data that does not perform theta connection, and the field group related to the native equivalence connection Data, the equivalence connection is performed according to the original equivalence connection execution condition; the field group data of the original equivalence connection, the field group data of the field group data without theta connection, and the field group not involving the native equivalence connection are performed.
  • the data, as well as the intermediate results, are equivalently joined based on the common primary key field to obtain the query result.
  • this step is: performing the equivalence connection based on the common primary key field on the field group data in the intermediate result and the field group data without theta connection, and obtaining the query result. .
  • the obtained r6 is mapped into an output field sequence.
  • the output field sequence is obtained, it is sent to the data block query engine 101.
  • the form of the first field group and the second field group and the execution condition to be constructed in step S501, and the manner of processing the field group data disclosed in steps S504 and S505 are only for filtering the field group data.
  • step S502 the method further includes:
  • the execution condition includes a single table filter condition
  • filtering is performed on the read data according to the single table filter condition
  • step S503 the read data is grouped according to the field group to be constructed to form field group data, including:
  • the filtered data is grouped according to the field group to be constructed to form field group data;
  • step S503 is performed.
  • the execution condition in the generated execution plan includes a single table filter condition, and therefore, the condition is required as the filter bar. And deleting the data that is not satisfied in the filter condition in the data read in step S502.
  • step S503 the method further includes:
  • the execution condition includes a native equivalence connection execution condition
  • the equivalence connection is performed on the field group data according to the original equivalence connection execution condition
  • step S504 is performed according to the execution step. Performing the field group data after the equivalence connection to connect, and obtaining an intermediate result that meets the execution condition;
  • step S504 is directly executed.
  • step S505 can be understood as: performing, according to the execution step, the field group data that does not perform theta connection in the intermediate result and the field group data, and the field group data that does not involve the native equivalence connection.
  • the query result is obtained based on the equivalence connection of the common primary key field.
  • the above query statement includes two ">" Theta connection conditions and a "c_custkey ⁇ n" single table filter condition. As shown in the following table, when the single table filter threshold n is set to 5, 10, 15, the execution of the present invention is employed.
  • the multi-table connection control method can return the query result in 6-7 seconds, and the method disclosed in the present invention requires 147-524 seconds to return the query result, and the maximum acceleration ratio of the method of the present invention can reach 87.3 times.
  • Another embodiment of the present invention further discloses an optimizer. Referring to FIG. 5, the method includes:
  • a memory 701 configured to store a program and data generated during program running
  • the processor 702 is configured to implement the following functions by running a program in the memory 701:
  • Parsing the query statement obtaining the query condition, the table name, and the field corresponding to the table corresponding to the table name in the query statement; decomposing the fields involved in the query statement in each table in the first class table to obtain the first class table a plurality of first field groups of each table in the table; a method of forming a second field group by using a field involved in each query table in the second type of table, obtaining each of the second type of table a second field group of the table; generating an execution plan according to the query statement, the first field group, and the second field group;
  • the query condition indicates that the table participating in the node connection operation and the field involved in the query statement satisfies the first preset condition is a first type table, and the first preset condition is used to indicate that the table participates in the field involved in the query statement.
  • the Cartesian product calculation causes the data operation amount to exceed the preset threshold;
  • the second type table is a table other than the first type table in the table corresponding to the table name;
  • the execution plan is used to control the reading of the first field.
  • the set first preset condition may include:
  • the number of fields involved in the query statement exceeds a first preset threshold
  • the storage overhead of the field involved in the query statement exceeds the first preset space threshold
  • the number of fields involved in the join operation of the table in the query statement exceeds a second preset threshold
  • the storage overhead of the field involved in the join operation of the table in the query statement exceeds the second preset space threshold.
  • the processor 702 performs decomposing the fields involved in each query table in the first class table to obtain each table in the first class table.
  • the first field group it is specifically used to:
  • the fields and primary keys participating in the theta connection operation in the table respectively constitute the first sub-class field group, and the fields and primary keys in the table that are involved in the query statement but are not involved in the theta connection operation.
  • the processor 702 executes the execution plan according to the query statement, the first field group, and the second field group, and is specifically used to:
  • Generating an execution plan according to the query statement, the first sub-class field group, the second sub-class field group, and the second field group, and the execution plan is used to control reading the first sub-class field group, the second sub-class field group, and the second The data corresponding to the field in the field group, and performing the connection operation satisfying the execution condition on the read data according to the execution step, and obtaining the query result that meets the requirements of the query statement.
  • processor 702 when the processor 702 performs the operation of the field and the primary key participating in the theta connection operation in the table to form the first sub-class field group operation, specifically:
  • processor 702 can refer to the content in the method example corresponding to FIG. 2 in the specific working process of the foregoing embodiment, and details are not described herein again.
  • the processor 702 is further configured to: before generating an execution plan according to the query statement, the first sub-class field group, the second sub-class field group, and the second field group:
  • the first sub-class field group is decomposed, and multiple field groups are obtained; determining that the second preset condition is satisfied Whether the plurality of field groups obtained by decomposing the first sub-class field group respectively include a primary key; if the plurality of field groups obtained by decomposing the first sub-class field group satisfying the second preset condition do not include the primary key, Add a primary key in a field group that does not contain a primary key;
  • the processor 702 is further configured to: after adding the primary key in the field group not including the primary key, and including the primary key in the plurality of field groups obtained by decomposing the first sub-category field group that satisfies the second preset condition Thereafter, the step of generating an execution plan according to the query statement, the first subclass field group, the second subclass field group, and the second field group is performed.
  • the foregoing second preset condition includes:
  • the number of fields in the first subclass field group exceeds a third preset threshold
  • the storage overhead of the fields in the first subclass field group exceeds the third preset space threshold.
  • the processor 702 is further configured to:
  • the first sub-class field group is decomposed according to a predetermined rule to obtain a plurality of field groups; Whether the plurality of field groups obtained by the decomposition include a primary key; if the plurality of field groups obtained by the decomposition do not all include the primary key, the primary key is added to the field group not including the primary key;
  • the foregoing preset rule includes: decomposing a field in the first sub-category field group that is connected to the same table into the field group, and decomposing the common field between any two field groups to satisfy the third preset. Condition, the third preset condition is used to determine the association between the two field groups formed by the decomposition;
  • the processor 702 is further configured to: after adding the primary key in the field group not including the primary key, and after determining that the plurality of field groups obtained by the decomposition include the primary key, executing the query according to the query, the first sub-class field group, and the second A sub-field group and a second field group, which generate the steps of the execution plan.
  • the present invention also discloses a server cluster.
  • n is greater than or equal to 1
  • each server includes a processor 801 and a memory 802; wherein:
  • the processor 801 of the n servers constitutes a distributed computing platform, configured to receive an execution plan, the execution plan includes a first field group and a second field group to be constructed, an execution step and an execution condition; and reading the first field group to be constructed And data corresponding to the fields in the second field group; grouping the read data according to the first field group and the second field group to be constructed to form field group data; performing the connection that satisfies the execution condition on the field group data according to the execution step Operation, obtaining the query result; mapping the query result to the output field sequence; sending the output field sequence;
  • the memory 202 of the n servers constitutes a distributed storage system for storing data corresponding to the fields.
  • the processor 201 can be understood as an executor in each server, and the memory 202 can be understood as a storage node in each server.
  • the execution condition includes: theta connection execution condition and the equivalent connection execution condition; and the distributed computing platform performs to satisfy the field group data according to the execution step.
  • the connection operation of the condition is executed, when the query result is obtained, it is specifically used to:
  • theta connection is performed on the field group data, and an intermediate result conforming to the execution condition of the theta connection is obtained; according to the execution step, the intermediate result and the field group data of the field group data not connected with theta are connected in an equal value, and the equivalent value is obtained.
  • the result of the query that connects the execution conditions is obtained.
  • the distributed computing platform composed of the processor 801 can be referred to the content of the method example corresponding to FIG. 5 in the specific working process of the foregoing embodiment, and details are not described herein again.
  • the distributed computing platform after the distributed computing platform reads the data of the fields involved in the first field group and the second field group to be constructed, the distributed computing platform is further configured to:
  • Determining whether the execution condition includes a single table filter condition if the execution condition includes a single table filter condition, performing filtering on the read data according to the single table filter condition;
  • the distributed computing platform performs the operations of grouping the read data according to the first field group and the second field group to be constructed, and is used for:
  • the filtered data is grouped according to the first field group and the second field group to be constructed to form field group data.
  • the distributed computing platform groups the read data according to the field group to be constructed, and after forming the field group data, the distributed computing platform is further configured to:
  • the execution condition includes a native equivalence connection execution condition, and performing an equivalence connection on the field group data according to the execution step according to the execution condition of the native equivalent connection;
  • the distributed computing platform performs the connection of the field group data according to the execution step, and obtains the query result that meets the execution condition, and is specifically used to: perform the connection of the field group data after performing the equivalence connection according to the execution step, and obtain the compliance execution condition. Query results.
  • the distributed computing platform composed of the processor 801 can refer to the content of the corresponding method example in the specific working process of the foregoing embodiment, and details are not described herein again.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Operations Research (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种执行多表连接操作的控制方法及对应装置，其中，将参与theta连接操作、且在所述查询语句中涉及的字段满足第一预设条件的表、在所述查询语句中所涉及的字段进行分解，得到多个第一字段组，可以实现以多个字段组的形式分步完成theta连接操作，能够实现降低了一次进行连接操作时笛卡尔积计算的数据量，大幅度减少的网络传输开销、计算开销和内存开销，提升执行效率的效果。

Description

执行多表连接操作的控制方法及对应装置 技术领域
本发明涉及计算机技术领域，更具体地说，涉及一种数据库多表连接查询技术。
背景技术
关系型数据库，是创建在关系模型基础上的数据库，关系模型就是指二维表格模型,用于将数据组织为多张由行和列组成的表。存储关系型数据库中的表时，因计算机内存采用一维结构的顺序编址，所以可以采用逐条输入表中各行数据的方式(即行存)，或者批量导入表中各列数据的方式(即列存)。
在关系型数据库中的表存储后，可基于关系代数对表执行多种连接操作，例如：对两张表执行theta连接，即合并两张表中满足theta条件的字段。
目前，在基于分布式计算框架的关系型数据库系统中，存储系统和数据库查询引擎通常相互独立，所述数据库查询引擎无法感知存储系统中数据的组织方式。为此，依据查询语句对存储系统存储的表执行theta连接操作时，通常需要首先读取表中在查询语句中涉及的所有字段并计算笛卡尔积，然后根据查询条件过滤出连接结果。当执行theta连接操作的表较大时，对表中在查询语句中涉及的所有字段进行笛卡儿积的计算会导致数据运算量剧烈膨胀，从而大幅增加计算开销、内存开销和节点间的网络传输开销，严重降低性能。
发明内容
有鉴于此，本发明提供一种执行多表连接操作的控制方法及对应装置，以解决现有技术中进行多表theta连接操作时，由于对表中在查询语句中涉及的所有字段计算笛卡尔积导致的数据运算量剧烈膨胀，计算开销、内存开销和节点间的网络传输开销大，性能低的问题。
为了实现上述目的，现提出的方案如下：
本发明的第一方面提供了一种执行多表连接操作的控制方法，包括以下步骤：
解析查询语句，获得查询条件、表名以及所述表名对应的表在所述查询语句中涉及的字段。
将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组；其中，所述查询条件表明参与theta连接操作、且在所述查询语句中涉及的字段满足第一预设条件的表为第一类表，所述第一预设条件用于表明表在所述查询语句中涉及的字段执行theta连接操作时，进行笛卡儿积计算导致数据运算量超于预设阈值。
采用将第二类表中的每张表在所述查询语句中涉及的字段，构成一个第二字段组的方式，获得所述第二类表中的每张表的第二字段组；其中，所述第二类表为所述表名对应的表中除所述第一类表以外的表。
依据所述查询语句、所述第一字段组和所述第二字段组，生成执行计划，所述执行计划用于控制读取所述第一字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
从上述过程可以看出：将参与theta连接操作、且在所述查询语句中涉及的字段满足第一预设条件的表、在所述查询语句中所涉及的字段进行分解，得到多个第一字段组，以实现以多个字段组的形式分步完成theta连接操作，能够实现降低了一次进行连接操作时笛卡尔积计算的数据量，大幅度减少的网络传输开销、计算开销和内存开销，提升执行效率的效果。
本发明的第二方面公开了一种优化器，包括：存储器，用于存储程序以及程序运行中产生的数据；处理器，用于通过运行所述存储器中的程序，实现以下功能：
解析查询语句，获得查询条件、表名以及所述表名对应的表在所述查询语句中涉及的字段；将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组；采用将第二类表中的每张表在所述查询语句中涉及的字段，构成一个第二字段组的方式，获得所述第二类表中的每张表的第二字段组；依据所述查询语句、所述第一字段组和所述第二字段组，生成执行计划。
其中，所述查询条件表明参与theta连接操作、且在所述查询语句中涉及的字段满足第一预设条件的表为第一类表，所述第一预设条件用于表明表在所述查询语句中涉及的字段执行theta连接操作时，进行笛卡儿积计算导致数据运算量超于预设阈值；所述第二类表为所述表名对应的表中除所述第一类表以外的表；所述执行计划用于控制读取所述第一字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
在一个实现方式中，所述第一预设条件包括：
表在所述查询语句中涉及的字段的数量超过第一预设阈值。
表在所述查询语句中涉及的字段的存储开销超过第一预设空间阈值。
表在所述查询语句中涉及参与theta连接操作的字段的数量超过第二预设阈值。或者，
表在所述查询语句中涉及参与theta连接操作的字段的存储开销超过第二预设空间阈值。
在一个实现方式中，所述将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组，包括：
对于所述第一类表中的每张表，分别将表中参与theta连接操作的字段和主键构成第一子类字段组，将表中在所述查询语句中涉及、但未参与theta连接操作的字段和主键构成第二子类字段组；
其中，所述依据所述查询语句所述第一字段组和所述第二字段组，生成执行计划，包括：
依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划，所述执行计划用于控制读取所述第一子类字段组、所述第二子类字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
在一个实现方式中，所述将表中参与theta连接操作的字段和主键构成第一子类字段组，包括：
将表中参与theta连接操作的所有字段和主键构成一个第一子类字段组。
或者；将表中与同一个表的字段进行theta连接操作的字段和主键构成一 个第一子类字段组。
在一个实现方式中，所述依据查询语句、第一类字段组、第二类字段组和第三类字段组，生成执行计划，包括：
将第一类字段组、第二类字段组和第三类字段组作为执行计划中包含的待构建的字段组。
依据查询语句中的查询条件，生成执行条件中的theta连接执行条件和原生等值连接执行条件。
为满足查询语句对应的查询结果的要求，将以第一类表的主键为等值连接参数，对第一类字段组和第二类字段组执行查询条件生成的中间结果执行等值连接的条件，作为执行条件中的派生等值连接执行条件。
为满足查询语句对应的查询结果的要求，将执行条件的执行顺序作为执行步骤。
在一个实现方式中，构成所述第一子类字段组和所述第二类子字段组之后，还包括：
若所述第一类表中的某张表的第一子类字段组中的字段用于参与多个表的theta连接操作，按照预定规则分解所述第一子类字段组，得到多个字段组，其中，所述预设规则包括：所述第一类字段组中与同一个表进行theta连接的字段分解为一个字段组，且分解成的任意两个字段组之间的共有字段满足第三预设条件，第三预设条件用于判断分解构成的两个字段组之间的关联性。
判断所述分解得到的多个字段组是否均包含主键。
若所述分解得到的多个字段组未均包含主键，则在未包含主键的字段组中添加主键，并返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
若所述分解得到的多个字段组均包含主键，则返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
从上述过程可以看出：若第一类表中的某张表的第一子类字段组用于参与多个表的theta连接，说明该第一子类字段组中的字段还可以进一步分解，再降低参与theta连接操作时的字段数量。
在一个实现方式中，所述依据所述查询语句、所述第一子类字段组、所述 第二子类字段组和所述第二字段组，生成执行计划之前，还包括：
若所述第一类表中的某张表的第一子类字段组中字段满足第二预设条件，分解所述第一子类字段组，得到多个字段组。
判断对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组、是否均包含主键。
若所述对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组、未均包含主键，则在未包含主键的字段组中添加主键，并返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
若所述对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组均包含主键，则返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
从上述过程可以看出：判断出第一子类字段组中的字段满足第二预设条件，说明第一子类字段组中的字段进行theta连接操作时的笛卡尔积执行过程，还会存在数据运算量剧烈膨胀的问题，因此将第一子类字段组中的字段分解，得到多个字段组，进一步降低参与theta连接操作时的字段数量，降低笛卡尔积执行过程的数据运算量。
在一个实现方式中，所述第二预设条件包括：
所述第一子类字段组中字段的数量超过第三预设阈值。
所述第一子类字段组中字段的存储开销超过第三预设空间阈值。
本发明的第三方面提供了一种执行多表连接操作的控制方法，包括以下步骤：
接收执行计划，所述执行计划包含待构建的第一字段组和第二字段组、执行步骤和执行条件。
读取所述待构建的第一字段组和第二字段组中字段对应的数据。
依据所述待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据。
按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果。
将所述查询结果映射为输出字段序列。
发送所述输出字段序列。
从上述过程可以看出：依据执行计划中的待构建的第一字段组和第二字段组，对读取的数据进行分组，形成字段组数据，且对字段组数据进行连接操作，实现了以多个字段组的形式分步完成theta连接操作，能够实现降低了一次进行连接操作时笛卡尔积计算的数据量，大幅度减少的网络传输开销、计算开销和内存开销，提升执行效率的效果。
本发明的第二方面公开了一种服务器集群，包括n个服务器，n大于或等于1，每个所述服务器均包括处理器和存储器；其中：
所述n个服务器的处理器组成分布式计算平台，用于接收执行计划，所述执行计划包含待构建的第一字段组和第二字段组、执行步骤和执行条件；读取所述待构建的第一字段组和第二字段组中字段对应的数据；依据所述待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据；按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果；将所述查询结果映射为输出字段序列；发送所述输出字段序列。
所述n个服务器的存储器组成分布式存储系统，用于存储字段对应的数据。
在一个实现方式中，所述读取所述待构建的第一字段组和第二字段组中涉及的字段的数据之后，还包括：
判断所述执行条件中是否包含单表过滤条件。
若所述执行条件中包含单表过滤条件，则依据所述单表过滤条件，对读取的数据执行过滤。
其中，所述依据所述待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据，包括：
依据所述待构建的第一字段组和第二字段组对过滤后的数据进行分组，形成字段组数据。
在一个实现方式中，所述执行条件包括：theta连接执行条件和等值连接执行条件；所述按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果，包括：
按照所述执行步骤对所述字段组数据进行theta连接，得到符合所述theta连接执行条件的中间结果。
按照所述执行步骤、对所述中间结果和所述字段组数据中未进行theta连接的字段组数据进行等值连接，得到符合所述等值连接执行条件的查询结果。
在一个实现方式中，所述依据所述待构建的字段组对读取的数据进行分组，形成字段组数据之后，还包括：
判断所述执行条件包含原生等值连接执行条件，按照执行步骤，依据所述原生等值连接执行条件对所述字段组数据执行等值连接。
其中，所述按照所述执行步骤对所述字段组数据进行连接，得到符合所述执行条件的查询结果，包括：
按照执行步骤对执行等值连接后的字段组数据进行连接，得到符合所述执行条件的查询结果。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为分布式计算框架的数据库系统的结构示意图；
图2为本发明实施例公开的执行多表连接操作的控制方法的流程图；
图3为本发明另一实施例公开的执行多表连接操作的控制方法的流程图；
图4为本发明另一实施例公开的执行多表连接操作的控制方法的流程图；
图5为本发明另一实施例公开的执行多表连接的控制方法的流程图；
图6为本发明另一实施例公开的对字段组执行执行条件的展示图；
图7为本发明另一实施例公开的优化器的结构示意图；
图8为本发明另一实施例公开的服务器集群的结构示意图。
具体实施方式
图1为本发明实施例提供的一种分布式计算框架的数据库系统，包括数据 库查询引擎101和n个服务器102，其中，数据库查询引擎101包含优化器11，每个服务器均包含执行器12和存储节点13，n个服务器102中的执行器组成分布式计算平台，n个服务器102中的存储节点组成分布式存储系统。
具体的，数据库查询引擎101接收用户输入的查询语句，优化器11解析查询语句，生成执行计划；数据库查询引擎101发送执行计划到n个服务器102。n个服务器102中执行器组成的分布式计算平台接收执行计划，依据执行计划，读取n个服务器102中的存储节点组成分布式存储系统中存储的字段数据，处理读取的字段数据并生成查询结果，并将查询结果映射为输出字段序列，向数据库查询引擎101发送所述输出字段序列。
本发明提供的执行多表连接操作的控制方法，其目的是解决现有技术中进行多表theta连接操作时，由于对表中在查询语句中涉及的所有字段计算笛卡尔积导致的数据运算量剧烈膨胀，计算开销、内存开销和节点间的网络传输开销大，性能低的问题。
参见图1和图2，本发明实施例公开的一种执行多表连接操作的控制方法，应用于优化器11，所述控制方法包括：
S201、解析查询语句，获得查询条件、表名以及表名对应的表在查询语句中涉及的字段；
需要说明的是，用户想对存储的多个表执行连接操作时，在数据库查询引擎101中输入查询语句，数据库查询引擎101接收到查询语句后，向优化器11转发。优化器11接收到查询语句，解析该查询语句得到查询条件、表名以及表名对应的表在查询语句中涉及的字段；其中，查询条件可以分为：theta连接条件、等值连接条件以及单表过滤条件；theta连接条件指代对两张表中的字段执行theta连接操作，等值连接条件指代对两张表中的字段执行等值连接操作，单表过滤条件指代对某张表中的某字段按照过滤条件筛选数据。
以下以一个实例具体说明，该实例中，用户输入查询语句如下：
Select c_custkey,c_name,c_nationkey,n_nationkey,n_regionkey,n_name,o_custkey,o_orderdate r_regionkey,r_name FROM customer,nation,orders,region where c_nationkeyθn_nationkey and c_custkeyθo_custkey and n_regionkeyθr_regionkey。
其中，customer表主键为c_custkey，nation表主键为n_nationkey，region表主键为r_regionkey。
优化器11接收到上述查询语句，解析后得到的查询条件包括：
theta连接条件：c_nationkeyθn_nationkey、c_custkeyθo_custkey以及n_regionkeyθr_regionkey；
查询语句中的表名包括：customer、nation、orders以及region；其中：
上述四个表名对应的表在查询语句中涉及的字段为：
customer表涉及的字段包括：c_custkey，c_name和c_nationkey；nation表涉及的字段包括：n_nationkey，n_regionkey和n_name；orders表涉及的字段包括：o_custkey和o_orderdate；region表涉及的字段包括：r_regionkey和r_name。
还需要说明的是，上述实例中，θ指代theta连接，可以包括的连接运算符包括：＞、≥、＜、≤等等；此实例中没有写出具体的运算符，仅用θ来替代。但在本发明公开的执行多表连接操作的控制方法实际使用过程中，需要在查询语句中明确写明θ为哪种运算符。
S202、识别查询条件中表明参与theta连接操作、且在查询语句中涉及的字段满足第一预设条件的表为第一类表，否则识别为第二类表；
本步骤中，将步骤S201中获得的所有表名对应的表，分别依据判断是否参与theta连接操作和在查询语句中涉及的字段是否满足第一预设条件这两个条件，来识别是否为第一类表。
其中，第一预设条件的设定原则为：判断表中在查询语句中涉及的字段进行theta连接操作时，笛卡尔积的计算过程，是否会导致数据运算量超于预设阈值。需要说明的是，若表中在查询语句中涉及的字段进行theta连接操作时的笛卡尔积执行过程的计算，会导致数据运算量超于预设阈值，说明数据运算量剧烈膨胀。为了解决数据运算量剧烈膨胀的问题，则需要把表归为需要进行字段分解的表。
还需要说明的是，第一预设条件具体可以为：表在查询语句中涉及的字段的数量超过第一预设阈值；或者可以为表在查询语句中涉及的字段的存储开销超过第一预设空间阈值；还可以为表在所述查询语句中涉及参与theta连接操作的字段的数量超过第二预设阈值；或者，表在查询语句中涉及参与theta连 接操作的字段的存储开销超过第二预设空间阈值。
接上述实例中，以第一预设条件为表在查询语句中涉及的字段的存储开销超过预设空间阈值为例说明，且设定customer表中的c_custkey，c_name和c_nationkey三个字段、以及nation表中的n_nationkey，n_regionkey和n_name三个字段的存储开销均超过预设空间阈值，orders表和region表在查询语句中涉及的字段的存储开销未超过预设空间阈值。
则在本实例中，经步骤S201中得到的查询条件可知：
customer表与orders表存在基于字段c_custkeyθo_custkey的连接操作，且customer表中的c_custkey，c_name和c_nationkey三个字段的存储开销均超过预设空间阈值，因此，customer表为第一类表；
nation表与region表存在基于字段n_regionkeyθr_regionkey的连接操作，且nation表中的n_nationkey，n_regionkey和n_name三个字段的存储开销均超过预设空间阈值，因此，nation表为第一类表。
orders表和region表为第二类表。
还需要说明的是，对本步骤识别的第一类表执行步骤S203；对本步骤识别的第二类表执行步骤S206。
S203、对于第一类表中的每张表，将表中参与theta连接操作的字段构成第一类字段组，将表中在查询语句中涉及、但未参与theta连接操作字段构成第二类字段组；
需要说明的是，由于第一类表中的表，其在查询语句中所涉及的字段参与theta连接操作，且表在查询语句中涉及的字段满足第一预设条件，说明该表的字段在进行theta连接操作时的笛卡尔积执行过程，会存在数据运算量剧烈膨胀的问题，因此，将第一类表在查询语句中涉及的字段分解，得到第一类表中的每张表的多个字段组，实现以多个字段组的形式分步完成theta连接操作，能够实现降低了一次进行连接操作时笛卡尔积计算的数据量，大幅度减少的网络传输开销、计算开销和内存开销，提升执行效率的效果。
分解第一类表中的每张表在查询语句中涉及的字段，得到第一类表中的每张表的对个第一字段组的方式，可以如步骤S203的方式，将表中参与theta连接操作的字段构成第一类字段组，将表中在查询语句中涉及、但未参与theta连接操作的字段构成第二类字段组，还可以有其他的分解方式，只要能满足降 低实际参与执行theta连接操作的字段数量即可。例如：将参与theta连接操作的字段和未参与theta连接操作、但在查询语句中涉及的字段中的部分字段构成字段组，剩余的字段构成字段组。并且，本步骤中的第一类字段组可以称之为第一子类字段组，第二类字段组也可以称之为第二子类字段组。
还需要说明的是，本步骤中，第一类字段组可以为：将表中参与theta连接操作的所有字段构成一个字段组；第二类字段组可以理解成将表中在查询语句中涉及、但未参与theta连接操作的所有字段构成的一个字段组。
可选地，构成的第一类字段组的方式还可以为：将表中参与theta连接操作的字段构成的多个字段组。
具体的，将第一类表中参与theta连接操作的字段，按照预定规则分解得到多个第一类字段组；其中，预定规则为：与同一个表进行theta连接的字段分解为一个字段组，且任意两个不同字段组之间的共有字段满足第三预设条件。并且，具体内容可参见以下对应图4的实施例中步骤S409的内容，此处不再赘述。
接上述实例中，对customer表中字段分解形成的第一类字段组包括：(c_custkey，c_nationkey)，对customer表中字段分解形成的第二类字段组包括：(c_name)。对nation表中字段分解形成的第一类字段组包括：(n_regionkey，n_nationkey)，对nation表中字段分解形成的第二类字段组包括：(n_name)。
S204、判断第一类字段组和第二类字段组是否均包含主键；
若第一类字段组或第二类字段组未包含主键，则执行步骤S205和S207；
若第一类字段组和第二类字段组均包含主键，则执行步骤S207；
S205、对未含主键的字段组中添加主键；
其中，主键为非空字段，若第一类字段组或第二类字段组中未包含主键，需添加主键，其目的是，将第一类字段组和第二类字段组在执行theta连接操作后的结果，可以再通过主键进行等值连接，以得到完整的查询结果。并且，需要对第一类字段组和第二类字段组中未包含主键的字段组，添加该字段组所属的表的主键。
还需要说明的是，步骤S203-S205是实现将第一类表中的每张表中参与theta连接操作的字段和主键构成第一类字段组；将第一类表中的每张表中在所述查询语句中涉及、但未参与theta连接操作的字段和主键构成第二类字段 组的一种实现方式，还可以有其他的实现方式，此处不再说明。
接上述实例，第一类字段组(c_custkey，c_nationkey)和(n_regionkey，n_nationkey)均包含主键，第二类字段组(c_name)和(n_name)均不包含主键，对第二类字段组添加主键后，形成字段组(c_name，c_custkey)和字段组(n_name，n_nationkey)。
S206、对第二类表中的每张表，将表中在查询语句中涉及的字段构成一个字段组，可以称为第三类字段组(也可称之为第二字段组)；
接上述实例中，orders表和region表为获取的表中除第一类表以外的表，orders表中在查询语句中涉及的字段构成的第三类字段组为：(o_custkey，o_orderdate)；并且，region表在查询语句中涉及的字段构成的第三类字段组为：(r_regionkey，r_name)。
S207、依据查询语句、第一类字段组、第二类字段组和第三类字段组，生成执行计划。
其中，执行计划包含待构建的第一类字段组、第二类字段组和第三类字段组、执行步骤和执行条件，生成的执行计划用于控制读取第一类字段组、第二类字段组和第三类字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合查询语句要求的查询结果。
还需要说明的是：执行条件包括theta连接执行条件和等值连接执行条件，且等值连接执行条件又可分为原生等值连接执行条件和派生等值连接执行条件；其中，theta连接执行条件与查询语句中的theta连接条件相同；原生等值连接执行条件与查询语句中的等值连接条件相同；派生等值连接执行条件一般以第一类表的主键为等值连接参数，对第一类表分解出的第一类字段组和第二类字段组执行查询条件生成的中间结果执行等值连接。
因此，步骤S207中，依据查询语句、第一类字段组、第二类字段组和第三类字段组，生成执行计划的一种实现方式可以包括：
将第一类字段组、第二类字段组和第三类字段组作为执行计划中包含的待构建的字段组；
依据查询语句中的查询条件，生成执行条件中的theta连接执行条件和原生等值连接执行条件；
为满足查询语句对应的查询结果的要求，将以第一类表的主键为等值连接 参数，对第一类字段组和第二类字段组执行查询条件生成的中间结果执行等值连接的条件，作为执行条件中的派生等值连接执行条件；
为满足查询语句对应的查询结果的要求，将执行条件的执行顺序作为执行步骤。
可选地，参见图3，在本发明的另一实施例公开的执行多表连接操作的控制方法，包含步骤S301～S307，其中，步骤S301-S307的内容可参见对应图2的实施例中的步骤S201-S207，此处不再赘述。
并且，本实施例中，在步骤S305之后以及步骤S304的判断结果为第一类字段组和第二类字段组均包含主键之后，均还包括：
S308、判断第一类字段组中字段是否满足第二预设条件；
需要说明的是，需要判断第一类表中的每张表的第一类字段组中的字段是否满足第二预设条件。
其中，第二预设条件和第一预设条件的设定原则相同，也用于判断第一类字段组中的字段进行theta连接操作时，笛卡尔积的计算过程是否会导致数据运算量超于预设阈值，因此，第二预设条件也可以是从字段数量和字段存储开销来设定，例如：第一类字段组中的字段的数量超过第三预设阈值，或者第一类字段组中的字段的存储开销超过第三预设空间阈值。当然，设定的第二预设条件中字段数量对应的第三预设阈值和字段存储开销对应的第三预设空间阈值可以和第一预设条件可以相同，也可以小于第一预设条件中的值。
若判断出第一类表字段组中字段不满足第二预设条件，则执行步骤S307；
其中，若第一类表中的每张表的第一类字段组中的字段均不满足第二预设条件，则说明第一类表中的每张表的第一类字段组均不需要进一步分解，可直接执行步骤S307。
若判断出第一类表中的某张表的第一类字段组中的字段满足第二预设条件，则执行步骤S309、分解满足第二预设条件的第一类字段组，得到多个字段组；
其中，判断出第一类字段组中的字段满足第二预设条件，说明第一类字段组中的字段进行theta连接操作时的笛卡尔积执行过程，还会存在数据运算量剧烈膨胀的问题，因此将第一类字段组中的字段分解，得到多个字段组，进一 步降低参与theta连接操作时的字段数量，降低笛卡尔积执行过程的数据运算量。
还需要说明的是，步骤S309中，分解第一类字段组中字段的方式可以有多种，例如：均分方式，即将第一类字段组中的字段等分，形成多个字段组；或者采用由存储开销大的字段和存储开销小的字段构成字段组的方式；甚至还是可以随机分解等等。
并且，在对第一类字段组中的字段进行分解可以理解为一次分解，即仅分解一次，形成两个字段组；还可以理解为多次分解，可以设定多次分解的次数，作为截止分解的条件，或者分解一次后，对分解后的字段组执行步骤S308和S309，直到分解后的字段组中的字段不满足第二预设条件为止。
还需要说明的是，步骤S309中执行过程中，若第一类字段组中有多个字段参与同一个theta连接操作的情况，对这样的字段不做分解，例如：theta连接操作t1.a1+t1.a2>t2.b，其中，t1表的a1字段和a2字段参与同一个theta连接操作—t1.a1+t1.a2>t2.b，此时，对t1表的a1字段和a2字段不做分解。
S310、判断分解得到的字段组是否均包含主键；
若分解得到的字段组中有字段组不包含主键，则执行步骤S311、在不包含主键的字段组中添加主键；
需要说明的是，若分解得到的字段组中有字段组不包含主键，则需要将该字段组所属的表的主键添加到字段组中。并且，在不包含主键的字段组中添加主键的目的，与上述实施例中步骤S205相同，此处不再赘述。
若分解得到的字段组均包含主键，则执行步骤S307。
还需要说明的是，若在步骤S310和S311之后执行步骤S307时，步骤S307具体为：依据查询语句、第一类字段组分解后的字段组、第二类字段组和第三类字段组，生成执行计划。
可选地，参见图4，在本发明的另一实施例公开的执行多表连接操作的控制方法，包含步骤S401～S407，其中，步骤S401-S407的内容可参见对应图2的实施例中的步骤S201-S207，此处不再赘述。
其中，本实施例中，在步骤S405之后以及步骤S404的判断结果为第一类字段组和第二类字段组均包含主键之后，还包括：
S408、判断第一类字段组中的字段是否用于参与多个表的theta连接；
需要说明的是，对分解得到的第一类表中的每张表的第一类字段组，都需要判断字段组中的字段是否用于参与多个表的theta连接。
若判断出第一类表字段组中字段不用于参与多个表的theta连接，则执行步骤S407；
其中，若第一类表中的每张表的第一类字段组均不用于参与多个表的theta连接，则说明第一类字段组的字段进行theta连接操作时的笛卡尔积执行过程，存在数据运算量剧烈膨胀的问题的概率较低，因此可以执行步骤S407。
若判断出第一类表中的某张表的第一类字段组中的字段用于参与多个表的theta连接，则执行步骤S409；
S409、按照预定规则分解第一类字段组，得到多个字段组；
需要说明的是，若第一类表中的某张表的第一类字段组用于参与多个表的theta连接，说明该第一类字段组中的字段还可以进一步分解，再降低参与theta连接操作时的字段数量。
其中，分解第一类字段组的预定规则为，与同一个表进行theta连接的字段分解为一个字段组，且任意两个不同字段组之间的共有字段满足第三预设条件。第三预设条件用于判断分解构成的两个字段组之间的关联性，若两者的关联性较高，则不应该将两个字段组分解，应合并为一个字段组，若两者的关联性较低，说明两者字段组互相的影响性较小，可以为独立的两个字段组执行theta连接。其中，两个字段组的公共字段用于评判两个字段组之间的关联性的，公共字段的数量越少或者占有的空间开销越低，则说明两个字段组的关联性越低，因此所述第三预设条件可以是，共有字段的数量占两个字段组中所有字段数量的比例低于指定值，还可以是，共有字段的空间开销占两个字段组所有字段空间开销的比例低于指定值。还需要说明的是，两个字段组的所有字段中不包含重复的字段。
例如：t1表的a1字段、a2字段和a3字段和t2表中的字段有theta连接，t1表的a1字段、a2字段、a4字段和a5字段与t3表中的字段有theta连接。
t1表中的a1字段、a2字段、a3字段、a4字段和a5字段构成了第一类字段组，在公共字段a1字段和a2字段满足第三预设条件时，才会分解a1字段、a2字段、a3字段、a4字段和a5字段构成的第一类字段组，得到由a1字段、 a2字段和a3字段构成的字段组，以及由a1字段、a2字段、a4字段和a5字段构成字段组。
接上述实例，第一类字段组(c_custkey，c_nationkey)中，c_custkey用于与orders表中o_custkey字段进行theta连接，c_nationkey用于与nation表中n_nationkey字段进行theta连接，因此，第一类字段组(c_custkey，c_nationkey)满足步骤S408，需要对其进行分解，且分解得到的字段组为：(c_custkey)和(c_nationkey)。
S410、判断分解得到的字段组是否均包含主键；
若有字段组不包含主键，则执行步骤S411、在字段组中添加主键；
需要说明的是，若分解得到的字段组中，有字段组不包含主键，则需要将该字段组所属的表的主键添加到字段组中。并且，在不包含主键的字段组中添加主键的目的，与上述实施例中步骤S205相同，此处不再赘述。
接上述实例，步骤S409中分解第一类字段组(c_custkey，c_nationkey)后得到的字段组(c_nationkey)没有主键，则为其添加主键，形成字段组(c_nationkey，c_custkey)。
若分解得到的字段组均包含主键，则执行步骤S407。
还需要说明的是，若在步骤S410和S411之后执行步骤S407时，步骤S407、依据查询语句、第一类字段组、第二类字段组和第三类字段组，生成执行计划，具体为：依据查询语句、第一类字段组分解后的字段组、第二类字段组和第三类字段组，生成执行计划。
可选地，参见图3和图4，执行图4中步骤S408之前，还可以执行图3中步骤S308-步骤S311，即采用判断是否参与多个表的theta连接的方式，来判断是否分解第一类字段组之前，可以先采用判断字段是否满足第二预设条件的方式，来判断是否分解第一类字段组。
或者，执行S407之前，还可以执行图3中步骤S308-步骤S311，即先采用判断字段是否满足第二预设条件的方式，来判断是否分解第一类字段组，再采用判断是否参与多个表的theta连接的方式，来判断是否分解第一类字段组。
其中，步骤S308-步骤S311具体内容可参见对应图3的实施例的内容，此处不再赘述。
本发明另一实施例还公开了一种执行多表连接的控制方法，参见图1和图5，应用于由n个服务器组成的服务器集群，n大于或等于1，本实施例公开的控制方法包括：
S501、接收执行计划；
其中，执行计划包含待构建的第一字段组和第二字段组、执行步骤和执行条件；其中，执行条件包含：theta连接执行条件和等值连接执行条件，且执行步骤和执行条件的要求，请参见对应图2的实施例中步骤S207的内容，此处不再赘述。
还需要说明的是，执行计划中的第一字段组可以是：上述对应图2的方法实施例中步骤S203中的第一类字段组和第二类字段组；也可以是对应图3的方式实施例中步骤S309中分解得到的多个字段组；还可以是对应图4的方式实施例中步骤S409中分解得到的多个字段组。并且，执行计划中的第二字段组，是指：上述对应图2的方法实施例中步骤S206中的第三类字段组。
S502、读取待构建的第一字段组和第二字段组中字段对应的数据；
具体的，待构建的第一字段组和第二字段组中字段对应的数据可存储于n个服务器中的n个存储节点，由n个服务器102中的执行器组成的分布式计算平台接收到执行计划，从由n个存储节点组成分布式存储系统中读取所述待构建的字段组中字段对应的数据。
S503、依据待构建的第一字段组和第二字段组，对读取的数据进行分组，形成字段组数据；
其中，待构建的字段组为优化器11得到的字段组，将组成字段组中的字段对应的数据读取之后，将读取得到的数据按照字段组展示的方式组合，形成字段组数据。
参见图6，接上述实例，对customer表中c_custkey,c_name,c_nationkey三个字段的数据，组合形成如下的字段组数据：
cp1:c_custkey
cp2:c_custkey，c_nationkey
cp3:c_custkey，c_name
对nation表中的n_nationkey,n_regionkey,n_name三个字段的数据，组合 形成如下的字段组数据：
np1:n_nationkey，n_name
np2:n_nationkey，n_regionkey
对orders表中的o_custkey和o_orderdate字段中的数据，组成字段组数据：o_custkey，o_orderdate。
对region表中的r_regionkey和r_name字段中的数据，组成字段组数据：r_regionkey，r_name。
S504、按照执行步骤对字段组数据进行theta连接，得到符合theta连接执行条件的中间结果；
参见图6，针对theta连接条件c_custkeyθo_custkey，对cp1:c_custkey和字段组：o_custkey，o_orderdate做笛卡尔积，得到满足c_custkeyθo_custkey连接条件的数据r1；针对theta连接条件_nationkeyθn_nationkey，对字段组cp2:c_custkey，c_nationkey和np2:n_nationkey，n_regionkey做做笛卡尔积，得到满足c_nationkeyθn_nationkey的数据r2；针对theta连接条件n_regionkeyθr_regionkey，对字段组np2:n_nationkey，n_regionkey和字段组：r_regionkey，r_name做笛卡尔积，得到满足n_regionkeyθr_regionkey连接条件的数据r3。
S505、按照执行步骤、对中间结果和字段组数据中未进行theta连接的字段组数据进行等值连接，得到符合等值连接执行条件的查询结果；
需要说明的是，步骤S504中得到的中间结果同样为带有数据的字段组，执行等值连接操作时，以字段组共有的主键字段为等值连接参数。
本发明的实例中，参见图6，以主键c_custkey为等值连接的连接条件，对r1和cp3:c_custkey，c_name执行等值连接，得到r4；以主键n_nationkey为等值连接的连接条件，对r2和r3执行等值连接，得到r5；最后以主键c_custkey为等值连接的连接条件，对r4和r5执行等值连接，得到r6。
还需要说明的是，等值连接执行条件可以包含有原生等值连接执行条件和派生等值连接执行条件；原生等值连接执行条件是指依据查询语句的查询条件中包含的字段间的等值连接而形成的执行条件，派生等值连接执行条件是指为了实现得到查询结果而进行的基于共同主键字段的等值连接的执行条件。
若等值连接执行条件包含有原生等值连接执行条件，本步骤可以成：先对字段组数据中未进行theta连接的字段组数据、且涉及原生等值连接的字段组 数据，按照原生等值连接执行条件进行等值连接；再对执行完原生等值连接的字段组数据、字段组数据中未进行theta连接的字段组数据、且不涉及原生等值连接的字段组数据、以及中间结果进行基于共同主键字段的等值连接，得到查询结果。
若等值连接执行条件未包含有原生等值连接执行条件，本步骤即为：对中间结果和字段组数据中未进行theta连接的字段组数据进行基于共同主键字段的等值连接，得到查询结果。
S506、将查询结果映射为输出字段序列；
具体的，本发明的实例中，将得到的r6映射成输出字段序列。
S507、发送输出字段序列。
具体的，得到输出字段序列之后，向数据块查询引擎101发送。
需要说明的是，步骤S501中待构建的第一字段组和第二字段组和执行条件的形式，以及步骤S504和S505公开的处理字段组数据的方式，仅是实现将字段组数据进行过滤，得到符合执行条件的查询结果的一种实现方式。可以确定的是，针对待构建的字段组和执行条件的其他实现方式，若相对现有技术来说，均采用将进行theta连接操作的字段分解成多个字段组，实现以多个字段组的形式分步完成连接操作，能够实现降低了一次进行连接操作时笛卡尔积计算的数据量，大幅度减少的网络传输开销和计算开销，提升执行效率的效果，那均可以落在本发明方案的保护范围之内。
可选地，在本发明的另一实施例中，在步骤S502之后，还包括：
判断执行条件中是否包含单表过滤条件；
若判断出执行条件中包含单表过滤条件，则依据单表过滤条件，对读取得到的数据执行过滤；
此种情况下，步骤S503、依据所述待构建的字段组对读取的数据进行分组，形成字段组数据，包括：
依据待构建的字段组对过滤后的数据进行分组，形成字段组数据；
若判断筛选条件中没有包含单表过滤条件，则执行步骤S503。
其中，在优化器解析的查询语句中，若查询条件包含单表过滤条件，则生成的执行计划中执行条件会包含单表过滤条件，因此，需要以此条件为过滤条 件，删除步骤S502中读取得到的数据中不满足该过滤条件的数据。
可选地，在本发明的另一实施例中，在步骤S503之后，还包括：
判断执行条件中是否包含有字段等值连接执行条件；
若执行条件中包含有原生等值连接执行条件，按照执行步骤，依据原生等值连接执行条件对字段组数据执行等值连接；
可以理解的是，若执行条件中包含有原生等值连接执行条件，那在执行步骤中还应该包含有执行原生等值连接的步骤；另外，在此种情况下，步骤S504为按照执行步骤对执行等值连接后的字段组数据进行连接，得到符合执行条件的中间结果；
若执行条件中未包含原生等值连接执行条件，则直接执行步骤S504。
还需要说明的是，本实施例中，步骤S505可以理解为：按照执行步骤、对中间结果和字段组数据中未进行theta连接的字段组数据、且不涉及原生等值连接的字段组数据进行基于共同主键字段的等值连接，得到查询结果。
下面以“Select c_custkey,c_name,c_nationkey,n_nationkey,n_name,o_custkey,o_orderdate FROM customer,nation,orders where c_nationkey>n_nationkey and c_custkey>o_custkey and c_custkey<n”查询语句为例，说明本发明公开的执行多表连接的控制方法。
上述查询语句中包含两个“>”Theta连接条件和一个“c_custkey<n”单表过滤条件，如下表所示，当单表过滤阈值n设置为5、10、15时，采用本发明的执行多表连接的控制方法可在6～7秒返回查询结果，而未采用本发明公开的方法需要147～524秒才能返回查询结果，本发明方法的最高加速比可达87.3倍。
过滤条件 分解映射优化执行时间(秒) Spark SQL执行时间(秒)
n＝5 7 147
n＝10 6 365
n＝15 6 524
本发明另一实施例还公开了一种优化器，参见图5，包括：
存储器701，用于存储程序以及程序运行中产生的数据；
处理器702，用于通过运行存储器701中的程序，实现以下功能：
解析查询语句，获得查询条件、表名以及表名对应的表在查询语句中涉及的字段；将第一类表中的每张表在查询语句中所涉及的字段进行分解，得到第一类表中的每张表的多个第一字段组；采用将第二类表中的每张表在查询语句中涉及的字段，构成一个第二字段组的方式，获得第二类表中的每张表的第二字段组；依据查询语句、第一字段组和第二字段组，生成执行计划；
其中，查询条件表明参与theta连接操作、且在查询语句中涉及的字段满足第一预设条件的表为第一类表，第一预设条件用于表明表在查询语句中涉及的字段执行theta连接操作时，进行笛卡儿积计算导致数据运算量超于预设阈值；第二类表为表名对应的表中除第一类表以外的表；执行计划用于控制读取第一字段组和第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合查询语句要求的查询结果。
可选地，本发明的另一实施例中，依据上述第一预设条件的用途，设定的第一预设条件可以包括：
表在所述查询语句中涉及的字段的数量超过第一预设阈值；
表在所述查询语句中涉及的字段的存储开销超过第一预设空间阈值；
表在所述查询语句中涉及参与theta连接操作的字段的数量超过第二预设阈值；或者，
表在所述查询语句中涉及参与theta连接操作的字段的存储开销超过第二预设空间阈值。
可选地，本发明的另一实施例中，处理器702执行将第一类表中的每张表在查询语句中所涉及的字段进行分解，得到第一类表中的每张表的多个第一字段组时，具体用于：
对于第一类表中的每张表，分别将表中参与theta连接操作的字段和主键构成第一子类字段组，将表中在查询语句中涉及、但未参与theta连接操作的字段和主键构成第二子类字段组；
其中，处理器702执行依据查询语句、第一字段组和第二字段组，生成执行计划时，具体用于：
依据查询语句、第一子类字段组、第二子类字段组和第二字段组，生成执行计划，执行计划用于控制读取第一子类字段组、第二子类字段组和第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合查询语句要求的查询结果。
可选地，本发明的另一实施例中，处理器702执行将表中参与theta连接操作的字段和主键构成第一子类字段组操作时，具体用于：
将表中参与theta连接操作的所有字段和主键构成一个第一子类字段组；
或者具体用于：将表中与同一个表的字段进行theta连接操作的字段和主键构成一个第一子类字段组。
需要说明的是，处理器702在上述实施例的具体工作过程均可参见对应图2的方法实例中的内容，此处不再赘述。
可选地，本发明的另一实施例中，处理器702在依据查询语句、第一子类字段组、第二子类字段组和第二字段组，生成执行计划之前，还用于：
若判断第一类表中的某张表的第一子类字段组中字段满足第二预设条件，分解第一子类字段组，得到多个字段组；判断对满足第二预设条件的第一子类字段组进行分解得到的多个字段组是否均包含主键；若对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组未均包含主键，则在未包含主键的字段组中添加主键；
其中，处理器702还用于在未包含主键的字段组中添加主键之后，以及在判断出对满足第二预设条件的第一子类字段组进行分解得到的多个字段组中均包含主键之后，执行依据查询语句、第一子类字段组、第二子类字段组和第二字段组，生成执行计划的步骤。
可选地，上述的第二预设条件包括：
第一子类字段组中字段的数量超过第三预设阈值；
第一子类字段组中字段的存储开销超过第三预设空间阈值。
其中，本实施例中，处理器702具体工作过程可参见对应图3的实施例中，步骤S308～S311的内容，此处不再赘述。
可选地，本发明的另一实施例中，处理器702在构成第一子类字段组和第二子类字段组之后，还用于：
若判断出第一类表中的某张表的第一类字段组中的字段用于参与多个表的theta连接操作，按照预定规则分解第一子类字段组，得到多个字段组；判断分解得到的多个字段组是否均包含主键；若分解得到的多个字段组未均包含主键，则在未包含主键的字段组中添加主键；
其中，上述的预设规则包括：第一子类字段组中与同一个表进行theta连接的字段分解为一个字段组，且分解成的任意两个字段组之间的共有字段满足第三预设条件，第三预设条件用于判断分解构成的两个字段组之间的关联性；
其中，处理器702还用于在未包含主键的字段组中添加主键之后，以及在判断出分解得到的多个字段组均包含主键之后，执行依据查询语句、第一子类字段组、第二子类字段组和第二字段组，生成执行计划的步骤。
其中，本实施例中，处理器702具体工作过程可参见对应图4的实施例中，步骤S408～S411的内容，此处不再赘述。
本发明还公开了一种服务器集群，参见图6，包括n个服务器，n大于或等于1，每个服务器均包括处理器801和存储器802；其中：
n个服务器的处理器801组成分布式计算平台，用于接收执行计划，执行计划包含待构建的第一字段组和第二字段组、执行步骤和执行条件；读取待构建的第一字段组和第二字段组中字段对应的数据；依据待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据；按照执行步骤对字段组数据进行满足执行条件的连接操作，得到查询结果；将查询结果映射为输出字段序列；发送输出字段序列；
n个服务器的存储器202组成分布式存储系统，用于存储字段对应的数据。
其中，处理器201可以理解成每个服务器中的执行器，存储器202可以理解成每个服务器中的存储节点。
可选地，本发明的另一实施例中，执行条件包括：theta连接执行条件和等值连接执行条件；分布式计算平台执行按照执行步骤对字段组数据进行满足 执行条件的连接操作，得到查询结果时，具体用于：
按照执行步骤对字段组数据进行theta连接，得到符合theta连接执行条件的中间结果；按照执行步骤、对中间结果和字段组数据中未进行theta连接的字段组数据进行等值连接，得到符合等值连接执行条件的查询结果。
需要说明的是，由处理器801组成分布式计算平台在上述实施例的具体工作过程均可参见对应图5的方法实例中的内容，此处不再赘述。
可选地，本发明的另一实施例中，分布式计算平台读取所述待构建的第一字段组和第二字段组中涉及的字段的数据之后，还用于：
判断执行条件中是否包含单表过滤条件；若执行条件中包含单表过滤条件，则依据所述单表过滤条件，对读取得到的数据执行过滤；
其中，分布式计算平台执行依据待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据的操作时，具体用于：
依据待构建的第一字段组和第二字段组对过滤后的数据进行分组，形成字段组数据。
可选地，本发明的另一实施例中，分布式计算平台依据待构建的字段组对读取的数据进行分组，形成字段组数据之后，还用于：
判断执行条件包含原生等值连接执行条件，按照执行步骤，依据原生等值连接执行条件对字段组数据执行等值连接；
其中，分布式计算平台执行按照执行步骤对字段组数据进行连接，得到符合执行条件的查询结果时，具体用于：按照执行步骤对执行等值连接后的字段组数据进行连接，得到符合执行条件的查询结果。
需要说明的是，由处理器801组成分布式计算平台在上述实施例的具体工作过程均可参见对应的方法实例中的内容，此处不再赘述。
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本发明的精神或范围的情况下，在其它实施例中实现。因此，本发明将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (22)

  1. 一种执行多表连接操作的控制方法，其特征在于，包括：
    解析查询语句，获得查询条件、表名以及所述表名对应的表在所述查询语句中涉及的字段；
    将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组；其中，所述查询条件表明参与theta连接操作、且在所述查询语句中涉及的字段满足第一预设条件的表为第一类表，所述第一预设条件用于表明表在所述查询语句中涉及的字段执行theta连接操作时，进行笛卡儿积计算导致数据运算量超于预设阈值；
    采用将第二类表中的每张表在所述查询语句中涉及的字段，构成一个第二字段组的方式，获得所述第二类表中的每张表的第二字段组；其中，所述第二类表为所述表名对应的表中除所述第一类表以外的表；
    依据所述查询语句、所述第一字段组和所述第二字段组，生成执行计划，所述执行计划用于控制读取所述第一字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
  2. 根据权利要求1所述的控制方法，其特征在于，所述第一预设条件包括：
    表在所述查询语句中涉及的字段的数量超过第一预设阈值；
    表在所述查询语句中涉及的字段的存储开销超过第一预设空间阈值；
    表在所述查询语句中涉及参与theta连接操作的字段的数量超过第二预设阈值；或者，
    表在所述查询语句中涉及参与theta连接操作的字段的存储开销超过第二预设空间阈值。
  3. 根据权利要求1所述的控制方法，其特征在于，所述将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组，包括：
    对于所述第一类表中的每张表，分别将表中参与theta连接操作的字段和主键构成第一子类字段组，将表中在所述查询语句中涉及、但未参与theta连 接操作的字段和主键构成第二子类字段组；
    其中，所述依据所述查询语句、所述第一字段组和所述第二字段组，生成执行计划，包括：
    依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划，所述执行计划用于控制读取所述第一子类字段组、所述第二子类字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
  4. 根据权利要求3所述的控制方法，其特征在于，所述将表中参与theta连接操作的字段和主键构成第一子类字段组，包括：
    将表中参与theta连接操作的所有字段和主键构成一个第一子类字段组；
    或者；将表中与同一个表的字段进行theta连接操作的字段和主键构成一个第一子类字段组。
  5. 根据权利要求3或4所述的控制方法，其特征在于，构成所述第一子类字段组和所述第二类子字段组之后，还包括：
    若所述第一类表中的某张表的第一子类字段组中的字段用于参与多个表的theta连接操作，按照预定规则分解所述第一子类字段组，得到多个字段组；其中，所述预设规则包括：所述第一子类字段组中与同一个表进行theta连接的字段分解为一个字段组，且分解成的任意两个字段组之间的共有字段满足第三预设条件，所述第三预设条件用于判断分解构成的两个字段组之间的关联性；
    判断所述分解得到的多个字段组是否均包含主键；
    若所述分解得到的多个字段组未均包含主键，则在未包含主键的字段组中添加主键，并返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤；
    若所述分解得到的多个字段组均包含主键，则返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
  6. 根据权利要求3、4或5所述的控制方法，其特征在于，所述依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生 成执行计划之前，还包括：
    若所述第一类表中的某张表的第一子类字段组中字段满足第二预设条件，分解所述第一子类字段组，得到多个字段组；
    判断对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组、是否均包含主键；
    若所述对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组、未均包含主键，则在未包含主键的字段组中添加主键，并返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤；
    若所述对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组均包含主键，则返回执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
  7. 根据权利要求6所述的控制方法，其特征在于，所述第二预设条件包括：
    所述第一子类字段组中字段的数量超过第三预设阈值；
    所述第一子类字段组中字段的存储开销超过第三预设空间阈值。
  8. 一种执行多表连接的控制方法，其特征在于，包括：
    接收执行计划，所述执行计划包含待构建的第一字段组和第二字段组、执行步骤和执行条件；
    读取所述待构建的第一字段组和第二字段组中字段对应的数据；
    依据所述待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据；
    按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果；
    将所述查询结果映射为输出字段序列；
    发送所述输出字段序列。
  9. 根据权利要求8所述的控制方法，其特征在于，所述读取所述待构建的第一字段组和第二字段组中涉及的字段的数据之后，还包括：
    判断所述执行条件中是否包含单表过滤条件；
    若所述执行条件中包含单表过滤条件，则依据所述单表过滤条件，对所述 读取的数据执行过滤；
    其中，所述依据所述待构建的第一字段组和第二字段组对所述读取的数据进行分组，形成字段组数据，包括：
    依据所述待构建的第一字段组和第二字段组对过滤后的数据进行分组，形成字段组数据。
  10. 根据权利要求8所述的控制方法，其特征在于，所述执行条件包括：theta连接执行条件和等值连接执行条件；所述按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果，包括：
    按照所述执行步骤对所述字段组数据进行theta连接，得到符合所述theta连接执行条件的中间结果；
    按照所述执行步骤、对所述中间结果和所述字段组数据中未进行theta连接的字段组数据进行等值连接，得到符合所述等值连接执行条件的查询结果。
  11. 根据权利要求8所述的控制方法，其特征在于，所述依据所述待构建的字段组对读取的数据进行分组，形成字段组数据之后，还包括：
    判断所述执行条件包含原生等值连接执行条件，按照执行步骤，依据所述原生等值连接执行条件对所述字段组数据执行等值连接；
    其中，所述按照所述执行步骤对所述字段组数据进行连接，得到符合所述执行条件的查询结果，包括：
    按照执行步骤对执行等值连接后的字段组数据进行连接，得到符合所述执行条件的查询结果。
  12. 一种优化器，其特征在于，包括：
    存储器，用于存储程序以及程序运行中产生的数据；
    处理器，用于通过运行所述存储器中的程序，实现以下功能：
    解析查询语句，获得查询条件、表名以及所述表名对应的表在所述查询语句中涉及的字段；将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组；采用将第二类表中的每张表在所述查询语句中涉及的字段，构成一个第二字段组的方式，获得所述第二类表中的每张表的第二字段组；依据所述查询语句、所述第一字段组和所述第二字段组，生成执行计划；
    其中，所述查询条件表明参与theta连接操作、且在所述查询语句中涉及 的字段满足第一预设条件的表为第一类表，所述第一预设条件用于表明表在所述查询语句中涉及的字段执行theta连接操作时，进行笛卡儿积计算导致数据运算量超于预设阈值；所述第二类表为所述表名对应的表中除所述第一类表以外的表；所述执行计划用于控制读取所述第一字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
  13. 根据权利要求12所述的优化器，其特征在于，所述第一预设条件包括：
    表在所述查询语句中涉及的字段的数量超过第一预设阈值；
    表在所述查询语句中涉及的字段的存储开销超过第一预设空间阈值；
    表在所述查询语句中涉及参与theta连接操作的字段的数量超过第二预设阈值；或者，
    表在所述查询语句中涉及参与theta连接操作的字段的存储开销超过第二预设空间阈值。
  14. 根据权利要求12所述的优化器，其特征在于，所述处理器执行将第一类表中的每张表在所述查询语句中所涉及的字段进行分解，得到所述第一类表中的每张表的多个第一字段组时，具体用于：
    对于所述第一类表中的每张表，分别将表中参与theta连接操作的字段和主键构成第一子类字段组，将表中在所述查询语句中涉及、但未参与theta连接操作的字段和主键构成第二子类字段组；
    其中，所述处理器执行依据所述查询语句所述第一字段组和所述第二字段组，生成执行计划时，具体用于：
    依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划，所述执行计划用于控制读取所述第一子类字段组、所述第二子类字段组和所述第二字段组中字段对应的数据，并按照执行步骤对读取的数据进行满足执行条件的连接操作，得到符合所述查询语句要求的查询结果。
  15. 根据权利要求14所述的优化器，其特征在于，所述处理器执行所述将表中参与theta连接操作的字段和主键构成第一子类字段组操作时，具体用于：
    将表中参与theta连接操作的所有字段和主键构成一个第一子类字段组；
    或者具体用于：将表中与同一个表的字段进行theta连接操作的字段和主键构成一个第一子类字段组。
  16. 根据权利要求14或15所述的优化器，其特征在于，所述处理器在构成第一子类字段组和第二子类字段组之后，还用于：
    若判断出所述第一类表中的某张表的第一类字段组中的字段用于参与多个表的theta连接操作，按照预定规则分解所述第一子类字段组，得到多个字段组，其中，所述预设规则包括：所述第一子类字段组中与同一个表进行theta连接的字段分解为一个字段组，且分解成的任意两个字段组之间的共有字段满足第三预设条件，所述第三预设条件用于判断分解构成的两个字段组之间的关联性；判断所述分解得到的多个字段组是否均包含主键；若所述分解得到的多个字段组未均包含主键，则在未包含主键的字段组中添加主键；
    其中，所述处理器还用于在未包含主键的字段组中添加主键之后，以及在判断出所述分解得到的多个字段组均包含主键之后，执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
  17. 根据权利要求14、15或16所述的优化器，其特征在于，所述处理器在依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划之前，还用于：
    若判断所述第一类表中的某张表的第一子类字段组中字段满足第二预设条件，分解所述第一子类字段组，得到多个字段组；判断对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组是否均包含主键；若所述对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组未均包含主键，则在未包含主键的字段组中添加主键；
    其中，所述处理器还用于在未包含主键的字段组中添加主键之后，以及在判断出对满足所述第二预设条件的第一子类字段组进行分解得到的多个字段组中均包含主键之后，执行依据所述查询语句、所述第一子类字段组、所述第二子类字段组和所述第二字段组，生成执行计划的步骤。
  18. 根据权利要求17所述的优化器，其特征在于，所述第二预设条件包括：
    所述第一子类字段组中字段的数量超过第三预设阈值；
    所述第一子类字段组中字段的存储开销超过第三预设空间阈值。
  19. 一种服务器集群，其特征在于，包括n个服务器，n大于或等于1，每个所述服务器均包括处理器和存储器；其中：
    所述n个服务器的处理器组成分布式计算平台，用于接收执行计划，所述执行计划包含待构建的第一字段组和第二字段组、执行步骤和执行条件；读取所述待构建的第一字段组和第二字段组中字段对应的数据；依据所述待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据；按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果；将所述查询结果映射为输出字段序列；发送所述输出字段序列；
    所述n个服务器的存储器组成分布式存储系统，用于存储字段对应的数据。
  20. 根据权利要求19所述的服务器集群，其特征在于，所述分布式计算平台读取所述待构建的第一字段组和第二字段组中涉及的字段的数据之后，还用于：
    判断所述执行条件中是否包含单表过滤条件；若所述执行条件中包含单表过滤条件，则依据所述单表过滤条件，对读取的数据执行过滤；
    其中，所述分布式计算平台执行所述依据所述待构建的第一字段组和第二字段组对读取的数据进行分组，形成字段组数据的操作时，具体用于：
    依据所述待构建的第一字段组和第二字段组对过滤后的数据进行分组，形成字段组数据。
  21. 根据权利要求19所述的服务器集群，其特征在于，所述执行条件包括：theta连接执行条件和等值连接执行条件；所述分布式计算平台执行所述按照所述执行步骤对所述字段组数据进行满足所述执行条件的连接操作，得到查询结果时，具体用于：
    按照所述执行步骤对所述字段组数据进行theta连接，得到符合所述theta连接执行条件的中间结果；按照所述执行步骤、对所述中间结果和所述字段组数据中未进行theta连接的字段组数据进行等值连接，得到符合所述等值连接执行条件的查询结果。
  22. 根据权利要求19所述的服务器集群，其特征在于，所述分布式计算 平台在依据所述待构建的字段组对读取的数据进行分组，形成字段组数据之后，还用于：
    判断所述执行条件包含原生等值连接执行条件，按照执行步骤，依据原生等值连接执行条件对所述字段组数据执行等值连接；
    其中，所述分布式计算平台执行按照所述执行步骤对所述字段组数据进行连接，得到符合所述执行条件的查询结果时，具体用于：
    按照执行步骤对执行等值连接后的字段组数据进行连接，得到符合所述执行条件的查询结果。
PCT/CN2016/101366 2016-09-30 2016-09-30 执行多表连接操作的控制方法及对应装置 WO2018058671A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201680089781.3A CN109791543B (zh) 2016-09-30 2016-09-30 执行多表连接操作的控制方法及对应装置
PCT/CN2016/101366 WO2018058671A1 (zh) 2016-09-30 2016-09-30 执行多表连接操作的控制方法及对应装置
US16/370,343 US11301470B2 (en) 2016-09-30 2019-03-29 Control method for performing multi-table join operation and corresponding apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/101366 WO2018058671A1 (zh) 2016-09-30 2016-09-30 执行多表连接操作的控制方法及对应装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/370,343 Continuation US11301470B2 (en) 2016-09-30 2019-03-29 Control method for performing multi-table join operation and corresponding apparatus

Publications (1)

Publication Number Publication Date
WO2018058671A1 true WO2018058671A1 (zh) 2018-04-05

Family

ID=61763009

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/101366 WO2018058671A1 (zh) 2016-09-30 2016-09-30 执行多表连接操作的控制方法及对应装置

Country Status (3)

Country Link
US (1) US11301470B2 (zh)
CN (1) CN109791543B (zh)
WO (1) WO2018058671A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110489452A (zh) * 2019-08-21 2019-11-22 中国科学院深圳先进技术研究院 多路数据流θ连接优化方法及系统
CN110955666B (zh) * 2019-12-17 2023-10-03 用友网络科技股份有限公司 数据处理方法和数据处理装置
US11947927B2 (en) * 2019-12-26 2024-04-02 Yahoo Assets Llc Sorting unsorted rows of a composite dataset after a join operation
CN111352950B (zh) * 2020-03-04 2023-05-16 上海达梦数据库有限公司 数据库表等值连接的优化方法、装置、服务器及存储介质
CN112199390B (zh) * 2020-09-30 2023-05-30 上海达梦数据库有限公司 一种数据库中数据查询方法、装置、设备及存储介质
CN112615910B (zh) * 2020-12-10 2021-10-29 中国科学院深圳先进技术研究院 一种数据流连接优化方法、系统、终端以及存储介质
CN117390055B (zh) * 2023-12-13 2024-03-01 火石创造科技有限公司 Jooq连表语句生成方法、设备以及介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101840430A (zh) * 2010-04-28 2010-09-22 北京握奇数据系统有限公司 智能卡数据库多表操作方法及装置
CN102262675A (zh) * 2011-08-12 2011-11-30 北京握奇数据系统有限公司 数据库查询方法及智能卡
WO2015184762A1 (zh) * 2014-11-06 2015-12-10 中兴通讯股份有限公司 数据库查询方法及装置

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7062481B2 (en) * 2001-09-28 2006-06-13 Ncr Corp. Eliminating group-by operations in a join plan
US20050131893A1 (en) * 2003-12-15 2005-06-16 Sap Aktiengesellschaft Database early parallelism method and system
US8126870B2 (en) * 2005-03-28 2012-02-28 Sybase, Inc. System and methodology for parallel query optimization using semantic-based partitioning
US7908242B1 (en) * 2005-04-11 2011-03-15 Experian Information Solutions, Inc. Systems and methods for optimizing database queries
WO2008034213A1 (en) * 2006-09-18 2008-03-27 Infobright Inc. A method and system for data compression in a relational database
US8150850B2 (en) * 2008-01-07 2012-04-03 Akiban Technologies, Inc. Multiple dimensioned database architecture
US20110214165A1 (en) * 2010-02-26 2011-09-01 David Kerr Jeffreys Processor Implemented Systems And Methods For Using Identity Maps And Authentication To Provide Restricted Access To Backend Server Processor or Data
US8935232B2 (en) * 2010-06-04 2015-01-13 Yale University Query execution systems and methods
CN102467521B (zh) * 2010-11-08 2013-09-04 北大方正集团有限公司 一种易扩展的多级分类检索方法及系统
CN102081678B (zh) * 2011-03-14 2013-07-03 华中科技大学 一种数据库查询中最优执行计划的搜索方法
CN102521406B (zh) * 2011-12-26 2014-06-25 中国科学院计算技术研究所 海量结构化数据复杂查询任务的分布式查询方法和系统
US10120902B2 (en) * 2014-02-20 2018-11-06 Citus Data Bilgi Islemleri Ticaret A.S. Apparatus and method for processing distributed relational algebra operators in a distributed database
US9870397B2 (en) * 2014-08-19 2018-01-16 International Business Machines Corporation Processing multi-way theta join queries involving arithmetic operators on MapReduce
US10133778B2 (en) * 2015-11-20 2018-11-20 Sap Se Query optimization using join cardinality
US10496643B2 (en) * 2016-02-08 2019-12-03 Microsoft Technology Licensing, Llc Controlling approximations of queries
CN105930407B (zh) * 2016-04-18 2019-05-17 北京思特奇信息技术股份有限公司 一种分布式数据库跨库关联查询方法及系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101840430A (zh) * 2010-04-28 2010-09-22 北京握奇数据系统有限公司 智能卡数据库多表操作方法及装置
CN102262675A (zh) * 2011-08-12 2011-11-30 北京握奇数据系统有限公司 数据库查询方法及智能卡
WO2015184762A1 (zh) * 2014-11-06 2015-12-10 中兴通讯股份有限公司 数据库查询方法及装置

Also Published As

Publication number Publication date
CN109791543A (zh) 2019-05-21
US20190228014A1 (en) 2019-07-25
CN109791543B (zh) 2021-02-12
US11301470B2 (en) 2022-04-12

Similar Documents

Publication Publication Date Title
WO2018058671A1 (zh) 执行多表连接操作的控制方法及对应装置
US10185755B2 (en) Orchestration of data query processing in a database system
US10558664B2 (en) Structured cluster execution for data streams
US10452632B1 (en) Multi-input SQL-MR
US10007700B2 (en) Query optimization for group-by extensions and distinct aggregate functions
US7181460B2 (en) User-defined aggregate functions in database systems without native support
US9576026B2 (en) System and method for distributed SQL join processing in shared-nothing relational database clusters using self directed data streams
CN107729371B (zh) 区块链的数据索引及查询方法、装置、设备及存储介质
WO2016011904A1 (zh) 数据库访问方法及装置、数据库系统
WO2015110062A1 (zh) 一种分布式数据存储方法、装置和系统
CN106611037A (zh) 用于分布式图计算的方法与设备
WO2015074466A1 (zh) 一种数据查询方法及装置
US20180276277A1 (en) Data Query Method and Apparatus, and Database System
EP2590088B1 (en) Database queries enriched in rules
WO2017088666A1 (zh) 一种数据存储方法和协调节点
US10180960B2 (en) Query processing
CN107870949A (zh) 数据分析作业依赖关系生成方法和系统
CN113918605A (zh) 数据查询方法、装置、设备以及计算机存储介质
US20230161765A1 (en) System and method for disjunctive joins using a lookup table
CN110471935B (zh) 一种数据操作的执行方法、装置、设备和存储介质
CN108549688B (zh) 一种数据操作的优化方法、装置、设备和存储介质
CN113779084A (zh) 基于分布式的时序数据查询方法、设备、介质及产品
US20230205769A1 (en) System and method for disjunctive joins
CN106326295B (zh) 语义数据的存储方法及装置
CN114817512B (zh) 问答推理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16917421

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16917421

Country of ref document: EP

Kind code of ref document: A1