FR2701322A1 - Process for parallel execution of relational database enquiries and use of this process - Google Patents

Abstract

The present invention relates to a process for parallel execution of relational database enquiries within a parallel multiprocessor environment with distributed memory consisting in incorporating monitoring into the enquiry during its compilation, the monitoring being achieved by combining a set of atomic monitoring operations (lw, sw, gw, stp, sp, gp, mp) in order to construct a monitoring graph which is superimposed on the graph of the enquiry, this graph expressing the dependencies between the data through the links between the nodes (I1, I2, 3, I6) of operations and the types of sequencing dependence through builder nodes of two types (SEQ, PIPE).

Description

METHOD FOR PARALLEL EXECUTION OF DATABASE QUERY
RELATIONAL AND USE OF THE PROCESS.

The present invention relates to a method for parallel execution of a request for relational databases in a multi-processor environment with distributed memory and the use of such a method.

 If it has been envisaged to process relational database requests in parallel to increase either the number of transactions that it is possible to process per second, or reduce the response time of complex requests, such a possibility raises problems related to the control necessary for the synchronization of the different parts of the request and the distribution of work between the different processors of a machine with multiple parallel processors with distributed memory.

A first object of the invention is therefore to solve the problems raised above.

This object is achieved by the fact that the method of parallel execution of query of relational databases in a parallel multi-processor environment with distributed memory consists in integrating the control in the request during its compilation, the control being obtained by combining a set of atomic control operations to materialize a control graph which is superimposed on the query graph, this graph expressing the dependencies between the data by the links between the operation nodes and the types of sequencing dependency by constructor nodes of two types.

According to another particularity, the control graph expresses the grouping of nodes in focus (home) of a base relationship, the co-localization of several operations on the same focus of the multiprocessor architecture, the combination of several operations in local operation and the combination of several local operations in association.

According to another particular feature, the first type of constructor node (PIPE) allows the execution of a next operation as soon as the previous operation has produced a tuple.

According to another particular feature, the second type of constructor node (SEQ) allows the execution of the next operation only after the complete completion of the previous operation.

According to another particular feature, the synchronization between operations executed on groups of different nodes and operations executed on the same group is carried out by two families of control primitives allowing the establishment of a control mechanism.

According to another particular feature, the first family consists of propagation primitives (stp, gp, mp, sp), the other family of waiting primitives (1w, gw, mw, sw), the control mechanism being unable to be constituted only by the connection of two primitives of opposite type (wait, propagation).

According to another particular feature, the waiting primitives send to the next control operation which is a propagation primitive, an activation message when at least one end of flow type (EOS) message is received.

According to another particular feature, a multiple wait primitive verifies that the number of end-of-flow message received is equal to the number of local operations executed by the same node and producing the end-of-flow message.

According to another particular feature, a global wait primitive verifies that the number of end-of-flow message received is equal to the number of node of the home on which one or more operations are executed in parallel.

According to another particular feature, the propagation primitives send an end-of-flow message to at least one instance of the following control operation when the activation message is received.

According to another particular feature, a multiple propagation primitive sends as many end-of-flow messages as there are next local operations.

According to another particular feature, a global propagation primitive sends as many end-of-flow messages as there are nodes of the home on which the following local operation or operations are executed.

According to another particular feature, a propagation and trigger primitive also sends a trigger message to the next local operation, in order to trigger it.

According to another particularity, program portion tables formed from the primitives make it possible to reconstruct by the successive association of several tables the graph corresponding to the parallel execution of a query on a relational database in a multi-processor environment. with distributed memory.

Another object of the invention is to propose a use of the method according to the invention.

This goal is achieved by the fact that the use of the method is carried out in the parallelization phase of a request manager responsible for compiling the requests expressed in ESQL language by executing an analysis phase, a physical optimization phase a parallelization phase and a code generation phase in C ++ language. Other characteristics and advantages of the present invention will appear more clearly on reading the description below made with reference to the accompanying drawings in which
FIG. 1 represents a graph representing a request on a database associated with the control graph,
FIG. 2A represents the distribution of a request on the nodes of a parallel multi-processor system,
FIG. 2B represents a parallel multi-processor system with distributed memory,
FIG. 2C schematically represents in the form of a graph the different possibilities of connection between the operating nodes and the constructor nodes,
FIG. 2D represents the materialization of the graphs of the request represented in FIG. 2C in the form of a control graph representing the sequence of the control primitives,
FIG. 3 schematically represents a local operation and the allocation of the processes to this operation,
FIG. 4 represents a request containing two local operations grouped in two associations,
FIGS. 5 to 13 represent the different possibilities of query graphs associating the different combinations between the operation nodes and the constructor nodes, and the translation in the form of a control graph is formed of a sequence of control primitives corresponding to the graph request.

FIG. 14 represents the block diagram of a request manager in which the parallelization method is used.

The invention relates to a method for parallel execution of database requests in a multi-processor environment with distributed memory such as that shown in FIG. 2B. In such an environment, each processor 31, 3i, 3n is associated with a distributed memory 11, 1i, 1n and the processors communicate with each other by nodes 41, 4i, 4n. An extended relational database management system developed on such a machine with parallel processors makes it possible to take advantage of parallelism to both increase the number of transactions that can be processed per second and reduce the response time for requests complex.

An example of a relational database is shown in Appendix 1.

 This base includes three relationships, a first "car" comprising the following attributes: vehicle number (vehicle no.), Brand, type distributed by hash on the attribute (vehicle no.). A second "buy" relationship includes the attributes of identification of the person (social security number), identification of the car (vehicle number) and date (date). The buy relation is distributed by hash on the n0 attribute of veh. in association with the car relationship. A third "control" relation includes the attributes auto identifier (vehicle number), organization identifier (organization number), and the date (date).

This relation is distributed by hash on the identification attribute of the organization.

Each processor (3i) therefore has its own local memory (li) and is the only one to access the data in the memory. The processors communicate by sending messages through the nodes (4i). The storage of the data constituted by the attributes of the base is distributed over all the nodes of the machine. Thus, each relation of the database is partitioned into a packet, each packet being placed on a node of the machine. For a given relation, for example car, a subset of the nodes of the machine is used for the storage of the relation. Thus the car relationship will for example be stored on the first three nodes, 41, 42, 43 of the machine associated with the memories M1, M2, M3. This subset of machine nodes is called the "home" of the relationship.

Similarly, as shown in FIG. 2A, a relation R is stored on a set of nodes Ni to Nn. On this relation and by the processes associated with the "home" of the relation, a "select" selection operation is carried out which is used as result in a "join" operation with a relation S stored on a set Ni to Ni + n of nodes constituting the "home" of the relation S. The simplest method of allocation of process to obtain the parallel execution of the request consists in assigning to each node, operation of the graph of the request, a set process responsible for executing it. The number of processes used for the processing of an operation on a relation will depend in fact on the number of nodes used for the storage of the relation, ie the size of the "home" of the relation.

The parallel execution model of the invention supports three types of parallelism.

The first is intra-operation parallelism which is based on the horizontal partitioning of a relationship into several packets. This makes it possible to carry out a processing (for example "select" operation) on a relation (R) by several processes which execute in parallel each on a packet (Sel 1, Seli,
Selm).

The parallel execution model of the invention supports two other types of parallelism, inter-operation parallelism and the pipeline. Interoperational parallelism can be implemented when two or more operations are independent in terms of sharing intermediate data, for example the "select" selection operations on the "home" CONTROL and on the "home" (PURCHASED) figure 1. The pipeline (figure 3) is a form of fine-grained parallelism where two operations, for example "Select" and "store" in figure 3 exchange intermediate data in producer-consumer mode.

When formulating a query in SQL language which consists, for example, of finding Renault automobiles which have been purchased and not checked since January 1, 1986, such a query is expressed in SQL language as shown in Figure 2D. A possible plan for executing such a request can be generated by a physical optimizer program and correspond to the plan shown in appendix 2.

This leads to a program represented in appendix 3 and corresponding to the request which is represented by a graph in FIG. 1 on which the sequence of the program is represented. This graph of FIG. 1 includes in the shaded areas in dark, the representation of the local operations and in the boxes in lighter gray, the constructions of association.

The constructions of local operations isolate an execution entity in the program, for example the processor Pj on which the selection Selj of the node Nj is executed as represented in FIG. 3. This construction groups together the data operations which communicate locally with each node of a given "home", for example by exchanging intermediate data in a local pipeline mode which is called, internal local pipeline.

The data operations which are part of the local operation represent a sub-graph which has a simple root, (either with data flow or activation messages) and one or more sheets (data communication operation: sending, local shipment or storage). With each received message, all data operations in the local operation are performed synchronously.

An association (A1, A2, FIG. 4) isolates in the graph of the parallel execution plan a component which is a subgraph of data and of command operations located on a "home". This component A1 is composed of local operations (Select, diff, transmit) and associated command operations which communicate locally on each node of the "home".

Each element of an association is distributed on all the nodes of the "home" where the association is located: Two local operations located in the same association exchange data in one or two modes: the first mode is a local pipeline mode (13 , 16) called external local pipeline and the second mode is through a stored intermediate relation (SI).

In a variant of the invention, it is possible that the operation is executed on a "home" called floating point where the nodes forming part of the "home" are chosen at the time of the execution of the request according to criteria such as the load of the processors at this moment and not according to the localization of the data. In this case, the data used by the operation are transferred to this floating "home".

FIG. 2A illustrates the allocation of processes for a query comprising a selection and a join. The selection is made on the "home" of R, that is to say that a process on each node of "home" executes the selection on a packet of R and the join is executed on the "home" of S, the results of the selection being sent to the "home" of S. There are therefore as many processes executing the selection as there are nodes "m" storing the relation R and as many processes executing the join as there are nodes storing the relation S. The processes corresponding to two remote operations, that is to say located on different "homes" of the machine, such as the selection and joining operations of FIG. 2A need to exchange intermediate data. functionality is ensured by communication operations which allow in particular to redistribute data from one "home" to another. Data sends are always carried out in pipeline, that is to say that the tuples are sent to the recipient operation as and when they are produced.

 In the execution model, the intermediate data can be either consumed in pipeline (ex: 11, 12, 13, 16 figure 1) as and when they arrive on the "home" of the recipient operation, are stored in intermediate relationships (S1, S2, Figure 1) in the "home" of the operation that uses the materialized relationship. In this case, the recipient operation of the pipeline is a "store" operation (loc op3, loc op4, Figure 1) which materializes the intermediate relationship. The arrival of data intended for an operation causes the activation of the process which is responsible for executing this operation.

The producer and consumer processes are therefore active simultaneously and do not require synchronization.

When two or more operations (Select, diff, figure 4) of the request are located on the same "home" (PURCHASED, CAR), example the nodes i, a, n, M of the car relationship (figure 4) and exchange locally on this “home” of intermediate data in pipeline 13, 16, the execution of these operations can be carried out by a unique process on each node of the “home”. This is the case of the "Select" and "Diff" operations on the nodes i, a, m which are carried out by the processes Pi to Pm. In this case, the process executes successively by calling procedures, all the operations for a tuple at the input of the first operation, before the next tuple is processed. This mode of execution of operations is called "Internal local pipeline".

The operation subgraph is called "Localized operation" or "local op".

Such a sub-graph represented in FIG. 4 constitutes the minimum synchronization entity.

The intermediate results produced on a node by the selection operation are therefore the only ones which can join with the tuples coming from the relation Si present on the same node. The two operations "select" and "Diff" belong to the same local operation "local op1" and are therefore executed by a single process for each node.

In addition, two local operations ("localppl", "local ~ op2") can locally exchange intermediate data as shown in Figure 4. The locality of the exchanges is delimited by an association constructor which allows to isolate in the graph of the requests a set of operations that exchange data locally. Thus the operations "locppl", "loc ~ op2" and "loc ~ op4" (Figure 1) exchange data, and are grouped in the association A3. Local operations can exchange data either in pipeline (asynchronous local pipeline) (ex "loc ~ op1" with "locpp2, fig 1), or through an intermediate relationship (" loc ~ op 4 "with" loc pp2 through S2 ) .The processes allocated to the two operations "loc op1" "loc # op2" are different and the communications between these processes are explicit. Local synchronization mechanisms are then necessary between these processes.

In addition, communication operations make it possible to redistribute data from one "home" to another or to send the results of an association to another local ~ op of another association.

Only root and leaf operations of a subgraph can communicate with distant processes.

An example of use of a "transmit, local ~ transmit, and, store" communication operation is given below for a communication operation which transmits the result II of a previous operation to the node of the "home" S by addressing. computed (hash) "hash" on the second attribute of Il. The send operation is distributed on R's "home" like the operation that produces it.

Il = {home (R)} 12 = {home (R)} tansmit (Il, home (S), hash on 11.2)
The request of figure 4 which results in the program below also gives an example of use of the local operation ~ transmit thome (buy, car)}, Assoc A3 (loc ~ op1) 13 = Select (car, car.brand = "Renault", (car n0 vehicle)) 16 = Difference (13, S1) Join thome (S)} (12, H on ..) 17 = local transmit (16) End ~ loc opi loc ~ op 2
Result = join (S2, 17, 17.nOvéh. = S2, novéh. (17.n véh)) End loc op 2
End assoc A3
In addition, the synchronization of the parallel instances of the same operation such as for example the parallel instances of the different local operations "loc ~ op1 (fig. 1) on each node of the" home "(car) as well as more generally the synchronization of the operations of a request require control mechanisms, since the triggering of an operation is obtained by the triggering of all the processes associated with its execution and its termination is equivalent to the termination of all these processes. 'an operation consumes data in pipeline, its triggering is linked to the arrival of tuple on the pipeline and its termination is linked to the end of pipelined data. However, the end of pipelined data itself depends on the end of the The operation that produces them. The control mechanisms provide all of these functionalities through control operations that exchange control messages.

Some of these control operators are centralized while others are distributed. There are three classes of operations corresponding to three types of intra-request parallelism supported, namely control operations related to intra-operation parallelism, those related to inter-operation parallelism and those related to the pipeline.

The implementation of a query results in a program which itself can be represented by a sequencing graph structure. Such a graph is represented in FIG. 2C and comprises two types of nodes, operation nodes mentioned opl, op2, op3, op4, op5, op6 and manufacturer nodes which include a label (START, END, LPIPE, PIPE, SEQ,
LSEQ) which corresponds to the type of sequencing dependency that the constructor node represents.

 A constructor node such as LPIPE or PIPE with n inputs and p outputs which are operation nodes. Each of the operation inputs is executed with the sequencing dependency specified by the constructor.

It is very important to note that a data operation can never be the output of a sequential constructor such as SEQ, LSEQ and a pipeline constructor such as PIPE, LPIPE.

We previously indicated how each operation is divided into localized sub-operations. All these sub-operations need to be synchronized and controlled to guarantee correct overall execution.

Each embodiment described above requires a different control mechanism. In particular, to guarantee a sequential execution between two operations, it must be guaranteed that the second operation is only initialized after the end of the first. This implies the ability to detect the completion of the execution of an operation and the ability to initiate an operation. For the detection of the completion, knowing that an operation is distributed in a number of sub-operations, each located on a node, it is necessary to detect the local completion of each sub-operation and the global completion of the operation . To launch, you must be able to launch all the sub-operations of an operation. This implies the ability to explicitly notify the end of a data stream by a signal (EOS) (End of Stream) at each sub- operation, receipt of this signal then authorizes detection of local completion to occur. To allow operations to detect operations, launch operations and synchronize operations, we have created atomic command operations which are divided into two groups of operations, the operations of wait and spread operations.

Remember that an atomic operation or element cannot be decomposed. It is the smallest component.

Waiting operations wait for end-of-data stream (EOS) messages and then activate a propagator that propagates end-of-data stream (EOS) messages and sometimes initializes messages. Standby operations always consume end of data stream messages and produce activation messages, while propagators consume activation messages and produce end of data flow messages, sometimes with activation. The standby operations are Iw which stands for local standby, is a distributed control operation associated with a distributed operation. Each instance of the local wait performs the detection of the local completion of one of the operation's sub-operations. It waits for an end of flow message (EOS) for the completion of the sub-operation.

C2: act = Iw (Cî: EOS, stream) is equivalent to sending the C2 activation message (act) to the next control operation when the EOS type C1 message is received, signifying that the operation producing the flow ( stream) is complete. Otherwise wait for the end of the operation producing the stream to send C2.

gw which means global wait, is a centralized control operation which waits for the global completion of a given operation, ie the completion of all the sub-operations. Each local completion is notified by an end-of-flow message (EOS) produced by a propagator.

C2: act = gw (Cî: EOS) is equivalent to sending the C2 activation message (act) to the next control operation if the number of C1 EOS type messages received is equal to the number of nodes in the home of the control operation producing Cl messages.

The number of nodes of the home is determined at the initialization of the program for processing the request and memorized by a parameter which is the home of the previous local operation.

mw which stands for multiple wait, is a control operation used for synchronization of the detection of completion. She waits for n messages
EOS, n being the number of operations to be synchronized which can be either centralized or distributed.

C2: act = mw (Cî: EOS) is equivalent to sending the C2 activation message (act) to the next control operation if the number of Cl messages of type EOS is equal to the number of control operations producing the message Cl.

N.B .: the number of control operations producing Cl is> 1.

 sw which means simple wait, waits for an end of flow message (EOS). This operation is mainly used for the conversion of an EOS type message into an activation message. It can be either centralized or distributed.

C2: act = sw (C1: EOS) is equivalent to sending the C2 activation message (act) to the next control operation when the EOS type C1 message is received.

The propagation operations are sp which means simple propagation, is an operation of propagation of an EOS end-of-flow message to a wait operation.

C2: EOS = sp (Cl: act) is equivalent to sending the C2 end of flow message to the next control operation when the activation message (act) is received.

stp which means simple activation and propagation, is a distributed command operation associated with a distributed operation. Each instance of the stp operation performs the local activation of a sub-operation of the operation and propagates an EOS end-of-flow message to its associated wait operation (Iw).

(D2: trigger, C2: EOS) = stp (CA: act) is equivalent to sending the D2 trigger message to the local op consuming the D2 message and the C2 end-of-flow message to the next control operation when the message Activation key (act) is received.

gp which means global propagation operation, is a centralized operation which propagates EOS end-of-flow messages to a waiting operation distributed on each node of the home.

C2: EOS = gp (C1: act) is equivalent to sending the C2 end-of-flow message to all instances of the following control operation when the activation C1 message (act) is received. There are as many messages sent as there are nodes in the home of the next local operation, the number of messages being stored by a parameter determined at the initialization of the program by the cardinal of the home of the next local operation.

mp which means multiple propagation, is used for the propagation of synchronization EOS end-of-flow messages. This operation propagates n EOS messages, n being the number of operations to synchronize. This operation can be either centralized or distributed.

C2: EOS mp (Cî: act) is equivalent to sending the message C2 of end of flow to all the following control operations when the message C1 of activation (act) is received.

N.B .: the number of control operations consuming C2 is> 1.

We will now examine the different operation or constructor control diagrams which are represented in FIGS. 5 to 13. FIG. SA represents an example of an operation graph attached to associated constructor nodes which corresponds to the command operation diagram shown in Figure 5B. In FIG. 5, an operation 1 receives inputs in sequential (SEQ) or local sequential (LSEQ) mode and emits outputs in pipeline (PIPE) or local pipeline (LPIPE) mode. The operation receiving two constructor inputs, the associated control diagram in FIG. 5B uses an SP operation per constructor input, which makes it possible for each constructor input to propagate an EOS message to a wait operation which is constituted by the operation (mw) if the operation has more than one constructor entry. This atomic command operation (mw) then activates either a propagation operation (stp) which triggers the execution of each instance in the suboperations d 'an operation in the case where the constructor inputs are sequential, or launches a simple propagation (sp) shown in dotted fig. 5B in the case where the manufacturer inputs are of the pipeline type (PIPE, LPIPE) shown in dotted lines fig. 5A. One of these two propagation commands sends the EOS message to a local wait command (Iw) which sends an activation message to the next atomic operation which is a multiple propagation command (mp) in case the opi operation has more than one manufacturer output. This multiple propagation operation (mp) is associated with a wait command (sw) for each output branch. In the case where the operation (ope) has a single output, the atomic operation (mp) is replaced, as shown in FIG. 5B in dotted lines by a simple propagation operation (sp) associated with a simple waiting operation (sw).

FIG. 6A represents the diagram of a local pipeline type constructor operation (LPIPE), having as input an operation (opi) and two operations as output (op2, op3).

FIG. 6B represents the diagram of atomic control operations generated by the graphs corresponding to the diagram of FIG. 6A. This diagram is also valid in the case of a local constructor of the sequential type (LSEQ).

The end of an opi operation generates, as we have seen for FIG. 5B, an atomic operation (sw) which activates an atomic operation (mp) with multiple propagation to propagate two EOS messages to the simple wait operations (sw) each associated with the start of the respective op2 and op3 operations.

FIG. 7A represents an operation sequence graph starting with a constructor (START) which launches three operations (opi, op2, op3) of which two opl and op2 are co-local. It will be recalled that a group of colocal operations designates operations which are located on the same "home", that is to say on the same set of nodes (Ni to Nm). These co-local operations are represented in FIG. 7A by the fact that they are surrounded by a dotted line, and are executed by the same processor.

As the start constructor appears at the top of a sequencing graph, this constructor has no operations input. This is why no local input part appears in the control diagram. Figures 7B and 7C represent the translation of this sequencing graph into atomic operations.

FIG. 7B represents the global part of the constructor node, the role of the message sw is to receive the start system message. The multiple propagation operation (mp) duplicates this message if necessary in as many messages as there are groups of co-local operations. In the context of FIG. 7A, there are two groups of co-local operations, the operation op3 and the group of operations opi, op2 which are associated operations.

The atomic operation (mp) is associated with as many atomic operations of simple waiting (sw), as there are co-local operations. Then a global propagation by group (gp) operation is used to distribute the control information to all the nodes of a "home" in the group. Thus a first atomic operation (gp) distributes the control information to the node N1, Nm of the "home" of the opl and op2 operations, while the other atomic operation (gp) distributes the control information to the node Ni,
Nn of the "home" of operation op3.

FIG. 7C represents the local part of the outputs on the nodes of co-local operations (ope, op2). An operation (sw) simply collects locally on each node N1, Nm of the "home" of the group concerned by the operations, the control information sent by the global part of the operation (gp). This control information is duplicated by the multiple propagation mp for all of the group's operations. Each propagation operation (mp) associated with a node (N1, Nm) generates two simple wait operations (sw) on each node for the two operations opi and op2.

Figures 8 to 12 show the control diagram corresponding to two global manufacturers, one (SEQ) in Figure 12, the other (PIPE) in Figure 8 pipe. The only difference between the manufacturer SEQ and the manufacturer
PIPE is that the constructor SEQ implies that the output operations must be activated, but this functionality appears at the level of the operation command diagram. Here the control diagram is made up of three parts: the local input part, the global part and the local output part. The local entry part of Figure 8 has two co-local operations (ope, op2) because surrounded by a dotted line.

The local entry part illustrated in FIG. 9A comprises a single propagate operation for each operation (ope, op2) of a group. This propagation operation (sp) is followed by an operation mw (multiple wait) in the case where the entries of the group of co-located operations has more than one operation, which is represented in the figure iOA where the operations opl and op2 are co-located or in other words associated and therefore each (sp) coming from an operation is expected by the operation (mw).

This operation is followed by a simple propagation (sp) towards the global part represented in FIG. 1 OA. In the case where the operation is distributed over several nodes as represented in FIG. 9B, each node generates after the execution of the operation, for example the operation op3 in FIG. 9B or the operation opî in FIG. 9C a single propagate atomic operation (sp).

The global part is represented for a PIPE operation in FIG. 10A and for an SEQ operation in FIG. 10B. This global part comprises a global gathering carried out by two gathering operations (gw) in the case of FIG. 10A which makes it possible to wait for the EOS signals from the end of the co-localized operations opl, op2 or of the operations op3 distributed over several nodes . In the case where several global gathering operations are used, each global waiting operation (gw) is followed by an sp (single propagate) operation which regroups on a multiple wait operation (mw) (multiple wait) in the case where there is more than one input group for co-located operations. The operation (mw) is followed by a global propagation gp operation associated with each group of co-located operations.

FIG. 10B represents the case of a global part having several graphs of output operations, and an input operation. In this case, the scheme begins with a global gathering operation (gw) which is followed by a multiple propagation operation (mp) which makes it possible to initialize several simple waiting operations (sw), by group of exit operations.

Each simple wait (sw) is followed by a global propagation operation (gp) per group of co-located operations which allows the initialization of the operations of each group.

The local output part of the control diagram is represented in FIG. 8 for several groups of operation in outputs. This diagram includes following the global propagation operation (gp), a simple gathering operation (sw), followed by a multiple propagation operation (mp) which allows to initialize a simple waiting operation (sw ) by operation of the group of co-located operations.

 For the other operations (op2, figure 1 1 C, or op4, figure il A) the local part of exit is constituted by an atomic operation of simple waiting (sw).

The last element consists of the constructor element FIN, the diagram of which is shown in fig. 13A and the corresponding commands shown in fig. 13B include a global gathering operation (gw) per entry group of co-located operations. Each operation (gw) activates a multiple gathering operation (mw) which begins with a single propagation operation (sp) associated with each (gw). The various operations (sp) activate a multiple wait operation (mw) which itself activates the simple propagation operation (sp) launching the end of processing operation by the end of flow message (EOS). , thanks to the control diagrams corresponding to the various FIGS. 5 to 13 and to the control operation tables corresponding to the commands associated with these graphs, we arrive in the case of a sequencing graph such as that represented in FIG. 2C, translate this sequencing graph into a succession of command operations which are represented in FIG. 2D and which consist of a grouping of the various elementary command operations which we have seen previously.

The start node, which is the first node in the program, is the first node to be processed. This processing uses the table which stores the global schema of the global constructors and generates an operation (sw) for the input of the constructor and then an operation (mp), since the constructor has two output operations which will not be co- localized, then, an operation (sw) followed by an operation (gp) are produced for each output operation and associated with the operation (mp). All the command operations produced at this time are global, the table corresponding to the local diagram of output of the global constructors applied to the output of the manufacturer START then indicates that an operation (sw) for each operation co-located with this operation is produced and associated with the corresponding operation (gp). The next node to be processed is the opl operation node. For this operation which has a manufacturer input START and manufacturer output LPIPE, an operation (stp) is then generated when processing the start of the operation (opl) This operation (stp) is associated with the operation (sw) previously generated. Processing the operation output (ope) produces an operation (Iw). All the operations generated are co-local with the opi operation.

The context of the op2 operation being exactly the same as the opl operation, with simply the replacement of a PIPE constructor by an LPIPE constructor, the command operations generated are identical.

The LPIPE constructor node having an input and an output operation.

Thus, no command operation is to be added because the only commands required by the local manufacturers concern synchronization.

The processing of the PIPE node which also has an input and an output operation generates an operation (sp) followed by two global operations (gw) and (gp) which respectively ensure the detection of the overall completion of the op2 operation on the different nodes (N1 to Nn) and the global propagation to the op3 operation on the nodes (N1 to Nm). An operation (sw) co-local with the operation op3 is then generated during the output processing of the PIPE constructor.

The next operation is the op3 operation which has two pipeline constructors as input and a sequential constructor output (SEQ). EOS information from the two pipeline manufacturers needs to be synchronized. This synchronization is ensured by the two operations (sp), one for each manufacturer followed by an operation (mw) and an operation (sp) introduced during the processing of the global part using the global table. For the exit from the processing of the op3 operation, an operation (Iw) is generated.

Finally, it will be noted that all of the command operations introduced in this phase are co-local with the op3 operation and therefore distributed over the same association. The constructor SEQ has a co-local input and two co-local output operations (op4, op5). This means that multiple propagation will occur. For entry, an operation (sp) is introduced to propagate the local completion of operation op3. The global operations corresponding to the constructor SEQ are introduced using the global table. These operations are respectively an operation (gw) which detects the completion of the op3 operation followed by a global operation (gp) (the SEQ exit operations belong to the same group of co-local operations). of the manufacturer then generates the distributed command operations relating to the manufacturer's output operations and co-local with these operations. They consist of operations (sw) and (mp) followed by an operation (sw) for each of the op4 and op5 operations.

The processing of op4 and op5 operations generates the same command operations, knowing that each of these operations has a manufacturer input SEQ and a manufacturer output LSEQ. These co-local command operations with the op4 and op5 operations are a first operation (stp) and then an operation (Iw) for each of the op4 and op5 operations.

The LSEQ constructor has two input operations and one output operation.

The two input operations must each be synchronized by an operation (sp) and an operation (mu). No command for exit is required.

Operation op6 is scheduled for input with an LSEQ constructor and for output with a FIN constructor. Processing the input to this node introduces an operation (stp) and the output an operation (il). Finally the FIN constructor with the help of the input tables and the global constructor completes the command graph with a distributed operation (sp) relating to the op6 operation and the centralized operations (gw) and (sp).

This method of parallel execution of basic request easily makes it possible, using a sequencing graph and tables translating to each of the constructor nodes or of the operation nodes, to transform this sequencing graph into an instruction sequence allowing the execution of a query on parallel processes.

So we have seen how, starting from a request represented by a graph, transform this request into a succession of atomic control operations allowing the parallel execution of this request in a multi-processor environment using a parallel language. lera (language for extended relational algebra). This process will be used in a request manager which compiles requests formulated in ESQL language (Extended queries language) in a version of generated code of parallel LERA language.

This compilation is carried out in several successive phases as shown in FIG. 14 which are respectively the analysis, the optimization, the parallelization and the generation of the code. The results of these various transformations carried out by these compilation steps are all expressed in different variants of the same intermediate language (LERA).

 The analysis of the ESQL query generates a LERA-SEM program (SEM for semantics) which is then optimized by a physical optimizer into a LERA-PHY program (PHY for physics). The result of the optimization is a query execution plan which minimizes a cost function expressed in terms of the environmental characteristics of the parallel execution. The execution plan described in LERA consists of simple atomic operations for which the localization operations and the global and local algorithms used are expressed by annotations. This is formulated as graphs. The parallelization step then compiles the execution plan so as to generate a parallel execution program where the data communications and the control processes necessary for the sequencing of the program are explained. A phase of code generation then transforms the LERA-PAR program in C ++ code which is compiled by a standard C ++ compiler. The resulting modules are either stored in a catalog for later execution, or directly executed by the data manager.

Thus, if we take again the example of the request appearing in appendix 1, which can generate by the physical optimizer an execution plan corresponding to annex 2, this execution plan will be expressed under the form of a graph represented in FIG. 1 in which the associations and the local operations are represented as well as the global pipelines and the local algorithms corresponding to the request. To this graph, the tables which correspond to each of the elementary graphs represented in FIGS. SA to 13 and translating these graphs in the form of sequence of atomic communication and command operations will allow the translation of the graph into different sequences of atomic operations which will be each addressed to the processor of the nodes corresponding to the processing to be carried out. This makes it possible to generate the succession of atomic operations appearing in appendix 3. The program which incorporates both the operations corresponding to the execution of the request and the atomic control operations which allow a parallel execution of this request with the desired sequencing.

Other modifications within the reach of the skilled person are also part of the spirit of the invention.

ANNEX 1
SELECT PURCHASE.n veh FROM PURCHASE, CAR
WHERE PURCHASED. Date>'010186'
and PURCHASE.n veh = CAR.n veh
and VOlTURE.marque = "Renault"
and VOlTURE.n veh not in
select Cn) veh from CONTROLE C where C.date>'010186'
APPENDIX 2
Select vehicles checked since 01/01/86
Il = select #home (CONTROLE), allselect, scanselect}
(CHECK C, C. date>'010186", (Cn veh))
Select vehicles purchased since 01/01/86 12 = select {home (CAR, 01/01/86 PURCHASED), allselect, scanselect}
(PURCHASED, PURCHASED.date>'010186', (PURCHASED.n veh))
Select Renault 13 vehicles = select {home (CAR, PURCHASED), allselect, scanselect}
(CAR, CAR. Brand = "Renault", (CAR.nve))
Select your Renauit vehicles not checked since 01/01/86 by difference in intermediate results 14 = difference {home (CAR, PURCHASED), assocdiff (| 1.n veh), scandiff}
(@ 3 {| pipe ~ int}, @ 1 (materialized})
Join the result with the result of the selection on Purchase 15 = join {home (CAR, PURCHASED), assocjoin (14.n veh), nestedloop}
(12 {materialized}, 14 {| pipe ~ ext}, 12 .n veh = 14.n veh, (12.n veh))
APPENDIX 3 {Spread (l)} ASSOC
C1 = sw (START)
C2 = mp (C1)
C23 = sw (C2)
C3 = gp (C23)
C24 = sw (C2)
C4 = gp (C24)
End ~ ASSOC {home (CONTROL)}
ASSOC
C5 = sw (C4)
(T1, C6) = stp (C5)
Local ~ op (T1)
I1 = select {scanselect} (CONTROL C, C. date>'010186". (Cn veh))
12 = send (ll, home (PURCHASED), distribute ~ on I1.n veh)
End ~ Local ~ op
C7 = Iw (C6,12)
C8 = sp (C7)
End ~ ASSOC {spread (1)}
ASSOC
C9 = gw (C8)
C10 = gp (C9)
End ~ ASSOC {home (PURCHASED, CAR)}
ASSOC
C11 = sw (C3)
(T2, C12) = stp (C11)
Local ~ op (T2)
13 = select {scanselect} (PURCHASED, PURCHASED.date>'010186", (PURCHASED, n veh))
S2 = store (l3)
End ~ Local ~ op
C13 = Iw (C12, S2)
C14 = sp (Cl 3)
Local ~ op (12)
If = store (12)
End ~ Local ~ op
C15 = | w (C10, S1)
C14 = sp (C15)
APPENDIX 3 (continued)
C15 = Iw (C10, S1)
C14 = sp (C15)
C16 = mw (C14)
(T3, C17) = stp (C16)
Local ~ op (T3)
14 = select {scanselect}
(CAR, CAR.brand = 'Renault ", (CAR, n veh))
15 = difference (14, S1)
16 = local ~ send (15)
End ~ Local ~ op
C18 = Iw (C17.16)
C19 = sp (C18)
Loca ~ op (16)
Result = join {nestedloop} (S2, # 6, # 6.n veh = S2.n veh, (# 6.n veh))
End ~ Loca ~ op
C20 = #w (C19, Result)
C21 = sp (C20)
End ~ ASSOC {spread (1)}
ASSOC
C22 = gw (C21)
END = sp (C22)
End ~ ASSOC

Claims (15)

CLAIMS: 1. Method for parallel execution of a query of relational databases in a parallel multi-processor environment with distributed memory consisting of integrating the control into the request during its compilation, the control being obtained by combining a set of atomic operations (Iw, sw, gw, stp, sp, gp, mp) control to materialize a control graph which is superimposed on the query graph, this graph expressing the dependencies between the data by the links between the nodes (11, 12 , 3, 16) of operations and types of sequencing dependency by constructor nodes of two types (SEQ, PIPE). 2. Method according to claim 1 characterized in that the control graph expresses the grouping of the nodes in focus (home, buy, car) of a relation of the base, the co-localization of several operations on the same node (select, store , figure 3) of the multi processor architecture, the meeting of several operations (selected store, selected diff local transmit) in local operation (loc ~ op4, loc ~ opl) and the meeting of several local operations (loc ~ op1, loc ~ op2, loc ~ op4) in association (A3). 3. Method according to claim 1 or 2 characterized in that the first type of constructor node (PIPE) allows the execution of a following operation as soon as the previous operation produces a tuple. 4. Method according to claim 2 characterized in that the second type of constructor node (SEQ) allows the execution of the following operation only after the complete completion of the previous operation. 5. Method according to claim 3 or 4 characterized in that the synchronization between operations executed on groups of different nodes and operations executed on the same group is performed by two families of control primitives allowing the establishment of a control mechanism. 6. Method according to claim 5 characterized in that a first family consists of propagation primitives (stp, gp, mp, sp), the other family of waiting primitives (Iw, gw, mw, sw), the control mechanism which can only be constituted by the connection of two primitives of opposite type (wait, propagation). 7. Method according to claim 6 characterized in that the waiting primitives send to the next control operation which is a propagation primitive, an activation message when at least one end of flow type message (EOS) is received. 8. Method according to claim 7 characterized in that a multiple waiting primitive verifies that the number of end-of-flow messages received is equal to the number of local operations executed by the same node and producing the end-of-flow message. 9. Method according to claim 7 characterized in that a global waiting primitive (gw) verifies that the number of end-of-flow messages received is equal to the number of nodes in the home (home) on which one or more operations are run in parallel. 10. Method according to claim 6, characterized in that the propagation primitives send an end-of-flow message to at least one instance of the following control operation when the activation message is received. 11. Method according to claim 10 characterized in that a multiple propagation primitive (mp) sends as many end-of-flow messages as the following local operation. 12 Method according to claim 10 characterized in that a global propagation primitive (gp) sends as many end-of-flow messages as there are nodes of the home on which one or more local operations is executed following (s). 13. The method of claim 10 characterized in that a propagation and trigger primitive (stp) further sends a trigger message to the next local operation, to trigger it. 14. Method according to claim 5 characterized in that program portion tables formed from the primitives and corresponding to the sequence of control primitive describing pieces of graph formed from operation nodes associated with the constructor node make it possible to reconstruct by l successive association of several tables the graph corresponding to the parallel execution of a query on a relational database in a multi-processor environment with distributed memory.  15. Use of the method according to one of the preceding claims, characterized in that it is carried out in the parallelization phase of a request manager responsible for compiling the requests expressed in E.S.Q.L. by executing an analysis phase, a physical optimization phase, a parallelization phase and a code generation phase in C ++ language.