JP2013232165A - Query integration method, query integration program, and integrated component creation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the number of components for processing execution to be created in creating the components by understanding queries with a flow definition.SOLUTION: A first integrated query creation unit 11 reads a flow definition 2 that defines processing and order of the processing, and when target processing and subsequent processing are of the same processing type and results are fixed due to a change in order of the pieces of processing, couples a phrase of a query in the target processing with a phrase of a query of the subsequent processing to create an integrated query. A second integrated query creation unit 13 incorporates processing that is not performed by the first integrated query creation unit 11 into a query of the subsequent processing as a sub-query to create an integrated query. An integrated component creation unit 20 creates components from the integrated query.

Description

  The present invention relates to a process for generating a query based on a flow definition, and more particularly to a query integration process for generating an integrated query in which a plurality of queries are integrated.

  A development execution environment for constructing a data processing system by combining components is known.

  Flow definition editing that creates a flow definition that defines the processing content and processing order from the input information when the user inputs information indicating how and what data is processed in what order. There is a system.

  Furthermore, there is a component generation system that generates a component that executes a query corresponding to each process from a flow definition.

  FIG. 36 is a diagram for explaining the relationship between the flow definition, the query, and the component in the conventional method. In the flow definition shown in FIG. 36, a scheme of target data (information such as a processing target field and its attribute) and a processing content (procedure) are defined for each processing.

  In the component generation system, a processing module (processing engine) corresponding to the processing type of each process defined in the flow definition is selected, a query corresponding to each process content is generated, and the query process is executed. A component including the module to be generated is generated, and further, data passing between the components is defined based on the processing order defined in the flow definition.

US Patent Application Publication No. 2009/0299986 JP 2006-107474 A JP 2011-118492 A

  In a conventional component generation system, a component is generated for each query that has a one-to-one correspondence with each process defined in the flow definition. Each time data is transferred between components, file writing / reading, communication via the network, etc. occur, so the number of data transfers increases according to the number of components, and communication takes time. It was. In particular, in the case of a flow definition of a large-scale system, the problem that the number of components to be generated becomes enormous and communication overhead becomes large cannot be ignored.

  The present invention provides a query integration method, program, and apparatus capable of integrating queries corresponding to processes in a flow definition as much as possible, reducing the number of generated queries and components, and reducing communication overhead between components. With the goal.

  In the query integration method disclosed as one aspect of the present invention, a computer reads a plurality of processes in which processing contents and attributes are defined and a flow definition in which the processing order of the plurality of processes is defined, and is defined by the flow definition. A first query corresponding to the target process and a second query corresponding to a subsequent process of the same processing type as the target process processed immediately after the target process. Is generated, and a process is repeated in which the integrated query is used as a first query corresponding to the target process.

  According to the disclosed query integration method, the number of queries generated from the flow definition can be reduced, the number of components corresponding to the query can be greatly reduced, and communication overhead can be reduced.

It is a figure which shows the example of a relationship between the process defined in the flow definition, an integrated query, and a component in the query integration method to disclose. It is a figure which shows the example of the integrated query produced | generated in a 1st integrated query production | generation process. It is a figure which shows the example of an integrated query based on the query language produced | generated in a 2nd integrated query production | generation process. It is a figure which shows the block structural example in one Example of the query integrated device to disclose. It is a figure which shows the structural example of the flow definition which a 1st integrated query production | generation part acquires. It is a figure which shows the example of a judgment condition of the possibility of integration in the integrated query production | generation process by phrase division | segmentation. It is a figure which shows the example of the calculation type of the process of a flow definition. It is a figure which shows the data definition example of a flow definition. It is a figure which shows the process definition example of a flow definition. It is a figure which shows a phrase template example. It is a figure which shows a phrase template example. It is a figure which shows a phrase template example. It is a figure which shows a phrase template example. It is a figure which shows the example of an outline processing flow of an integrated component production | generation apparatus. It is a figure which shows the more detailed process flow example of the integrated query production | generation process (step S2) by phrase division | segmentation. It is a figure which shows the example of determination whether there exists subsequent processing of object processing. It is a figure which shows the example of each phrase produced | generated. It is a figure which shows the example of each phrase and arithmetic type preserve | saved at the production | generation query memory | storage part. It is a figure which shows the example of each phrase produced | generated. It is a figure which shows the example of each phrase and arithmetic type preserve | saved at the production | generation query memory | storage part. It is a figure which shows the example of each phrase and arithmetic type preserve | saved at the production | generation query memory | storage part. It is a figure which shows the example of the produced | generated query. It is a figure which shows the example of a more detailed process flow of an integration possibility determination process (step S24). It is a figure which shows the example of a more detailed process flow of a calculation type | mold calculation process (step S243). It is a figure which shows the example of calculation type | mold determination. It is a figure which shows the example of calculation type | mold determination. It is a figure which shows the example of calculation type | mold determination. It is a figure which shows the more detailed process flow example of the integrated query production | generation process (step S3) by nesting. It is a figure which shows the example of the integrated query produced | generated. It is a figure which shows the example of the query preserve | saved at the production | generation query memory | storage part. It is a figure which shows the example of a more detailed process flow of a component production | generation process (step S4). It is a figure which shows the example of a setting of the component template corresponding to a query language. It is a figure which shows the example of a component template. It is a figure which shows the example of the integrated component produced | generated. It is a figure which shows the hardware structural example of an integrated component production | generation apparatus. It is a figure for demonstrating the relationship between the flow definition in a conventional method, a query, and a component.

  Hereinafter, a query integration method disclosed as one aspect of the present invention will be described.

  In the disclosed query integration method, when a query is generated from each process defined in the flow definition, a query (referred to as a first query) corresponding to a certain process (referred to as a target process) and a process immediately thereafter (subsequent process) And the query corresponding to the query (referred to as the second query) are generated as much as possible, the number of generated queries is reduced, and the number of components corresponding to the query is reduced.

  More specifically, it is examined whether or not the calculation result is changed even when the target process and the subsequent process are executed by changing the processing order. For example, whether or not there is an order dependency between the target process and the subsequent process is checked from the correspondence between the operation types based on the relational algebra. When both processes are operation types that do not change the operation result even if they are executed after changing the processing order, the first query and the second query are divided into phrase units, and elements are connected for each divided phrase. Generate a consolidated query. Then, the generated integrated query is set as the first query, the process in which the queries are integrated is set as the target process, and the generation of the integrated query is repeated with the subsequent process (the process subsequent to the current subsequent process) (the first query). 1 integrated query generation process).

  In the case of a process that cannot be integrated by the first integrated query generation process during the flow definition, for example, when the target process and the subsequent process are executed by changing the processing order, the calculation result changes or the subsequent process cannot be executed. In such a case, an integrated query is generated by incorporating the first query as a subquery into the second query (second integrated query generation process).

  FIG. 1 is a diagram illustrating an example of a relationship between processes, integration queries, and components defined in a flow definition.

  The flow definition shown in FIG. 1 is information in which a data definition and a process definition of data to be processed in each process and a process order are defined. Here, as processing for POS data of the POS system, processing P1 (date-to-time conversion processing), processing P2 (value range specification (time) processing), processing P3 (field selection), processing P4 (commodity category) Assume that a plurality of processes are defined, such as a join (join) process.

[First integrated query generation processing (integrated query generation processing by phrase division)]
In the disclosed query integration method, first, as a first integrated query generation process, the first process defined in the flow definition is set as a target process, and an integrated query by phrase division is generated.

  In order to generate an integrated query from the flow definition, a template for each phrase constituting the query is set in advance, and the integrated query is generated with reference to the phrase template. The phrase template will be described later.

  In the example of the flow definition shown in FIG. 1, when the target process is the process P1, the process P2 of the subsequent process matches the process type (real time), but the process P2 cannot be processed when the process order is changed. Become. Therefore, the query “SELECT... UDF.getTime (date and time) AS time FROM POS data” corresponding to the process A is generated without performing the first integrated query generation process, and the next process P2 is processed as the target process. I do.

  When the process P2 of the target process and the process P3 of the subsequent process are operation types whose execution order can be changed, the integrated query generation process by phrase division is executed for the processes P1 and P2.

  Specifically, as shown in FIG. 2, the first query corresponding to the process P2 is “SELECT... FROM data A WHERE 18:00 <= time AND time <= 20: 00”. When the corresponding second query is “SELECT ID, product ID, time FROM data B”, the query corresponding to each of the processes P1 and P2 is divided into constituent phrase units (SELECT clause, FROM clause, WHERE clause). Then, for each divided phrase, the phrases obtained by combining the corresponding phrase elements of the processes P1 and P2 are connected to obtain the integrated query “SELECT ID, product ID, time FROM data A WHERE 18:00 <= time AND time <= 20:00 "is generated.

  The next subsequent process P4 is an operation type in which the process result is different when the process order is changed from the integrated process (processes P2 and P3). Therefore, the query “SELECT... FROM data C JOIN merchandise category ON...” Corresponding to the process P4 is generated without performing the integrated query generation process.

  In this way, the integrated query generation process by phrase division is repeated while processes of the same process type continue.

[Second integrated query generation processing (integrated query generation processing by nesting)]
Next, as a second integrated query generation process, a nested integrated query is generated from the flow definition process P1.

  Specifically, an integrated query “SELECT ID” in which a query (first query) corresponding to the process P1 is incorporated as a subquery into an integrated query (second query) of the integrated process (P2 + P3) that is a subsequent process of the process P1. , Product ID, time FROM (SELECT... UDF.getTime (date and time) AS time FROM POS data) WHERE 18:00 <= time AND time <= 20: 00 "is generated.

  Further, an integrated query “SELECT... FROM (SELECT ID, product ID, product ID, product ID, product ID,. Time FROM (SELECT... UDF.getTime (date and time) AS time FROM POS data) WHERE 18:00 <= time AND time <= 20: 00) JOIN Product category ON ... is generated.

  By the way, the description rule for subquery insertion differs depending on the query language. Therefore, in the second integrated query generation process, an integrated query is generated using the subquery insertion rule corresponding to the query language. For example, when processing is performed by a query processing engine corresponding to a query language (HiveQL) that has a batch type process, an integrated query as shown in FIG. When the process is processed by a query processing engine corresponding to another query language (EsperEPL), an integrated query as shown in FIG. 3B conforming to the other query language is generated.

  As described above, according to the conventional method, four components are generated from four queries, whereas according to the disclosed query integration method, one integrated component for one integrated query obtained by integrating four queries. Will be generated. Therefore, since the number of components conventionally generated for each query is greatly reduced, the processing time and processing load can be greatly reduced by reducing the number of communication between components.

  Hereinafter, a query integration device disclosed as another aspect of the present invention will be described.

  FIG. 4 is a diagram illustrating a block configuration example in an embodiment of the disclosed query integration device. In the present embodiment, the query integration device 10 according to the present invention is configured in the integrated component generation device 1.

  The integrated component generation device 1 includes a query integration device 10, an integrated component generation unit 20, and a component template storage unit (component template repository) 21.

  The query integration device 10 includes a first integration query generation unit 11, a second integration query generation unit 13, a phrase division template storage unit (phrase division template repository) 15, and a generation query storage unit 17.

  The first integrated query generation unit 11 of the query integration device 10 refers to the phrase division template storage unit 15 and corresponds to the first query of the process (target process) defined in the acquired flow definition 2 and the subsequent process. For two queries, an integrated query by phrase division is generated.

  The first integrated query generation unit 11 generates an integrated query when the processing result (output data) does not change by changing the processing order of the target process and the subsequent process, and does not generate an integrated query when the processing result changes.

  In the present embodiment, the first integrated query generation unit 11 integrates, for example, a process of combining a plurality of inputs, a process of summarizing input data (average value calculation, data count, etc.) with the processes before and after that. And do not generate consolidated queries. Furthermore, the process of adding a certain field, the process of changing the value of a certain field, and the like are integrated with subsequent processes that perform calculations and comparisons with reference to the value of the field, and do not generate an integrated query.

  FIG. 5 is a diagram illustrating a configuration example of a flow definition input by the first integrated query generation unit 11.

  In the flow definition 2, as shown in FIG. 5A, a processing order of a plurality of processes P1, P2, P3,... For POS data is defined. Furthermore, in the flow definition 2, a schema (field name and data type) of each data to be processed as shown in FIG. 5B is defined as a data definition. Furthermore, in the flow definition 2, as a process definition, a process type, a process type and name, input data, output data, and process properties (variable attribute names, values, etc.) as shown in FIG. 5C are defined. ing.

  FIG. 6 is a diagram illustrating an example of a condition for determining whether or not integration is possible in the integrated query generation process based on phrase division.

  The data table shown in FIG. 6 is held in the first integrated query generation unit 11 in advance, and whether or not an integrated query can be generated by phrase division is set based on the calculation types of the target process and the subsequent process. ing. In the example of the data table shown in FIG. 6, the relationship that the target process and the subsequent process can be integrated is indicated by a circle (◯), the relationship that may not be integrated is indicated by a triangle (Δ), and the relationship that cannot be integrated is indicated by a cross. (X).

  FIG. 7 is a diagram illustrating an example of a calculation type of the flow definition process.

  “Extended” is a process of adding a new field to the input data and outputting it. For example, a field for storing the time converted from the date and time is added to the output data as in process P1 “date and time conversion”. Processing is applicable. “Selection” is a process of filtering input data under some condition, for example, a process of narrowing down input data by the condition of the WHERE clause, such as process P2 “value range designation”. “Projection” is a process of selecting only some fields of input data, and corresponds to a process of generating output data including only a specified field from input data, for example, process P3 “field selection”.

  Further, although not shown in FIG. 7, “summary” is a process of performing aggregation on the entire input data, and includes, for example, processes such as counting the number of cases, summation, and average. “Multiple inputs” is a process of combining a plurality of input data to generate one output data. “Multiple output” is processing in which one output data is referred to by a plurality of subsequent processes.

  The second integrated query generation unit 13 generates an integrated query by incorporating the first query corresponding to the target process as a subquery into the second query corresponding to the subsequent process.

  The phrase division template storage unit 15 stores a phrase template of each phrase constituting the query.

  The generated query storage unit 17 stores the queries generated by the first integrated query generating unit 11 and the second integrated query generating unit 13.

  FIG. 8 is a diagram illustrating a data definition example of the flow definition. The data definitions of POS data, data A, data B, and data C are shown from the top of the block shown in FIG. In the data definition, the data ID, data name, field name, and type (data type) of the data to be processed are defined.

  FIG. 9 is a diagram illustrating a process definition example of the flow definition. The process definitions of processes P1, P2, and P3 are shown from the top of the block shown in FIG. In the process definition, the process name, process type, process type, input data data ID (input data ID), output data data ID (output data ID), attribute name, and attribute name value are defined.

  10 to 13 are diagrams showing examples of phrase templates.

  A phrase template is prepared for each query language. For example, the phrase division template storage unit 15 stores phrase templates corresponding to two types of query languages, the query language EPL (Event Processing Language) and the HighQL, and the EPL phrase template is processed in real time. In addition, the phrase template of HiveQL is applied when the processing type is batch processing.

  FIG. 10 is an example of each phrase template of a query corresponding to the process of converting date and time into time. FIG. 10A is an example of a SELECT phrase for EPL, a phrase template of a FROM phrase, and FIG. It is a phrase template example of a SELECT phrase for FROMQL and a FROM phrase.

  FIG. 11 is an example of each phrase template of a query corresponding to a process for specifying a value range. FIG. 11A is an example of a phrase template for an EPL SELECT phrase, FROM phrase, and WHERE phrase, and FIG. 11B. Is an example of a phrase template for a SELECT phrase, a FROM phrase, and a WHERE phrase for HighQL.

  FIG. 12 is an example of each phrase template of a query corresponding to the process of selecting a field, FIG. 12A is an example of a SELECT phrase for EPL, a phrase template of a FROM phrase, and FIG. 12B is an example for HighQL. This is a phrase template example of a SELECT phrase and a FROM phrase.

  FIG. 13 is an example of a table showing the correspondence between the processing type and processing name and the applied phrase template. When the number of processing types and phrases increases, a column can be added to the table shown in FIG. 13 to register a template.

  Hereinafter, the operation of the integrated component generation apparatus 1 will be described.

  FIG. 14 is a diagram illustrating an example of an overview processing flow of the integrated component generation apparatus 1.

  The first integrated query generation unit 11 of the query integration device 10 provided in the integrated component generation device 1 acquires the flow definition 2, and adds a flag indicating that no query is generated to each process of the flow definition 2 (step S1). . Then, the first integrated query generation unit 11 performs an integrated query generation process by phrase division with reference to the phrase division template storage unit 15 and stores all the queries including the generated integrated query in the generation query storage unit 17 ( Step S2).

  Next, the second integrated query generation unit 13 performs an integrated query generation process by nesting the queries stored in the generated query storage unit 17, and stores all queries including the generated integrated query in the generated query storage unit 17. Save (step S3).

  The integrated component generation unit 20 refers to the component template storage unit 21, generates a component for each query stored in the generation query storage unit 17, and outputs the generated component 3 (step S4).

  FIG. 15 is a diagram illustrating a more detailed processing flow example of the integrated query generation processing (step S2) by phrase division.

  The first integrated query generation unit 11 determines from the top of the flow definition 2 whether there is a query non-generated process based on the flag (step S21), and if there is a query non-generated process (step S21). Y), a process for which a query has not been generated is selected from the flow definition 2 as a target process (step S22).

  The first integrated query generation unit 11 determines whether there is a subsequent process of the target process (step S23). Specifically, as illustrated in FIG. 16, the first integrated query generation unit 11 searches the flow definition 2 for a process having input data having the same ID as the output data of the target process. It is determined that there is a process.

  If there is a subsequent process in the target process (Y in step S23), the first integrated query generation unit 11 determines whether or not the target process and the subsequent process can be integrated (step S24). Details of the processing in step S24 will be described later.

  As a result of the process in step S24, if the target process and the subsequent process can be integrated (Y in step S25), the first integrated query generation unit 11 combines the elements for each phrase of the target process and the subsequent process. Each phrase of the integrated query is generated and stored as each phrase of the target process in the generated query storage unit 17 (step S26). Specifically, the first integrated query generation unit 11 refers to the target process of the flow definition 2 and the data definition of the subsequent process, and uses the phrase template corresponding to the process name from the phrase division template storage unit 15 to specify each phrase. Generate. When the data table of FIG. 13 is referenced and the target process is process P2 and the subsequent process is process P3, the SELECT phrase phrase template T11 (process P3), the FROM phrase phrase template T6 (process P2), and the WHERE phrase Phrase template T7 (process P2) is designated. Then, as shown in FIG. 17, each phrase (SELECT phrase q4, FROM phrase q5, WHERE phrase q6) in which the elements of the target process and the subsequent process are combined is generated for each phrase.

  The first integrated query generation unit 11 stores the sum of the calculation type of the target process and the subsequent process in the generation query storage unit 17 as the calculation type of the target process (step S27). Further, the first integrated query generation unit 11 deletes the subsequent process from the flow definition 2 (step S28).

  As illustrated in FIG. 18, when the target process is the process P2 and the subsequent process is the process P3, the generated query storage unit 17 stores the generated phrases q4, q5, q6, and an operation as the target process phrase. The type “selection + projection” is stored. Also, the diagonal line of the process name in FIG. 18 indicates that the process has been deleted.

  If the target process and the subsequent process cannot be integrated as a result of the process in step S24 (N in step S25), the first integrated query generation unit 11 generates each phrase of the target process query, The operation type is stored in the generated query storage unit 17 (step S29).

  When the target process is the process P1 and cannot be integrated with the subsequent process, the phrase template T1 (process P1) of the SELECT phrase and the phrase template T2 (process P1) of the FROM phrase are designated, and as shown in FIG. Each phrase (SELECT phrase q1, FROM phrase q2) is generated using the target processing elements as they are. As illustrated in FIG. 20, the generated query storage unit 17 stores the generated phrases q1 and q2 and the operation type “expansion” as target processing phrases.

  If there is no subsequent process in the target process as a result of the process in step S23 (N in step S23), the first integrated query generation unit 11 generates each phrase of the query of the target process, and the generated query storage unit 17 (Step S29). The first integrated query generation unit 11 combines each phrase of each process stored in the generation query storage unit 17 and stores it in the generation query storage unit 17 as a query of the process (step S210). As shown in FIG. 21, a query Q10 corresponding to the process P1 is generated, and a query Q11 corresponding to the process P2 (P2 + P3) is generated.

  FIG. 22A shows an example of the query Q10 for the process P1, and FIG. 22B shows an example of the query Q11 for the process P2.

  FIG. 23 is a diagram showing a more detailed processing flow example of the integration possibility determination processing (step S24).

  The first integrated query generation unit 11 refers to the target process and the subsequent process in the flow definition 2 (step S241), and the process types of the target process and the subsequent process are different, or there are a plurality of subsequent processes of the target process. It is determined whether to do so (step S242).

  When the processing types of the target process and the subsequent process are the same, and there are not a plurality of subsequent processes of the target process (N in step S242), the calculation types of the target process and the subsequent process are calculated (step S243).

  As a result of the process of step S243, when the operation type of the target process or the subsequent process does not include an operation type other than expansion, selection, or projection (N in step S244), the operation type of the target process does not include an extension. In this case (N in Step S245), the first integrated query generation unit 11 determines that integration is possible (Step S246). On the other hand, when the operation type of the target process includes an extension (Y in step S245), the first integrated query generation unit 11 further determines whether the operation type of the subsequent process includes an extension or selection (step S247). When the operation type of the subsequent process does not include expansion or selection (N in Step S247), the first integrated query generation unit 11 determines that the integration is possible (Step S246), and the operation type of the subsequent process expands or selects. If it is included (Y in step S247), it is further determined whether the field added in the target process is referenced in the subsequent process (step S248). If the field added in the target process is not referred to in the subsequent process (N in Step S248), the first integrated query generation unit 11 determines that the integration is possible (Step S246). If the field added in the target process is referred to in the subsequent process (Y in step S248), the first integrated query generation unit 11 determines that the integration is impossible (step S249).

  In the process of step S242, when the process type of the target process is different from that of the subsequent process, or there are a plurality of subsequent processes of the target process (Y in step S242), or the target process or the subsequent process is performed in the process of step S244 If the calculation type includes calculation types other than expansion, selection, and projection (Y in step S244), the first integrated query generation unit 11 determines that integration is not possible (step S249).

  FIG. 24 is a diagram illustrating a more detailed processing flow example of the arithmetic calculation processing (step S243).

  The first integrated query generation unit 11 determines whether the calculation type of the target process has been registered in the generation query storage unit 17 (step S431). If the calculation type has not been registered (N in step S431), The calculation type is empty (Φ) (step S432), and the schema of the input data and output data of the target process is acquired from the flow definition 2 (step S433).

  The first integrated query generation unit 11 determines whether there is a field not included in the input data in the output data (step S434). If the output data includes a field that is not in the input data (Y in step S434), the first integrated query generation unit 11 adds an extension to the operation type of the generation query storage unit (step S435), and adds the input data to the input data. If no field exists (N in Step S434), the process proceeds to Step S436.

  For example, in the process P1 (date-to-time conversion) shown in FIG. 25, since the new field “time” defined in the process definition is added to the output data, the operation type of the process P1 is determined to be extended.

  The first integrated query generation unit 11 determines whether all fields of the input data are included in the output data (step S436). If all the fields of the input data are not complete (N in step S436), the first integrated query generation unit 11 adds a projection to the operation type of the generation query storage unit (step S437), and all the fields of the input data are complete. If so (Y in step S436), the process proceeds to step S438.

  For example, in the process P3 (field selection) shown in FIG. 26, the fields of the input data and the output data are not aligned, and the output data does not include all the fields of the input data (mismatch (decrease)). The operation type of is determined to be projection.

  The first integrated query generation unit 11 determines whether a phrase template for the WHERE phrase exists in the target process (step S438). If the phrase template of the WHERE phrase exists in the target process (Y in step S438), the first integrated query generation unit 11 adds a selection to the operation type of the generation query storage unit (step S439), and the WHERE phrase in the target process If the phrase template does not exist (N in step S438), the process proceeds to step S4310.

  For example, in the process P2 (value range specification) shown in FIG. 27, the input data field and the output data field match, and the phrase template of the WHERE clause applied to the process exists, so the operation type of the process P2 is determined to be selection. Is done.

  The first integrated query generation unit 11 determines whether a phrase template other than SELECT, FROM, and WHERE exists (step S4310). If a phrase template other than SELECT, FROM, and WHERE exists in the target process (Y in step S4310), the first integrated query generation unit 11 adds others to the operation type of the generated query storage unit (step S4311), and the target If there is no phrase template other than SELECT, FROM, and WHERE in the process (N in step S4310), the process ends.

  If the operation type of the process has been registered (Y in step S431), the operation type of the target process is acquired from the generated query storage unit 17 (step S4312), and the process ends.

  FIG. 28 is a diagram illustrating a more detailed process flow example of the nested integrated query generation process (step S3).

  The second integrated query generation unit 13 adds a flag indicating that unnested integration is not performed to each query generated by the first integrated query generation unit 11 from the generated query storage unit 17. Then, the second integrated query generation unit 13 determines whether there is a process that has not been integrated by nesting (step S31).

  If there is a process that is not yet integrated by nesting (Y in step S31), the second integrated query generation unit 13 selects a process that is not yet integrated by nesting from the top of the query stored in the generated query storage unit 17. The target process (step S32). The second integrated query generation unit 13 determines whether there is a subsequent process of the target process (step S33). If there is a subsequent process of the target process (Y in step S33), the second integrated query generation unit 13 selects a subsequent process of the target process from the generated query storage unit 17 (step S34).

  The second integrated query generation unit 13 determines whether the processing types of the target process and the subsequent process are different, or whether there are a plurality of subsequent processes of the target process (step S35).

  When the processing types of the target process and the subsequent process are the same and there are not a plurality of subsequent processes of the target process (N in step S35), the second integrated query generation unit 13 nests the generation query of the target process and the subsequent process Are integrated to generate an integrated query (step S36). The second integrated query generation unit 13 saves the generated integrated query as a target process query in the generated query storage unit (step S37), and deletes subsequent processing from the generated query storage unit 17 (step S38).

  As shown in FIG. 29, the generation query Q10 of the process P1 and the generation query Q11 of the process P2 (P2 + P3) are integrated by nesting to generate an integrated query Q20. Then, as shown in FIG. 30, the generated query of the process P1 (date / time conversion) in the generated query storage unit 17 is replaced with the integrated query Q20, and the process P2 (value range specification) is deleted.

  If there is no subsequent process of the target process in the process of step S33 (N of step S33), or the process type of the target process and the subsequent process is different in the process of step S35, or there are a plurality of subsequent processes of the target process If it exists (Y in step S35), the flow definition is marked as integrated for the target process (step S39).

  FIG. 31 is a diagram illustrating a more detailed processing flow example of the component generation processing (step S4).

  The integrated component generation unit 20 determines whether there is a component non-generated process in the flow definition 2 (step S41). If there is a process in which no component is generated (Y in step S41), the integrated component generation unit 20 selects a process in which no component has been generated from the flow definition 2 (step S42), and based on the processing type of the process, the component A corresponding component template is acquired from the template storage unit 21 (step S43).

  In the component template storage unit 21, a component template corresponding to the query language is set as shown in FIG. FIG. 33 is a diagram illustrating an example of a component template. FIG. 33A shows an example of a component template C1 for EPL, and FIG. 33B shows an example of a component template C2 for HiveQL. Here, since the query language is selected according to the processing type, the component template C1 is selected if the processing type is real time, and the component template C2 is selected if the processing type is batch.

  The integrated component generation unit 20 acquires a query corresponding to the selected process from the generation query storage unit 17 (step S44), and generates a component based on the selected component template (step S45). If the processing type of the process P1 from the flow definition 2 is real-time, the component template C1 is selected. Therefore, as shown in FIG. 34, the process P1 stored in the generated query storage unit 17 is integrated into the component template C1. Query Q20 is applied to generate the component.

  The component 3 generated by the integrated component generation unit 20 is output and stored in a storage device, a storage medium, or the like.

  The integrated component generation apparatus 1 described above can be implemented as dedicated hardware including the processing unit illustrated in FIG.

  Further, the integrated component generation apparatus 1 includes a CPU 101, a memory 102, a storage device (hard disk) 103, an input device (keyboard) 104, an output device (display) 105, a network connection device 106, and the like as shown in FIG. It can be implemented by the computer 100 connected via a network or the like.

  Furthermore, the integrated component generation apparatus 1 can be implemented as a program that can be executed by the computer 100. In this case, a program that realizes the function of the processing unit of the integrated component generation apparatus 1 shown in FIG. 4 is installed and executed on the computer 100. That is, the computer 100 is caused to read an execution program that causes the computer to execute the functions of the first integrated query generation unit 11, the second integrated query generation unit 13, and the integrated component generation unit 20 of the query integration device 10 illustrated in FIG. , The integrated component generation apparatus 1 can be realized.

  The execution program is not only a recording medium such as a CD-ROM, CD-RW, DVD-R, DVD-RAM, DVD-RW, or flexible disk, but also other storage devices provided at the end of the communication line. It may be stored in a computer hard disk or the like.

  Note that the elements constituting the query integration device 10 of the integrated component generation device 1 may be realized in any combination. A plurality of components may be realized as one member, and one component may be configured from a plurality of members. In addition, the query integration device 10 is not limited to the above-described embodiment, and various improvements and changes may naturally be made without departing from the gist of the present invention.

  As described above, according to the disclosed query integration device 10, since the queries of each process included in the flow definition 2 are integrated, the number of queries to be generated can be greatly reduced, and the number of components corresponding to the query is also increased. It can be greatly reduced. Therefore, communication overhead and the like can be reduced according to the reduction in the number of components.

DESCRIPTION OF SYMBOLS 1 Integrated component production | generation apparatus 10 Query integration apparatus 11 1st integrated query production | generation part 13 2nd integrated query production | generation part 15 Phrase division | segmentation template memory | storage part 17 Generation | occurrence | production query memory | storage part 20 Integrated component production | generation part 21 Component template memory | storage part 2 Flow definition 3 Generation | occurrence | production component

Claims (4)

Computer
Load a flow definition in which multiple processes with defined process details and attributes and the processing order of the multiple processes are defined,
The process defined in the flow definition is extracted from the head and set as the target process, and the first query corresponding to the target process and the subsequent process of the same processing type as the target process processed immediately after the target process A query integration method, comprising: generating an integrated query that integrates a corresponding second query and repeating the process of using the integrated query as a first query corresponding to the target process. The computer is
In the process of generating the integrated query, when the second query and the first query are of an operation type in which the operation result does not change due to a change in the processing order, the corresponding phrases of the first query and the second query Are combined to generate the integrated query, and when the second query and the first query are of an operation type in which an operation result changes due to a change in processing order, the first query is changed to the second query. The query integration method according to claim 1, wherein the integrated query is generated by being incorporated as a subquery. On the computer,
Load a flow definition in which multiple processes with defined process details and attributes and the processing order of the multiple processes are defined,
The process defined in the flow definition is extracted from the head and set as the target process, and the first query corresponding to the target process and the subsequent process of the same processing type as the target process processed immediately after the target process A query integration program, comprising: generating an integrated query that integrates a corresponding second query and repeating the process of using the integrated query as a first query corresponding to the target process. A flow definition acquisition unit that reads a plurality of processes in which processing contents and attributes are respectively defined, and a flow definition in which a processing order of the plurality of processes is defined;
The process defined in the flow definition is extracted from the head and set as the target process, and the first query corresponding to the target process and the subsequent process of the same processing type as the target process processed immediately after the target process A query integration apparatus comprising: an integrated query generation unit that generates an integrated query that integrates a corresponding second query and sets the integrated query as a first query corresponding to the target process.

Images