JP2018510445A - Domain-specific system and method for improving program performance - Google Patents

Abstract

The present invention relates to a system and method for improving the performance of a computer program such as a database management system (DBMS). The method includes identifying invariant intervals of variables in the DBMS code based on a program representation (PR). Program interactions and domain assertions in the DBMS are estimated based on the PR and the DBMS ecosystem specifications. One or more candidate snippets are generated for the invariant interval of variables in the DBMS code, the PR, one or more execution summaries associated with the DBMS, the estimated program interaction, and the estimated domain assertion. Based on the identification. Spiffs are generated based on the one or more candidate snippets. The spiffs include a predicate query spiff, a hash join query spiff, an aggregate query spiff, a page stiff, and a string machine stiff. The DBMS code is modified based on the spec code generated from these spiffs. [Selection] Figure 1

Description

  The present disclosure relates generally to domain specialization that improves the performance of computer programs, and more particularly to identifying invariant sections of variables and specialized codes generated based at least in part on the identified invariant sections. The present invention relates to a system and method for improving the performance of a database management system by modifying a DBMS code by using.

  A database management system (DBMS) is a collection of software programs that manage the storage and access of data. Today, DBMSs have been adopted throughout a wide range of applications due to the increased amount of data generated and the need for storage and efficient access. With such ubiquitous development over the past 40 years, DBMSs are generally designed and operated based on several data models applicable to such areas. The relational data model is one of the most widely accepted models for commercial open source DBMSs. Considerable effort has been devoted to efficiently support this data model.

  Because of the generality of the relational data model, the relational database management system itself has generality in that it can process any schema specified by the user and any queries or modifications presented. Since relational operators operate on essentially any relation, they need to correspond to predicates specified for any attribute of the basic relation. With innovations such as an effective index structure, innovative concurrency control mechanisms, and sophisticated query optimization methods, relational DBMSs available today are very efficient. Further, due to such generality and efficiency, it can be used by being diffused in many areas.

  Regardless of the above, generality is realized by multiple indirect layers and sophisticated code logic. By utilizing invariant values that exist during the execution of such a system, the efficiency can be further increased with respect to the DBMS. The domain specialization technique disclosed in the present application was developed to automatically identify invariant values and to enable code specialization based on the invariant values.

  Embodiments of the present disclosure provide systems and methods for improving the performance of a database management system (DBMS). Briefly described, an embodiment of this method can be implemented, inter alia, as follows. A computer-implemented method for improving the performance of a DBMS includes: (i) identifying an invariant section of a variable in the DBMS code based on a compile time analysis of the DBMS source code; and (ii) an ecosystem specification of the source code and the DBMS. (Iii) estimating a program interaction in the DBMS based on the source code, invariant intervals identified by the variables in the DBMS code, and the estimated program interaction; One based on invariant intervals of variables in DBMS code, source code, one or more execution summaries associated with DBMS executed using different workloads, estimated program interactions, and estimated domain assertions Or Identifying a number of candidate snippets; (iv) generating specialized DBMS code based on the identified candidate snippets at various times including compile time and run time; and (v) available code Modifying the DBMS by insertion and possibly generating specialized code at runtime and then calling the specialized code.

  Other systems, methods, features, and advantages of the present disclosure will be apparent to those of ordinary skill in the art by reference to the following drawings and detailed description. All such additional systems, methods, features and advantages are intended to be included herein, within the scope of this disclosure, and protected by the accompanying claims.

  Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis may be placed upon clearly illustrating the principles of the present disclosure. Furthermore, in the drawings, the same reference numerals indicate corresponding parts throughout the drawings.

FIG. 2 is a block diagram illustrating a Spiff tool architecture according to an exemplary embodiment provided by the present disclosure. FIG. 3 is a block diagram of a region specialization process according to an exemplary embodiment provided by the present disclosure. FIG. 4 illustrates domain specializations that refine a computer science paradigm according to an exemplary embodiment provided by the present disclosure.

  Many embodiments of the present disclosure may be in the form of computer-executable instructions, such as algorithms executed by a programmable computer. However, the present disclosure can be similarly implemented in other computer system configurations. Certain aspects of the present disclosure can be embodied in a dedicated computer or data processor that is specifically programmed, configured, or constructed to execute one or more of the computer-executable algorithms described below. .

  The present disclosure can also be practiced in distributed computing environments where tasks or modules are performed by remote processing devices that are linked through a communications network. Further, the present disclosure can be implemented in an Internet-based or cloud computing environment, where shared resources, software, and information may be provided to devices such as computers on demand. In a distributed computing environment, program modules or subroutines may be located in both local and remote memory storage devices. Aspects of the present disclosure described below include magnetic and optical readable and removable computer disks, fixed magnetic disks, floppy disk drives, optical disk drives, magneto-optical disk drives, magnetic tapes, hard disk drives (HDDs), semiconductor drives (SSDs). ), A computer readable medium such as a non-volatile memory such as CompactFlash (registered trademark), or a computer readable medium electronically distributed on a network such as a cloud. Data structures and data transmissions specific to aspects of this disclosure are also within the scope of this disclosure.

  Although the present invention is described herein primarily with respect to relational DBMSs, it is not limited to such DBMS types. It will be readily appreciated that the present invention is applicable to any DBMS type, such as, but not limited to, hierarchical, network and object oriented DBMS types. In addition, although domain specification is primarily disclosed herein for DBMS, the concepts provided herein are applicable to any program that performs data manipulation, particularly complex analysis on data. It shall be understood. Specifically, it will be appreciated that the disclosed systems and methods are applicable to computer programs that require high runtime performance and that execute applications multiple times with different parameters or queries on the same data. Spiff, which represents an in-region specializer, is a code that dynamically creates a special code when executing a DBMS. “Regional specialization” is the process of inserting Spiff into the DBMS code so that the DBMS can specialize itself using runtime invariant values. Specialized codes (sometimes referred to herein as “speccode”) are faster and generally smaller than the original non-specialized code. Domain specialization is named for the fact that specialization code is generated and called “in the domain”, ie when the DBMS is deployed and running on the end user side. Spiff dynamically generates code specialized for a specific value of the runtime invariant value by using the actual value of the runtime invariant value (obtained at runtime).

  In Applicant's US patent co-pending application 14 / 368,265 to which this application claims priority, the term “microspecialization” corresponds to the term “regional specialization” as used herein. The term “Bee” corresponds to the term “Spiff” as used herein, and the instantiated Bee corresponds to “specialized code” as used herein (which is the result of Spiff). , HRE (Hive runtime environment) corresponds to “SRE” (Spiff runtime environment) used in this specification.

Architecture FIG. 1 is a block diagram illustrating a Spiff tool architecture according to an exemplary embodiment provided by the present disclosure.

In one embodiment, the present disclosure, as shown in FIG. 1, provides a Spiff tool architecture that automatically domain-specifies any program given the following three inputs:
Application source code, which is a natural input,
One or more workloads,
Ecosystem specification.

  Thus, in this architecture, domain specialization analyzes the application source code and eventually becomes a fully automated process using these three inputs, with specialized semantics that operate faster, but with the same semantics. Is generated.

Goals The goals of this architecture are:
1. Provides an end-to-end solution that uses a source file as a program or a series of related programs, and specializes the above source files, such as code for Spiff generation, Spiff compilation, Spiff instantiation, Spiff launch, and Siff garbage collection Provide automatically.
2. Domain independence is provided in that the architecture works with virtually any program compiled against any conventional architecture, such as a highly parallel graphics processing unit (GPU).
3. Minimize the information that the tool user needs to provide and maximize the information extracted from the program under analysis.
4). Divide the analysis into a series of tools, each performing only one conceptual task.
5. Enables tool scalability by allowing each tool to generate a small output that captures the analysis results of that tool.
6). Enable incremental development so that the tool can be tested first on a small program and then on a realistic program.
7). The ability to refine each tool to produce a more comprehensive output over time after first doing a partial maximum analysis (for example, searching for only some of the invariant values or the minimum candidate snippet) In order to enable continuous improvement.
8). The benefits of individual code transformations introduced by Spiff can be evaluated dynamically and / or independently, and all code combinations can be evaluated comprehensively and valid code without measuring each execution time By taking into account the characteristics of the transformation and the specific workload, the overall profit of the Spiff can be computed, thus enabling performance profit estimation.

Tools As shown in FIG. 1, the Spiff tool architecture includes many tools. These tools include invariant searchers, trackers, invariant testers, program interaction estimators, domain assertion estimators, snippet searchers, and Spiff constructors, each of which is described in detail below. With the exemplary Spiff tool architecture shown in FIG. 1, those skilled in the art will appreciate that many variations and other divisions are possible without substantially departing from the spirit and principles of the present disclosure.

  In this description, a specific program representation (PR) such as an abstract syntax tree (AST), which is a common computer readable structure of application source code, is used. The present invention also applies when the analysis is alternatively performed directly on the text source code of the application, a low-level intermediate representation (IR) of the source code, or an equivalent assembly or machine code representation of the source code. . Each program configuration used by PR is referred to as a program expression (PE). In the case of AST, PE is an AST node, in the case of IR, PE is an IR instruction, and in the case of source code, PE is a single sentence.

Invariant Searcher The invariant searcher receives the PR of the DBMS to be specialized and the tracking event (arbitrary) as inputs, and outputs zero or more invariant sections by executing static analysis on the PR.

Some definitions:
Invariant interval: defined by a single starting PE (or equivalently a single location in the source code) and a single ending PR that can be reached from the starting node during one or more possible executions. This is a set of paths that maintain the characteristics of. An example of such a characteristic is not described (the section can consist of a set of paths instead of a single path. For example, there is no branch of the if / else block that modifies the variable in question. , This variable remains unchanged on all code paths associated with these branches). Note that the invariant section begins at the start PE (that is, starts immediately when the variable becomes the assigned value (the start PE is always a statement that sets the value of the variable)) and ends immediately before the value is set again. Ends in PE. In each PE along this path, the value of the variable is the same as the other points along the path, so it is referred to as invariant.
Invariant section set: A set of invariant sections of a particular variable in which all contained invariant sections share the same starting node. The interval is probably not the maximum in that it ends more than necessary if it cannot be determined by analysis that the specified characteristics persist after execution of the PE.
Value Flow Tree (VFT): A tree that captures a copy of the value of one variable to another variable. When the value of the variable Y is assigned by the variable X, the VFT connects the X invariant section set to the Y invariant section set.

As an example, an invariant interval may exist over an interval in which the value of the variable (ie, the characteristic is a value) persists (eg, (for some constant N) “variable equals N”). This may omit certain types of optimizations such as:
Optimization based on program state (eg, memory allocation optimization in aggregate operations).
Optimization based on properties that are not in the form of “variable equals N” (eg, assuming a portion of code if (p! = NULL) S, the pointer p must be non-null in S; For example, it has been found that in functions called from S, redundant null checks should be eliminated by optimization).
Optimization based on derived values (eg, string length) that may not be explicitly stated in the code.
Optimization based on domain knowledge (eg concentration of a set of values that may appear in a column).

Example 1: if statement Consider the following Example1.

Here, the invariant searcher does not statically grasp whether the “if” sentence is true or false. For this reason, the invariant searcher shall output the following invariant interval set for the variable x:
For simplicity, the line number is used instead of the PE ID to identify the source location. A closed / open section is used.
Invariant section set # 1: Starts at row 1 and is one invariant section:
Invariant section # 1.1: Ends in line 3 Invariant section set # 2: Starts in line 10 and is one invariant section:
Invariant section # 2.1: ends in line 15 (ie at the end of the program)

  The invariant searcher may output the above in some structured format such as XML, but in the present disclosure, lists and sublists are used for simplicity.

  An invariant searcher may vary in its accuracy, but accuracy is required. Specifically, the invariant values generated by the invariant searcher should be accurate but not necessarily exhaustive. For example, x is actually unchanged in rows 1 to 5 and rows 1 to 9. However, since it is also accurate (accuracy is low), for example, the section is stopped at the beginning of the “if” sentence. Of course, in low-accuracy intervals, the snippet searcher and the Spiff composer (the tools described below) do not have much opportunity to domain-specific applications.

The invariant searcher can also output such an invariant interval set for all variables in the program. Consider this for the variable h:
Invariant section set # 3: Starts at line 11 and is one invariant section:
Invariant section # 3.1: Ends with row 13 Invariant section set # 4: Starts with row 13 and is one invariant section:
Invariant interval # 4.1: For variable y ending in row 15, assume that:
Invariant section set # 5: Starts at row 2 and is one invariant section:
Invariant interval # 5.1: For z ending in row 15, assume that:
Invariant section set # 6: Starts at row 12 and is one invariant section:
Invariant interval # 6.1: ending in row 15 Finally, for variable a, assume that:
Invariant section set # 7: Starts at row 14 and is one invariant section:
Invariant section # 7.1: ends in line 15

We refer to how variable h gets its value from variable x (the “flow” of the value from x). This creates a “flow” from the z value h. Therefore, in order to try all together, it is assumed that the VFT of x is given by an exemplary reference expression as shown in Example 2 below.

  The numerical values of the “from” and “to” attributes represent one of the invariant interval sets (IIS) described above. For this reason, the designated first row is the invariant section set # 1 to the invariant section set # 4.

Example 2: Pass-by-value function It is assumed that the variable “a” is unchanged in the function X (), but the value temporarily changes in the called function Y (a). When Y (a) returns, the value of “a” still has its (invariant) value. This situation is addressed because a function call that passes through the value of a variable is a copy of that value to another variable and is associated with its own set of invariant intervals.

Example 3 Loop without Assignment Consider the following example 3 code:

In order to understand the loop, the invariant searcher actually expands the loop. More precisely, we shall check if there is a variable assignment in the loop. As in this example, if there is no assignment, the invariant section reaching the loop will extend throughout the loop:
Invariant section set # 1: Starts at row 1 and is one invariant section:
Ends in invariant section # 1.1: 8

Example 4: Loop with assignments to existing variables However, with reference to Example 4, consider conditional variable assignment in a loop:

Here, the invariant searcher creates the following interval:
Invariant section set # 1: Starts at row 1 and is one invariant section:
Invariant interval # 1.1: Ends with invariant interval set # 2: Begins with row 7 and is one invariant interval:
Invariant section # 2.1: Ends with invariant section set # 3: Begins with row 9 and is one invariant section:
Ends in invariant interval # 3.1: 10

  Again, the low-precision but still accurate simplification excludes any loops that could potentially represent variables. Furthermore, it should be noted that the value of x cannot be changed by function calls, but rather the interval is not terminated at line 8 because the value is copied to a local variable some_other_func.

Example 5: Loop with creation of a new variable Referring to Example 5, consider the creation of a variable in a loop:

Here, the invariant searcher will create:
Invariant interval set # 1: Starts at row 4 and is one invariant interval:
Ends in invariant interval # 1.1: 8 (ie after the last iteration of the loop)

Exemplary Algorithm for Invariant Searchers An invariant searcher starts with a call graph leaf, ie a function that does not call any other function. It is also possible to calculate the VFT of the function and add an edge when the variable is copied (for example, h = x;). Then, after considering a function that calls only leaf functions, it is also possible to add edges of local variables and integrate sections (for example, when x is passed). It is also possible to consider a function that calls only a function for which VFT has been calculated by iteration.

  Extra care is required for recursive functions and cycles in the call graph. The call graph bottom-up scan is a static analysis of the program. Since the invariant value needs to be true in all passes, the invariant searcher uses the signature and assumes that any function that matches the call signature can be indirectly called.

  Note that memory allocation within the loop can result in many different runtime placements. When such an assignment occurs, the memory pointed to by the variable remains unchanged until the variable is assigned. Assignments within the loop result in assignments to new elements (eg, arrays) or variable overrides.

  For indirect function calls, the invariant searcher propagates the value of the function pointer to find a possible set of targets for each indirect call, and the value flow to the bottom up of the call graph as described above. The backward analysis step to propagate can be performed alternately. This alternate execution can be repeated until the set of function pointer objects is stable.

  The invariant searcher may further identify possible values for the variable. As an example, in the case of a variable join_type, there are very few different values assigned to the variable, and all of them may be grasped statically. This may be specified by the type of variable (enumeration) and may be discoverable by static analysis (e.g., examining all values assigned to the variable). If the set of possible values is small, the invariant searcher may record the invariant interval for each value.

Accuracy Each invariant interval returned by the tool shall be accurate. That is, it is guaranteed that the relevant variable is invariant across all paths between the start and end of the interval (not including the end). If there is an indirect assignment in either pass, the invariant searcher tool needs to ensure that all such assignments do not change the value of the specified variable.

  The analysis may be careful in two respects. First, there may be a detection failure ((a) section set or (b) sections that are accurate as individual sections within the section set but are not returned by the tool). The tool indicates where the variable is assigned (ie starting an invariant interval) but not analyzed by the tool (unknown interval set) and the incomplete interval set (unknown individual interval) The case is acceptable.

  Second, there may be a non-maximum section (a section that does not end with a sentence that changes the value unconditionally). This can be (a) an assignment that does not actually change the value, or (b) a non-assignment whose analysis is not accurate enough to determine that the value is unchanged (for example, it can change the value in the sentence) "For" sentence) may also be the cause.

  Also, accuracy requires that all value flow tree links are correct, each representing a copy of the value. However, these links can be non-maximal in that a link between section sets is not required even if the value of one section set is actually derived from another section set.

Tracker Another tool disclosed herein is referred to as a “tracker”. The tracker receives an executable file under the workload and outputs a series of tracking events. The output of the trace event usually records the execution of an instruction that can affect the data flow in the program, such as “enter loop”, “read variable”, or “function call”.

  Trace events are processed by a separate tool “summarizer” to generate an execution summary that outputs functions, sentences, and variables along with their execution statistics. Such information indicates “hot spots” in the application that can benefit from domain specialization.

  The impact of accuracy is that if a particular activity of interest occurs during execution, the associated tracking event is output and / or recorded, and any output and / or recorded tracking event corresponds to the occurrence of the activity of interest in a specified order To do.

Invariant Inspector Another tool, Invariant Inspector, uses a tracking event for a given execution to determine whether a violation of the identified invariant value (eg, identified by an invariant searcher) has occurred in that execution. (Or the developer can give instructions to the invariant tester by indicating the important variables to observe). Ideally, the invariant checker will not find violations for many executions of the DBMS executable over many workloads (this ensures that the invariant values found by the invariant searcher are correct).

  The invariant tester may further verify analysis by other tools (such as invariant searchers and trackers) by periodic operation. The user of the application may be instructed, for example, to activate an invariant tester and indicate that no violation was found. On the other hand, if a violation is found, the user may be informed that a violation has been found and may be further provided with a message to contact technical support for assistance.

  An invariant tester is another application, for example as a debugging tool employed by the developers of the tools described herein to ensure the accuracy of static analysis (eg, invariant values identified by an invariant searcher). There is.

Program interaction estimator The tool “Program Interaction Estimator” is a data file with PR (or source code, IR code, or equivalent representation of assembly or machine code, etc.) and ecosystem specifications along with related information. Estimate program interactions that are lists of Basically, the program interaction estimator is where the values are stored in the file in the program, where the values are later read from the file, and these values are removed from the file (or the file itself is removed). Confirm the location. These values will then be determined as long-term invariant values in the domain assertion inductor.

  The ecosystem specification includes (a) the data involved, (b) a fixed and variable data file, (c) a program that can create, access and destroy this data, and (d) any parallel Describes processing requirements. Although this disclosure focuses on files, in general, this specification is more generally related to read and write data from the outside world, including files, but user I / O, Other possible ways for other interaction with data and O / S by the program, such as stream to other processes / streams from other processes, and memory allocation and character encoding processing are also mentioned. While files are the most common method and are the heart of this description, it is to be understood that the invention can use any other data format as described above.

Table Spiff Use Cases As an aid to the description of the tools provided herein, some examples are described in the context of a prototypical DBMS (“minidb”). minidb. h and minidb. An excerpt from the c source file is shown in Example 6, which will be referred to repeatedly.

An ecosystem specification that can be provided by a developer as a configuration file that describes a non-trivial feature of a specific function related to data flow operations in an application (an exemplary ecosystem specification is shown in Example 7 below) is: Starting to become empty, the workload is read from standard input or a file, (b) the (workload) data can fluctuate, (c) only the minidb has access to the data, and (d) any specific It will describe that at most one minidb instance operates on the directory. The ecosystem specification is essential to understand that the schema is immutable throughout the execution of minidb.

  Minidb uses two types of data: a table that is a file that holds table rows, and a workload that is a file containing SQL statements. The name of the type is only for distinguishing these files in the following description. Each table exists in a directory (database).

  This ecosystem has one program called minidb. This program creates a table data file. By giving the line of code that performs this operation (eg, line 3 of the CreateTable function), the file being manipulated (here, the specific file described in that line of code) is communicated to the domain assertion estimator. The Conolas font used in the embodiment is the name of a function in the minidb source code. The verb “reads” indicates that the directory has not been created or removed by the application. In the case of a table data file, the file is indicated by the file passed to CreateTable (). The verb “creates” also suggests “opens”, “reads”, “writes”, and “removes” (the domain assertion estimator may find that each table exists in the database directory Thus, the inDirectory attribute and possibly the entire directory element may be unnecessary, and the ecosystem specification is shortened by one line).

  The program opens the workload data file, which suggests "reads". Here, the file is the one passed to GetNextCommand (). Alternatively, this file may be read from standard input at line 7 of GetNextCommand (). Multiple parallel instantiations of minidb may read the same workload file, but do not access or change the internal database directory or table file.

The program interaction extracted from PR (see Example 8) describes that minidb creates table files in this directory, reads and writes them, and then removes them, where each of these file operations occurs in the source Is strictly shown. Furthermore, the table header in the file is never changed in the file, and the file is uniquely identified by the variable “data_file_name”.

  The table data file is created first in the database directory. (In this example, only one application called minidb is used, so this can be specified in the data file rather than adding or removing data. is there). This file includes three data structures, TableHeader and a plurality of “RowHeaders”, each with a row (string). In subsequent analysis in the tool, it is not necessary to grasp the inclusion structure, and only the data structure that is read after being written is required. Of course, once the data has been written to the file, it can possibly be read many times before it is erased.

  The lifetime of the file extends beyond the individual execution of the application. After a file is created by one execution, data may be written to the file by another execution, and then the data may be read by another execution, and then the file may be removed by another execution . As an important semantics, the data written to the file is the same as the data that is later read from the file, after which the data is erased from the file or the file itself is removed. Another important semantic is that the PR knows the actual C structure that is read after being written to the file.

  Returning to the details of the prototypical DBMS (ie, minidb), interestingly, the erase is actually a file write. Line overwriting occurs, and the original line is erased.

  The logic of ExecuteDelete () is particularly complex: a temporary file is created, the tile line before the erased line is copied to the temporary file, the line after the erased line is copied, and then the temporary file The name is changed. The program interaction estimator may be adapted to handle these details by including such logic.

Table Spiff Instance Use Case A table Spiff instance is associated with a specific row in the database, and its handling is as described above.

Row Spiff
The concept of “row” is considered quite domain specific. However, the general concept is a partial concept of a data file that is read, written, and processed as a whole. There is also a concept of a query evaluation loop in each line, which can be generalized as a part of code used for processing each attribute part of the input file. Thus, in order to recognize the line Spiff, it is necessary to recognize the timing at which a part of the input file is processed, and the same code is repeatedly used for different parts having the same structure.

  Implementation of row Spiff requires (i) determination of invariant values to be used for data partitioning, (ii) placement of SpiffIDs in the data, and (iii) removal of data values, possibly determined by SpiffIDs. In the first step, a cost model that depends on the workload is used. In the second step, in order to actually change the structure of the input data, it is necessary to change each related program of the ecosystem (reading or writing the relevant part of the data). The third problem will be handled in the same manner.

  Thus, a unique aspect of the concept of row Spiff is that (a) identifying the part of the data that is processed in the unit and (b) changing the data so that it can be operated more efficiently in the program accessing this data It is to be.

Query Spiff Use Case A query Spiff is a combination of a query, a table, and a row invariant value. The latter two are solved above, but the query invariant values are found by the invariant searcher. In this case, the query invariant value does not persist for the entire minidb execution. This is because the workload from which the query is derived is only read and can be used by multiple minidb instantiations (for example, because parallelaccess is possible).

Exemplary Algorithm for Program Interaction Estimator As shown in FIG. 1, the program interaction estimator has two inputs: ecosystem specification and PR. While the ecosystem specification focuses on programs that read and manipulate data, the generated program interactions are made to files, especially the data structures in the program are written to and read from files. Focus on location. Thus, the program interaction estimator, or PID, analyzes file manipulation system calls, specifically fopen (), fwrite (), and remove (). In addition, the <datafile> and <workload> specified by the ecosystem specification are used as the starting point. In this case, a table and a workload are used (for example, as shown in Example 6). Parses the database, but finds it fairly quickly that this is just a directory read by OpenTable ()).

  Between these file operation calls, the PID observes the flow of FILE * values.

Workload files are particularly easy to analyze. The ecosystem specification specifies that this file is opened with GetNextCommand (): 13 (the file can also be read from the standard input). The PID determines the following by analyzing the source code referenced by the specification.
The file is named by query_file_name,
Associated with FILEquery_file and stdin,
Reading of this file should be only the character string read by GetNextCommand (): 8 and GetNextCommand (): 18.

  Therefore, the program interaction estimator outputs the determined information to the program interaction file as shown in Example 7.

Table files have more complex behavior. The ecosystem specification describes creates = "CreateTable (): 3" indicating that it is necessary to follow the data_file_name derived from the data structure TableHeader.table_file estimated by analyzing the reference source code. For this reason, the PID is
Call WriteTableHeader (): 3 (fwrite ()) from main (): case'C 'to CreateTable (): 2 (fopen ())
After main (), go back to many rows written (in case "I": fwrite () in WriteRow (): 3 and 6) (without case "D": fwrite () (this is difficult to detect) In ExecuteDelete (): 25) and the deleted line, and
By examining the reference source code, a file that has been deleted again in main (): 57: remove () can eventually become a subsequent flow.

From this series of overall operations on the table datafile associated with the C file table_header.table_file, the PID is
After the TableHeader data structure is written to the table file with WriteTableHeader (): 3,
0penTable (): read by 8,
Can be estimated.

  Interestingly, this is all done with TableHeader, written once and never erased from the file.

The PID has a RowHeader data structure,
WriteRow (): After being added to the table file with 3,
SequentialScan (): read by 7,
ExecuteDelete (): 25 is removed from the file,
Can be determined.

Finally, the PID is a string
WriteRow (): 6 added to table file,
SequentialScanO: read at 15
ExecuteDeleteQ: 15 can decide to be removed from the file.

Thus, the analysis performed by PID analyzes how each program operates on each file identified in the ecosystem specification by observing the following according to the value of a FILE type variable:
1. Where the file name came from (also a variable in the program),
2. Where the file is opened,
3. From where the file value flow occurs in the program,
4). Eventually, (i) after being written, (ii) read from the file and then (iii) erased from the file, and
5. The place where the file will eventually be erased or closed.

  Note that this analysis is entirely within the context of a single execution of a single program. If there are multiple programs, each is analyzed separately. There are many cases where each program is executed multiple times, but in this analysis, only a single execution is considered.

In PID analysis, first:
Search for file variables,
Calculate the value flow of these values,
Identify file open / close, write and erase operations along the value flow,
For each, it is necessary to identify specific information recorded in the program interaction.

  The domain assertion estimator, described below, captures and combines the pre-execution behavior extracted by the PID to show the data flow from the program to the file and the data flow back to the subsequent program (possibly subsequent execution). By understanding the whole, the invariant value flow of the entire program execution that could not be done by the conventional compiler analysis is calculated.

Domain Assertion Estimator The domain assertion estimator tool estimates domain assertions using PR, identified invariants, and program interactions. In the case of the table Spiff, the program interaction suggests schema information that becomes invariant. For table Spiff instances, program interaction is essential to understand where rows are created, accessed, updated, and deleted in the program. Query Spiff can utilize schema, row, and workload generation invariant values, along with a smaller range of invariant values within the scope of conventional compiler optimization techniques. In specifying domain-specific knowledge, some of these operations are described in some detail. The domain assertion estimator then derives the full lifetime of the value by combining invariant intervals, possibly over multiple invocations of the program, according to the values read and written to the file, and then signing with the domain assertion Turn into. A domain assertion estimator (DAD) can consider a limited set of invariant values if the workload is complete, i.e. fully characterizing possible behavior for data that may be applicable to some batch applications. The value can also be estimated. This information (domain assertions and possibly a set of possible invariants) is what traditional compilers do not have.

  An important aspect of domain specialization is the use of information that is not generally available to compilers. This information is in two forms: (i) domain specific knowledge and (ii) preliminary source knowledge. Both types of knowledge go beyond what a compiler can find or conclude by looking at the source code of a single program. Domain specialization is a much broader ecosystem of specialized programs such as data to read or manipulate, calling programs, other programs that also read or manipulate this data, operating systems involved, network routers, and storage systems Take into account. This ecosystem provides a lot of information that can be used in domain specialization to improve the efficiency of the specialization program (and its data).

  Compared to somewhat comprehensive preliminary source information, domain specific knowledge applies only to programs in a specific domain. An example of domain specific knowledge is "any change in table schema can be serialized". Serializability is a complex concept that originated from the database domain, but has also been extended to other parallel and distributed information processing applications. With such knowledge, it is possible to create a table Spiff that speeds up the DBMS, including precisely indicating where the table Spiff should be created and where it should be destroyed.

  The second form of domain specific knowledge is the form of program workload. As an example, “OLAP (online analytics processing) applications show little data variability, (complex) queries are often dominant during the day, and updates are infrequent and usually at night.” Such information is in the form of "This activity is more frequent than other activities", so by giving instructions for domain specialization, it is now possible to make better decisions based on information at the expense of work Then, some speedup will be performed later.

  An example of preliminary source knowledge is a particular part of a file that has been written and read by a single program, which will remain until code that modifies that part or code that erases the file is executed. Based on such knowledge, it is possible to create a Spiff that speeds up an arbitrary program that repeatedly processes an input file.

  The domain specific knowledge and preliminary source knowledge are preferably formalized so that the Spiff searcher can read a file containing domain assertions and preliminary source assertions that formally describe such knowledge. Then, the Spiff searcher reads a file containing DBMS source code (or more generally, source code of an arbitrary program described in the domain specification), and uses a Spiff invariant value used by the Spiff composer. Output.

Table Spiff use case By combining the information from the program interaction with the invariant section on main (): table_header.num_columns, the information required by the DAD to generate the domain assertion shown in Example 9 below is obtained.

  Note that ExecuteDelete (): 38 is not included. This is because the analysis shows that it is a name change.

Table Spiff Instance Use Case A database table row differs from a schema in that there are multiple rows for each table but only one schema exists. In the above, there is one value, table_header.num_columns, stored in the file associated with the table, but this value is written back to the file and then read back. Program interaction shows that in addition to this, various different values of row_data are written to the same file.

  A line is identified by its location in the data file, ie the file position (offset of the first byte of the line).

As shown in Example 10, the domain assertion generated by DAD is a function indicating that the function dependency is valid only when the current position in the execution of the program in the specified file to be read or written is a specific value. Differentiate rows by the keyword OFFSET on the left side of the dependency.

  Intuitively, data is present at the location where reading occurs, while the data is the same as the data that was written earlier at the location described by the start of the segment. According to DAD, it is known that multiple values of variables in a program are differentiated using OFFSET after being written to the same file. Since one of the segment end statements does not include OFFSET, all line invariant values of the entire file end when the file is deleted.

  According to DAD, it can also be seen that case “D” (minidb command interpreted in the implementation of minidb using a switch / case statement) moves a data packet between OFFSETs in the file. One skilled in the art will readily appreciate that the domain assertion format need only be extended to accommodate such movement.

  Other than OFFSET, row invariant values have the same structure as table schema invariant values.

  Furthermore, each file written to or read from an application is considered to be composed of data packets, each of which is a value in an external format in a local variable of a program that is written and read as a unit. For this reason, minidb. In c, a schema data packet (including the value of table_header.num_columns) is first arranged in the file, and then a series of row data packets (including the value of row_values) are arranged.

Query Spiff use case (Example 11)
There are two cases. The first is when the workload (ie query) comes from standard input, in which case domain assertions cannot be estimated. This is because the user can enter anything (and, of course, by using VFT, it can be determined that (many) invariant values are active during the query) Has been performed by an invariant searcher). The second is when the workload comes from a file, ie the file named in the startup argument, in which case the domain assertion is essentially the same as a row invariant value in processing OFFSET. In the second case, an actual file is indicated as a workload source by using a file name. These two cases are distinguished by the ecosystem specification, but in the second case, it is described that a specific workload file can also be specified. On the other hand, the creator of the workload file does not need to be grasped. In such a case, the query Spiff of the workload cannot be created.

  The second case may be used for workload specialization. The query SpiffID may be inserted into the workload, may be arranged as an association stored somewhere else, or may be used when the workload is executed. Good.

  In the following, only the first case where the query is not grasped until reading will be considered.

  It is assumed that a query Spiff that includes only valid invariant values in the query can be found by the aggressive optimization compiler. More importantly, however, Query Spiff combines such query invariant values with schema and row invariant values that the compiler cannot find. This is because such schema and row invariant values require knowledge of file read / write semantics. It is from this point of view that the query Spiff becomes a true Spiff.

Exemplary Algorithm for Domain Assertion Estimator The tracking segmentation element is derived from a program interaction in which a data file or data file components (here table headers and rows) are created, inserted, or deleted. The dependencies in the domain assertion come from the directory and file name and optionally OFFSET in the file. Since the invariant values splice these together, the value of the variable in the application can be viewed as flowing back from the application code to the file and then back to the application, resulting in a long-life invariant value that can determine the candidate snippet and final Spiff. It can be confirmed.

For a table Spiff use case where table_header.num_columns is characterized in a domain assertion, the DAD can determine:
main (): Case “C” calls CreateTable () This calls WriteTableHeader () This writes the table header to the file.

The table header is:
Read by the above and subsequent program startup,
This is until the file is deleted by main (): 57.

  This suggests that the table header in table <datafile> is written once and never modified by the program. By grasping this, the DAD can generate an appropriate tracking division element, and the tracking division element of the table <datafile> is created and deleted. The DAD can also create a dependency on table_header.num_columns.

  For the table Spiff instance use case, the associated data packets are row_data and row_values that are added to the table <datafile>, possibly deleted from the file, and eventually removed when the file itself is deleted. When a file is created, the DAD determines that each such packet can be identified by the OFFSET in which the program stores a plurality of values of row_data in the file.

  The above observation provides a DAD algorithm. For each FILE created or opened by the program, the DAD uses the VFT to find out where the file was originally created and where the name of the file came from. For each data structure stored in the file (these are the <data> elements in the program interaction), the DAD performs file operations (addition of the data structure to the file) performed on the data structure. (Possibly changing or removing data structures, and finally erasing files). These operations then suggest appropriate tracking segmentation elements. Eventually, from the program data structure (C or C ++ program data structure written to the file) used in these operations, the DAD examines the VFT to derive the value in such program data structure. By determining the location, dependencies can be suggested. The DAD also keeps track of execution for each FILE variable as it flows through the program, so that the file contains only one data packet (as in num_columns) or multiple (as in row_data). Although it is determined, this can also be determined by VFT. Multiple packets require OFFSET in domain tracking partitioning and dependencies.

Accuracy As accuracy affects, the generated domain assertions are complete and consistent with the input PR, invariant values, and program interactions.

Snippet Searcher As shown in FIG. 1, another tool “Snippet Searcher” takes as input:
One or more invariant values PR
One or more execution summary domain assertion program interactions, and
Cost model

The snippet searcher outputs one or more <spiff> elements, each containing one or more candidate snippets, each candidate snippet containing:
Interval of code identified by PE A set of invariant values A possible set of values for each invariant value The source location where each invariant value was first written to the file,
The source location from which the value was removed from the file,
The source position from which values are read out over time,
An appropriate lifetime for the candidate snippet, ie when it can create an associated Spiff (whether at compile time or run time), and
Optionally, recommended optimization adopted within the interval.

  Each domain assertion suggests an interval that is almost certainly wider than just one program execution for an interval recorded in an invariant value with a range within a single program execution.

  The snippet searcher uses domain assertions to expand the range of invariant values and improve the set of possible values for each invariant value. Each invariant interval overlaps (partially or entirely) the candidate snippet interval. In addition, each candidate snippet interval is adjusted by the cost model to maximize the savings calculated as the product of the execution cost of the optimized snippet derived from the execution summary and the Spiff call cost and the number of snippet evaluations. Minimize the size. For this reason, the snippet searcher needs to have an idea of the possible optimizations that the Spiff composer performs and the benefits of each such optimization (the latter depending on the cost model).

Query Spiff use case (Example 12)
There are two major differences between the query Spiff and the table Spiff discussed above. The first is dealt with in this step, but the snippet searcher uses invariant values from other than domain assertions. This is because queries typically do not persist (but see the discussion above regarding the workload given to standard input). Although the second will be dealt with later, the Spiff constructor requires explicit instructions from the ecosystem specification regarding the location of the boundary between compiling the Spiff code at runtime and instantiating the Spiff instance.

The snippet searcher estimates the following:
Code section from SequentialScan (): 40 (immediately after decompressing the data packet to row_values []) to SequentialScan (): 59 (end of method)
A set of invariant values main (). Query, in particular query.executor_routine, query.executor_command, query.num_predicates, query.predicate_list, and predicates [], read from stdin and query.schema according to the table Spiff use case
A set of possible values for each invariant (in this case, query.executor_routine is always SequentialScan (), query.executor_command is always SCAN_FWD, and for each predicate, column_id is (estimated from the assignment of BuildPredicates () to that region) Is an arbitrary int read from stdin, constant_operand is an unsigned long read from stdin, and operator_function is & EqualInt4 or & LessThanInt8),
The source location where the value of each invariant value was first determined (the value of the query is determined immediately after calling main (): 32, ie BuildAndPlanQuery ()).
The value of the query is never written to, removed from, or read from the file,
Appropriate lifetime of the candidate snippet (the lifetime is just in the “S” switch, and according to the ecosystem specifications, it can be seen that the compiler cannot be called, so executor_routine as SequentialScan (), executor_command as SCAN_FWD, 1 For num_predicates of ~ 6, only pre-compile Spiff, and for each such predicate operator_function is & EqualInt4 or & LessThanInt8),
For loop expansion in SequentialScan (): 47 that terminates with the value of schema-> num_columns.

  Although the method for determining fromValue and toValue is unclear, it is considered to indicate the boundary of the number of generated compile-time queries Spiff.

Exemplary Algorithm for Snippet Searcher The snippet searcher first extends invariant values throughout the program execution by tracking variables read from the file and where these values are inserted into the file. The result is an invariant interval that spans multiple runs. The tool also needs to keep track of when values are first written to the file and when they are erased.

  Another challenge of the snippet searcher is to use a cost model to indicate snippet boundaries. Therefore, in this tool, it is necessary to grasp the optimization that can be brought about by the Spiff composer and the conditions under which each optimization can be realized.

Accuracy As the accuracy of the tool affects, the candidate snippets generated by the tool are consistent with the input invariant, PR, execution summary, domain assertion, program interaction, and cost model, respectively. Assume that the specified invariant values are actually invariant on the snippet, the recommended optimization is consistent with these invariant values and their respective operations in PR, and possible values are actually possible.

As desirable but not essential:
Given the cost model, the returned snippet is most desirable,
The snippet is the largest in terms of cost model, which leads to increased costs.
The snippet is minimal in that the cost model leads to increased costs.
Given the cost model, recommended optimization is useful.

Spiff Configurator As shown in FIG. 1, another tool “Spiff Configurator” takes one or more candidate snippets and PR as input and generates specialized source code as output.

Specifically, for each input candidate snippet, the Spiff composer shall perform the following tasks:
1. A Spiff pattern that defines all pattern parameters and Spiff pattern functions. h Create a file.
2. Spiff implementation declaration. h Create a file.
3. Spiff implementation definition. c file is created.
4). Create a Spiff (of Dynamic Spiff) by inserting code in the appropriate place, call Spiff, and destroy Spiff (again of Dynamic Spiff).

  Each use case is associated with a specified branch of minidb, ensuring accuracy. Each branch includes a candidate snippet that generates the configuration.

  For actual conversion, it is considered beneficial to use TXL as a PR-PR conversion, and the converted PE is then converted back to the text source code to create a Spiff. TXL includes a parser, but there are cases where PE can be obtained directly. The TXL also includes a syntax tree amperer that can cooperate with the PE.

Some instructions based on domain knowledge may be required for the Spiff composer to function as described above. Specifically, the Spiff composer may require the following assignment / identification:
A specification of all static implementations to be generated (ie variables to be specialized and values in that case).
Rules for disambiguation when two or more static implementations apply Rules for creating dynamic implementations (whether they are allowed or cached anyway, how are they? Memory? Disk Above ?, cache size and management rules ?, generated dynamic implementation should be fully specialized, or only partly specific, some parameters should remain general Is it acceptable to compile dynamic implementations on-the-fly as needed, or to use dynamic implementations only when they are in the cache, and the answers to these questions vary ?
Should a fully general implementation be created (and used as a fallback)? Or are some variables always specialized in some way (this will determine the variables that need to be represented in the data block)?

  In general, the Spiff composer identifies all of the above in the input file. It is the job of the snippet searcher to find out the number of static implementations to be created, whether to be static / dynamic, variables that are specialized, variables that are not specialized, and the like. There is only one static implementation to be created, and the single static implementation is always called.

Referring to Example 13 of Query Spiff, the input is as follows, and compile-time query Spiff is shown for executor_command as SCAN_FWD, num_predicates of 1-6 as specified by snippet searcher, For each predicate, operator_function is & EqualInt4 or & LessThanInt8.

  As described above, the Spiff constructor requires explicit instructions from the ecosystem specification regarding the location of the boundary between compiling the Spiff code at runtime and instantiating the Spiff instance. Assume that such a boundary specification occurs in cases “S”, “I”, and “D” and that the ecosystem specification specifies that Spiff is not compilable in any of these three cases. (This emphasizes the user's knowledge of delays allowed. Note that compiling a new Spiff for a particular row is particularly beneficial to speed up the collection of workloads as a whole. However, the user still wants to specify that it will not compile, because certain workloads need to run faster due to domain specialization). For this reason, the Spiff composer includes an ecosystem specification as an input.

  In this way, the Spiff composer configures Spiff for a part of SequentialScan (), but since query.executor_command is always SCAN_FWD, one is configured for each value of num_columns at compile time. For each predicate, column_id is an arbitrary int, constant_operand is an unsigned long read from stdin, and operator_function is & EqualInt4 or & LessThanInt8 (Spiff0 is a non-specialization that can handle any number of num_columns). Related transformations are loop unrolling and constant convolution. The Spiff composer generates a Spiff pattern for 23 SpiffIDs when num_predicates = 2. As follows, the first is column_id = 2 and operator_function = & EqualInt4, and the second is column_id = 7 and operator_function = & LessThanInt8. Note that the computation of the query SpiffID is usually associated with a specific value of the Spiff pattern parameter. The Spiff composer generates an appropriate SpiffID using an application-specific ID generation mechanism. However, in this embodiment, 23 is assumed as the calculated SpiffID.

Example Algorithm of the Spiff Configurator The Spiff Configurator only determines whether to allow the compiler to perform optimization after showing invariant values or to perform optimization manually by generating different code .

  The Spiff composer then stitches the generated files by a near verbatim copy from the original source to the specialized source using the file name, row number, and number of columns in the associated PE. For this reason, the Spiff composer needs to do a very limited amount of parsing and syntax building, and most of its effort consists of copying code from the original source to the specialized source in place. .

Correctness As correctness affects, the code compiles and operates, but in semantics it is identical to the original code it replaces, but matches the input information.

  The following description provides additional examples that demonstrate the creation of the MiniDB table Spiff and the MiniDB query Spiff.

MiniDB table Spiff
In the following example, creation of a table Spiff from invariant values schema-> num_columns == CONSTANT in the SequentialScan () function shown in Example 4 is demonstrated.

Invariant Searcher In the above example, the invariant searcher shall identify the following invariant interval set with respect to the SequentialScan () :: schema-> num_columns variable:
Invariant section set # 1: Starts at row 52, one invariant section:
Invariant section # 1.1: ends at line 114

Also, the invariant searcher shall generate a VFT and indicate where the variable SequentialScan () :: schema-> num_columns gets a value:
SequentialScan () :: schema-> num_columns gets the value from ExecuteQuery () :: query->schema-> num_columns
ExecuteQuery () :: query->schema-> num_columns gets the value from main (): query->schema-> num_columns
main (): query->schema-> num_columns gets the value from main () :: table_header-> num_columns
main () :: table_header-> num_columns gets the value from fread () of OpenTable ().

  For this reason, the value of SequentialScan () :: schema-> num_columns ultimately comes from the call to fread () in OpenTable ().

Invariant checker The invariant checker is that main () :: table_header-> num_columns is assigned to row 634 only once, and the value of this variable has never been changed through a specific end node by that execution of the given workload. Make sure.

  If the domain assertion estimator is executed before the invariant searcher, the invariant checker can also confirm that the actual value is included in the possible values. Thus, it is considered that the range of the invariant searcher can be suppressed by concentrating the analysis on a specific value or variable.

Snippet Searcher The snippet searcher uses the execution summary analysis along with the cost model to calculate the computation time in case “S”, although cases “C”, “I”, and “D” are too expensive to create Spiff. Is sufficient to recommend specialization of the case.

  Let's start with a simple cost model that only describes that PEs (or other equivalent implementations) whose execution is less than a fixed number or percentage of times are not specialized.

  In such a case, the snippet searcher uses the domain assertion so that schema-> num_columns is invariant in the range from the creation of the data file to the removal of the file for the entire SequentialScan (). c: When the number of columns is stored in WriteTableHeader (): 3 executed immediately after 553, it is estimated that the value of the variable is first written to the file. This value has never been removed from the file, but the file itself has been removed with main (): 57. This indicates that Spiff can be created at the time of compilation. The snippet is assumed to spread from ExecuteTable (): 2θ through ExecuteTable (): 23. This is the scope of the snippet specialized on num_columns and is determined by looking at other statements that can be specialized on num_columns (essentially, num_columns is less frequently used, so it is far from this specialization opportunity. ing). However, since the separate indirect call is expensive, the snippet searcher extends this snippet throughout ExecuteQuery () with another use of this invariant value in line 7. Finally, the snippet searcher shall recommend loop expansion of this candidate snippet.

In this case, Spiff has only one Spiff function indicated by <snippet> shown in Example 14.

The snippet searcher can also infer that the data packet is created in cases “I” and “D” of main () and removed in cases “P” and “D” by domain assertion. More specifically, the snippet searcher estimates:
Code section from SequentialScan (): 16 (immediately after reading the data packet) to SequentialScan (): 38 (end of decompression for loop),
A set of invariant values (values from row_data and schema from invariant value analysis above),
A possible set of values for each invariant value (in this case the value of num_columns is 3, the value of the first column is hard-coded int and schema, the type of the first column is int, the second column Type is long, the third column type is int, an array of arbitrary characters),
The source location (WriteRow (): 18 and WriteRow (): 25) where each invariant was first written to the file,
The source location where the value was removed from the file (main (): 57 and ExecuteDelete (): 25),
Source position (main (): SequentialScan (): 3) from which the value is read every time
the appropriate lifetime of the candidate snippet constructed at the table stipulated time using query.schema invariant values (because it includes data packets that may be inserted, so if Spiff is instantiated at runtime, row_values is given, Data packets are removed after a few queries work, so it needs to be fast (because the number of possible row_values is very large)
For loop expansion in SequentialScan (): 16 when ending with schema-> num_columns value and using row_data and shcema values.

As shown in Example 15, in the analysis, a table invariant value in a wider range is combined with a row invariant value in a narrower range, and a different method is adopted for each. The former can generate code when a table Spiff is defined, while the latter involves instantiation of Spiff at runtime by providing values in the row_values array. In DBMS domain specialization, schema invariant values will play a major role in table Spiff instances and query Spiffs that contain continuously narrow ranges of invariant values.

Spiff composer This is the simplest use case because there are no instances. In this case, four variations are examined in detail.

createAt="compileTime"は、このＳｐｉｆｆが静的実装を有すべきことを記述したものである。
valueRead="OpenTable():8"は、変数が外界から読み出される場所を記述しているため、Ｓｐｉｆｆが選択される可能性がある場所を示している。
existsFrom="WriteTableHeader():3"は、変数が外界に書き出される場所を記述しているため、動的Ｓｐｉｆｆが作成される可能性がある場所を示している。静的Ｓｐｉｆｆの場合は、安全に無視することができる。
existsTo="main():57"は、「外界」がファイルである場合に、ファイルが消去／除去される場所を記述しているため、動的Ｓｐｉｆｆのガベージコレクションが行われる可能性がある場所を示している。静的Ｓｐｉｆｆの場合は、安全に無視することができる。
replaceFunction="ExecuteQuery()"は、特化すべき関数を記述したものである。ここでは１つだけ存在するが、一般的には多数存在可能である。
value="3"は、Ｓｐｉｆｆを（この場合、静的に）生成すべき不変数の値を記述したものである。ここでは１つだけ存在するが、一般的には多数存在可能である。 Variation 1: Single static implementation:
Consider the following input candidate snippet corresponding to a single invariant value of minidb, which is a static Spiff implementation shown in Example 16.

createAt = "compileTime" describes that this Spiff should have a static implementation.
Since valueRead = "OpenTable (): 8" describes a place where a variable is read from the outside world, it indicates a place where Spiff may be selected.
existsFrom = "WriteTableHeader (): 3" describes a place where a dynamic Spiff may be created because a place where a variable is written to the outside world is described. In the case of static Spiff, it can be safely ignored.
existsTo = "main (): 57" describes the location where the file is erased / removed when the "external world" is a file, so the location where dynamic Spiff garbage collection may occur Is shown. In the case of static Spiff, it can be safely ignored.
replaceFunction = "ExecuteQuery ()" describes the function to be specialized. There is only one here, but in general there can be many.
value = "3" describes the value of the invariable to generate Spiff (in this case, statically). There is only one here, but in general there can be many.

  This input consists of a Spiff pattern based on a single Spiff pattern function based on ExecuteQuery () and a static implementation that specializes the variable ExecuteQuery () :: query-> schema-> num_columns into a single literal value 3. To the Spiff composer.

Variant 2: Specialization with fine granularity The above example shows Spiff applied to replace the entire function (ExecuteQuery ()). In fact, it can be observed that only a few code segments of the function contain invariant values. Then, as shown in the following snippet, the few code segments can be converted to Spiff.

The candidate snippet shown below (shown in Example 17) is very similar to the above, but the interval is much smaller than the entire ExecuteQuery () function, with only three lines in the for loop. Therefore, the replaceFunction attribute has disappeared. Finally, the constant convolution recommendation on line 21 is omitted because the line is not within the section to be specialized (it can be left, but in that case it is ignored).

Variant 3: Adoption of Fixed Mounting Array In a specific embodiment, the identification of the Spiff implementation with a 1-byte integer is determined. Therefore, it is possible to have a total of 255 implementations in the same Spiff pattern, and the num_columns variable varies between 1 and 255 by operating as a Spiff pattern parameter (0 is selected to represent an invalid value). For this reason, the candidate snippet shown in the following Example 42 has not only the value 3 but also all the values 1 to 255 (that is, the fromValue and toValue attributes of the invariantIntervalSet element). Return to the function replacement.

Variation 4: Dynamic Spiff
In reality, each column of the table can have a specific data type. Assuming that there are eight data types (int2, int4, char, varchar, etc.), the static table Spiff of the three-column table requires a possible implementation. Therefore, in this scenario, the dynamic table Spiff is more suitable.

The candidate snippet given below (see Example 19) describes this with the createAt attribute. Here, the location in the application where the Spiff is created, that is, the CreateTable () function (createAt attribute (compileTime in the previous example)) ) And the location where Spiff is instantiated, ie the location in the OpenTable () function (instantiateAt attribute). Since the num_columns value is supplied when the snippet is instantiated, the invariantlntervalSet element has no fromValue or toValue attributes. Another important difference from the previous embodiment is the additional optimization recommendation for constant folding on column_definitions.

  Unlike static Spiff creation, dynamic Spiff is created at run time by inserting a Spiff compilation call into CreateTable () for the table Spiff.

Various types of Spiff designs (Here we use examples from the open source Postgres DBMS)
Predicate query Spiff
This Spiff is used by evaluating both general predicates in queries such as o_orderdate> = date '19940801' and join predicates such as o_orderkey = l_orderkey.

These predicates are evaluated by the ExecQual () function (Postgres). Specifically, the predicate is usually represented by a linked list. ExecQual () is applied repeatedly through this list and calls a specific evaluation function corresponding to each predicate. The code excerpt (PG9.3stock, src / backend / executor / execQual.c: 5125) presented to Example 20 shows such logic.

The evaluation function for each predicate is stored in a variable term. For each predicate of the form a> b, there are three components: operand # 1, operator, and operand # 2. In Postgres, the operator is evaluated by the function ExecEvalOper. This function (see Example 21) essentially performs a search according to the type of operator and fetches the address of the actual type-specific comparison function. ExecEvalOper () requires that the operands be stored in a separate linked list. In many cases, the length of this list is 2. The following is an example of specialization of the function in these cases.

  ExecEvalOper () is optimized by searching the comparison function only once. Then store the different functions in xprstate.evalfunc. Also, the function is called once and used as a predicate. Subsequent evaluation of the operator is performed by ExecMakeFunctionResultNoSets () (for scalar predicates considered within the current specialization scope).

  After that, ExecMakeFunctionResultNoSets () is repeatedly applied through the list of operands by calling the argument extraction function for each operand.

ExecEvalExpr is src / include / executor / executor. h: Macro defined in 72 as follows.
#define ExecEvalExpr (expr, econtext, isNull, isDone) \
((* (expr)> evalfunc) (expr, econtext, isNull, isDone))

  Therefore, if the operand is a constant, ExecEvalConst () is called, and the comparison function is called last.

  The bottlenecks found in predicate evaluation are firstly a loop that is applied iteratively through only two elements in the operand list, and secondly the extraction of individual operands. Specifically, for general predicates, one operand is usually a table column and the other operand is a constant. In such cases, the value (or address) of the constant can be “stored” directly in the code without having to call and fetch multiple functions. In the original implementation, it is necessary to extract the column ID of the table-derived operand by a plurality of function calls. Similarly, this column ID can also be stored directly in the specialized code.

For a join predicate, both operands are non-constant. The starting point of the operand can be one of three types: INNER_VAR (I), 0UTER_VAR (O), and scantuple (S). The starting point of the operand is also an invariant value based on the query. By grasping this invariant value, the routine for extracting the actual operand value can be further simplified. Theoretically, nine combinations are conceivable as starting points of the two operands, but in reality, only the following combinations are possible.

Hash join query Spiff
File src / backend / executor / nodeHashjoin. In the function ExecHashJoin () defined in c, the variable node-> js.jointype is invariant for a given query. According to the query, one value is selected from the set {JOIN_ANTI, JOIN_SEMI, JOIN_LEFT, JOIN_INNER}.

  In the same file and function, the variable List * joinqual is also unchanged for a given query.

  The hash join query Spiff reduces the number of if statements by removing all branches of the code, and more importantly reduces the size of the code.

The analysis needs to be able to handle complex data structures with pointers and heap allocation structures. For example, in order to remove the if statement in the for loop body of ExecHashJoin (), it is necessary to be able to determine the following expression (Example22).

Page Spiff
The page Spiff uses an invariant value in a disk / memory page managed by the DBMS for data storage. Such invariant values often include the number of rows stored on the page, the residual free space, and the empty page. In the Postgres page scan routine, there are additional invariant values such as scan direction and scan mode (pageatatime).

  More interestingly, page Spiff may enable more aggressive optimization. For example, page Spiff can optimize the data locality by recognizing the data layout once the page is read into memory. Also, if the data layout changes, instead of following the existing function call sequence in a separate process, Page Spiff can similarly improve instruction locality by exercising these calls one block at a time. it can.

  The page Spiff can specialize a long call sequence that declines data for some time and eventually results in data access, and can specialize all the code for many of the called functions.

The main advantage of page Spiff is that the invocation of the called function can generate a single specialized function that can fit into the instruction cache. Once this conversion is done, the other three mutually exclusive conversions can be used.
1. Active activation of the GetColumnsToLongs () spec code routine (as soon as the compressed tuple is extracted from the page, convert it to a decompressed tupletableslot and store it in the array that the spec code routine operates on).
2. Active partial decompression (calculates the maximum sequence required for the code that calls the specialized code and decompresses up to this maximum sequence)
3. Mild decompression (performs multiple decompression operations wherever the original code calls GetColumnsToLong ()).

  In this modified example, the determination is made using the selectivity of options. If the selectivity is high, it means that only a few lines are referenced, so use a mild decompression before applying the predicate.

  In general, it is best to place a call to GetColumnsToLong () so that instruction cache locality can be maximized by execution.

Aggregate query Spiff
Aggregation Spiff is designed to improve the efficiency of SUM and AVG aggregation functions. In particular, in evaluating aggregate functions with numeric data types, it has been found that Postgres incurs significant overhead in performing memory allocation and deallocation. In particular, Aggregation Spiff avoids such memory management overhead.

  In Postgres, a numeric type is represented by a byte string, and each digit is stored in an array of NumericDigit. According to this expression, although very precise control is possible, it is necessary to execute an arithmetic operation based on a character string, and performance is sacrificed.

  In a general implementation, it is necessary to execute memory allocation for each row because the number of digits existing in the value of each row may be different for each input row. In particular, in the evaluation of a * b, the resulting value range may greatly exceed the input value range. Nevertheless, in Postgres, there is a constant (NUMERIC_MAX_PRECISION) that defines the maximum number of digits that can be handled with respect to numerical values. Aggregation Spiff uses this value to perform slab allocation of the Spiff data part, but then reuses this by computing the corresponding aggregation function across all input rows, and allocates memory for each row. Eliminate.

  The evaluation of the aggregate function consists of two steps. For example, assuming the aggregate function SUM (a + b), in the first step, the result of the expression a + b is evaluated. In the second step, the value of a + b is accumulated for all input rows. In PostgreSQL, both the a + b and SUM () functions are evaluated by using the numeric_add () function. This function takes two inputs. For a + b, the two inputs are a and b, respectively. For the operation of SUM (x), the second input is x, which can essentially come from the scan row. The first input is a transition value that is the sum of the current row processed up to this point.

Evaluation of SUM ()
According to numeric_add (), two inputs are added, and the result value is copied to the return res variable allocated by make_result (). Then, nodeAgg. After res is returned to advance_transition_function () in c and this return value is copied to pergroupstate-> transValue, the return value becomes free. The next advance_transition_function () is executed to process the next line, and transValue is copied to the first input value of numeric_add () via the following snippet.
fcinfo-> arg [0] = pergroupstate->transvalue;
fcinfo-> argnull [0] = pergroupstate->transValueIsNull;

  This logic shows that transValue can actually be shared without having to be free across all rows. Therefore, in the case of the data part of EvaluateNumericAddSpiff, the necessary variables, ie, agg_temp_values-> result_value and agg_temp_values-> result_arg are assigned by using AllocateAggTempValues () at the beginning of the aggregation evaluation (note that these two variables are the same) Although it represents a value, in Postgres, these two variables need to be a return value and a temporary arithmetic argument, respectively).

Expression Evaluation As mentioned above, another use of numeric_add () is for arithmetic expressions such as a + b. In this case, the variable that stores the evaluation result of the expression is reused, but this is assigned first by make_result (). This variable is added to the Spiff data part as agg_temp_values-> expr_result_arg.

  Unlike the case of the evaluation of SUM () where the first input is directly derived from agg_temp_values-> result_value in the Spiff data part, both inputs in the evaluation of a + b need to be obtained using the existing Postgres implementation. Is a variable. In fact, in the evaluation of a + b, exeQual. numeric_add () is called from ExecEvalOper () in c. Therefore, as with the predicate Spiff, a Spiff (EvaluateAggregateExpression) that specializes the ExecMakeFunctionResultNoSets () function is created. And this Spiff calls the expression evaluation version of EvaluateNumericAddSpiff.

  In addition to +, the expression may include other operators such as-, *, and / or. The function that evaluates these operators is specialized as well as numeric_add ().

The following invariant values are considered as a summary of EvaluateNumericAddSpiff.
1) Side / execution path that calls numeric_add (). This is the execQual. c expression or nodeAgg. It can be derived from the evaluation of the SUM () function of c.
2) In evaluating the expression, the memory location of the result value can be made unchanged.
3) In the evaluation of SUM (), the memory location of the result value and the memory location of the first input can both be unchanged. These two variables can also share the same memory location.
4) Sharing a common memory segment across all rows is made possible by constants that indicate the highest precision bounds of numeric data types.

String matching Spiff
For example, there is a C function match that matches a character string x with another character string pattern y (including special characters such as wild cards). If the character string y (considered as a query constant) is known before the query is executed, a specialized function for matching an arbitrary character string against this specific character string pattern can be created.

As a specialization method, first, the following query specialization code (specification code) is created:
Code for a constant string of length 1 to 32
% Code for character string of query string And, by arranging various combinations of these specification codes in a line, it is possible to construct a specialized function that matches the character string against y. For example, suppose that there is a pattern “% abc% defg%”, and it is desired to create a specialized function for matching an arbitrary character string. Therefore, the following specification codes are arranged in a line:
% Specification code Three-character specification code to match "abc"
% Specification code (can be the same as the first code)
A 4-character specification code to match "defg",
End% spec code Each of these spec codes assumes that many characters in the string will remain after the collation is complete. When collation of one of the specification codes is finished, the remaining character string is passed to the next specification code in the column and the collation process is continued.

  Collation of certain parts of a string is achieved by using a combination of longlong, long, short, and char.

  By using the SpiffID stored as a local variable, given an arbitrary query string, it is easy to instantiate a series of query Spiff function pointers that each call the next stage with the start of the query Spiff. is there.

The following (Example 23) shows an example of a state of implementation for the character string “% abc% defg%” (using the pseudo code).

Once these specification code routines are created, a string is matched against this pattern by constructing a series of function calls as an array. The array looks like Example 24.

  These functions are sequentially called to collate character strings. Since a certain part with a length exceeding 32 can be divided into segments, a string of length 65 requires three instantiation specification codes of 32 characters, 32 characters, and 1 character. .

  More generally, there are methods where some arguments are immutable. These invariant values become part of the if statement, such as recursive calls and loops, deterministic. This sequence is expanded by a series of specification codes that are mutually called. This makes it a general transformation that works with loops and recursive and non-recursive calls.

  Since the actual set of specification codes to be called is not known until runtime (when the actual pattern to be matched is available), the Spiff instantiator fills an array of function pointers that specify the set of specification codes to be called It is also possible.

Spiff ranking by query In Spiff ranking by query, meta-Spiff is used to transform the existing specification code into analogs output by the compiler by examining the interpreted data structure and hot swapping. . In some embodiments, the switch / case block is converted into specialized code that can be combined at runtime according to the relationship between the various cases by using a hot swapping mechanism. Specifically, this is a case where a case is followed by another specific case during execution. Hot swapping replaces branch-based dispatcher calls with jumps to the target branch. This also applies to general dispatchers and interpretive execution models. Instead of interpreting the query plan and calling the corresponding plan node specific function, given the plan tree, all dispatcher calls can be replaced with a direct jump to the child plan node.

Storage of specialization codes When specialization is invoked, specialization codes (specification codes) may be generated and stored in various locations along the domain specialization process. For example, the specification code may include invariant values from both the oil field data 220 and the oil field simulator 230. The specification code may include invariant values from both the setting parameter 210 and the oil field simulator 230. In some embodiments, the specification code is stored in the Linux® operating system 230 and may include invariant values from the simulator and oil field data. In some embodiments, the specification code may be stored in an external router or external cloud service.

  The specification code stored in the simulator may be derived from the oil field data and the simulator, or may be identified by basic area specialization. Other Spiffs are stored with the operating system, router, cloud, and specialized code included in the designated application. In some embodiments, the specification code may flow from a storage location to a location where it can be called (an application that provides specialization candidates for later specialization). For example, the oil field data may store a specification code to be called in an external router. In some embodiments, a specification code identifier can be present with the data or application, and can be included in communication with subsequent applications to indicate the associated specification code to be called (later).

  FIG. 3 is a diagram illustrating domain specialization that refines the computer science paradigm according to an exemplary embodiment provided by the present disclosure. The figure includes four quadrants 310, 320, 330, and 340 for each scenario of data representation as data, code representation as data, data representation as code, and code representation as code.

  In the early stages of computer architecture from the garbage machine to the 1930s, data was distinguished from code. Data is manipulated, and the program code was instructions on how to manipulate the data to validate the operation. This can be represented as data represented in binary format in computer memory or storage, ie data stored as data and code represented as source code or code represented in some other way (eg, patch code), as shown in FIG. The quadrant 310 of FIG.

  In the 1940s, John von Neumann proposed a revolutionary architecture that stores machine code programs in computer memory as numbers, mixed code, and data (in fact, code can be manipulated as data and program execution It can be changed even if inside.) This architecture is shown in quadrant 320, where codes (machine language instructions) are represented as data.

  In the 1960s, there were several initial developments that led to code combinations in the form of Lisp functions with data (eg, the value of one of the arguments), which was paired with the value of that parameter. A function (code) that produces a Lisp continuation and has one less argument. This is a special way of storing / encapsulating data in code, as shown in quadrant 330.

  In the 1980s, a postscript language was invented. This is code that generates an image when executed. As shown in the quadrant 320, the postscript is a code as data generated by a formatter that obtains a document such as a Microsoft® word file as data and converts it into a program again. Since the PostScript file generated from the Microsoft® word file is not a directly printed image but an instruction for drawing each character of the document, the program is executed on a PostScript printer or converted to PostScript, for example. It is also possible to generate a bitmap image of a document by execution by a program.

  Domain specialization is a further advancement of this concept. In area specialization, invariant values, that is, data are acquired, and using these values, a special code version of a part of an application such as DBMS, which is executable code, is generated. For this reason, the relationship specification code is a result of specialization of the DBMS code using the relationship (data) schema. The tuple specification code is a result of using a data value in a tuple (table row). The O / S specification code is a specialization of the snippet based on a specific data value of a specific invariant value in the operating system snippet and is the same as the router specification code.

  The data representation as this code (shown in quadrant 330) can be generated from one application or another application snippet in one application, can be passed between applications, and depending on the target application as needed. Callable. Domain-specific technology provides a means to identify when such specification codes are valid when performance increases, when to generate them, invariant values to specialize, how to communicate between applications, and when to call them. provide.

  For any coherent region in the data file, determine invariant values in that region, track these values up to the application code area, construct a specification code from these areas, and then change these specification codes It is suggested that it can be associated again with the region. This viewpoint therefore focuses on the original data, rather than starting with code and specializing it.

  The above-described embodiments of the present disclosure, and in particular any “preferred” embodiments, are merely possible examples of implementations and are presented only for the purpose of providing a clear understanding of the principles of the present disclosure. It should be emphasized. The above-described embodiments of the present disclosure are capable of many variations and modifications without departing substantially from the spirit and principles of the present disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims (15)

A computer-implemented method for improving the performance of computer program code comprising:
Identifying invariant intervals of variables in the computer program code based on a program representation (PR), ie, an implementation of an abstract syntax tree or the like of the computer program code;
Estimating a program interaction within the computer program based on the PR and an ecosystem specification of the computer program;
Estimating a domain assertion based on the PR, the identified invariant interval of a variable in the computer program code, and the estimated program interaction;
One or more based on the invariant interval of variables in the computer program code, the PR, one or more execution summaries associated with the computer program, the estimated program interaction, and the estimated domain assertion Identifying a candidate snippet of;
Generating specialized computer program code based on the one or more candidate snippets;
Modifying the computer program code based on the generated specialized computer program code;
Hiding the specialized computer program code;
A computer-implemented method comprising: (A) the identified invariant interval spans multiple runs;
(B) the identified invariant interval includes at least one set of invariant intervals of a particular variable, and all invariant intervals in the set share the same start node;
The computer-implemented method of claim 1, comprising at least one of the following:   Each of the one or more candidate snippets comprises (a) a code interval identified by the PR or (b) a set of invariant values and a set of possible values for each variable. A computer-implemented method described in 1.   The one or more candidate snippets each include an appropriate lifetime of the candidate snippet, and each of the one or more candidate snippets is a recommended optimization employed within the appropriate lifetime of the candidate snippet The computer-implemented method according to claim 1, wherein the method is preferably included.   The step of generating specialized computer program code comprises: (a) creating the specialized computer program code by inserting the code at a suitable location of the computer program, calling the specialized computer program code; Breaking the code or (b) creating a specialization function that matches any string against a given string pattern, using a metaspecifier to interpret the data structure and hot Considering swapping, converting existing specialized computer program code, calling a query, reducing the size of the computer program code by removing branches of the computer program code, and using numerical values The slab allocation of the in-region specializer (Spiff) data portion, reusing the Spiff data portion by computing the corresponding aggregate function across all input rows in the computer program code, Removing a line-by-line memory allocation, wherein the numerical value is defined by the maximum number of digits that can be supported by the line, or using an invariant value in a disk or memory page on which the computer program is stored. 5. A computer-implemented method according to any one of claims 1 to 4, comprising reconstructing a data layout and optimizing data locality once the page is read.   The specialized computer program code is created at run time and called later, optionally further comprising determining whether a violation of the identified invariant interval occurs in a given run. The computer-implemented method according to any one of 5 A system configured to improve the performance of a computer program comprising:
An invariant searcher for identifying invariant intervals of variables in the computer program based on a program representation (PR) of the computer program;
An interaction estimator for estimating program interactions within the computer program based on the PR and ecosystem specifications of the computer program;
A domain assertion estimator that estimates domain assertions based on the PR, the identified invariant intervals of variables in the computer program code, and the estimated program interaction;
One based on the invariant interval of variables in the computer program code, the PR, one or more execution summaries associated with the computer program, the estimated program interaction, and the estimated domain assertion; Or a snippet searcher that identifies multiple candidate snippets;
An intra-region specializer (Spiff) composer that generates specialized computer program code based on the one or more candidate snippets;
A specialized source that receives the generated specialized computer program code and modifies the computer program code based on the generated specialized computer program code;
With a system.   The system according to claim 7, wherein the invariant searcher performs a static analysis on the PR to identify invariant intervals.   9. A system according to claim 7 or 8, wherein the identified invariant interval comprises at least a set of invariant intervals for a particular variable, and all invariant intervals in the set share the same starting node.   10. A system according to any of claims 7 to 9, further comprising an invariant inspector that determines whether a violation of the identified invariant interval occurs in a given execution.   11. A system according to any of claims 7 to 10, wherein each of the one or more candidate snippets includes a set of invariant values and a set of possible values for each variable.   The one or more candidate snippets each include an appropriate lifetime of the candidate snippet, and each of the one or more candidate snippets is a recommended optimization employed within the appropriate lifetime of the candidate snippet 12. A system according to any of claims 7 to 11, preferably comprising:   The Spiff Configurator inserts code into the computer program in place to create the specialized computer program code, call the specialized computer program code, and destroy the specialized computer program code; Existing specialized computers by creating specialized functions that match arbitrary strings against string patterns, using meta-specializers, and interpreting interpreted data structures and hot swapping Converting program code, combining query invariant values and raw invariant values that are valid in a query by schema, reducing the size of the computer program code by removing branches of the computer program code, Use within the area Slab allocation of the data unit (Spiff), and by reusing the Spiff data unit by computing the corresponding aggregate function over all input rows in the computer program code, memory allocation in units of rows By removing, the value is defined by the maximum number of digits that can be supported in line units, or the page is read using an invariant value in a disk or memory page where the computer program is stored. 13. A system as claimed in any of claims 7 to 12, further configured to reconfigure the data layout when done and optimize data locality. When executed by the processor of the computer equipment:
Determining a variable in said computer program whose value is invariant within an identified invariant interval by means of a static analysis on a program representation (PR) of the computer program;
Creating a specialized computer program code at a suitable location of the computer program, calling the specialized computer program code, and destroying the specialized computer program code, wherein the specialized computer program code is at least the Created based on the determined variables;
When the specialized computer program is invoked, modifying at least a portion of the computer program with the specialized computer program code;
A persistent computer-readable medium containing computer-executable instructions for causing the computer equipment to perform the operation.   The persistent computer readable medium of claim 14, wherein generating the specialized computer program code is further based on at least one of domain specific knowledge and preliminary source knowledge associated with the computer program.

Images