JP2011113148A - Macro development method and preprocessor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a macro development method allowing flexible and simple mounting of macro development processing including development of a multistage macro call in a preprocessor. <P>SOLUTION: The preprocessor executes: step S102 of cutting out a first token from a character string of a source code and generating token arrangement; step S104 of sequentially reading the first tokens into a buffer from the token arrangement; step S109 of performing output as a processing result when the first token inside the buffer is not a part corresponding to the macro call; step S111 of replacing the part corresponding to the macro call by contents of a corresponding macro definition and performing the development, when the first token is the part corresponding to the macro call and is completed as the macro call; step S112 of cutting out a second token from a character string obtained by developing the macro call; and step S113 of returning the second token to the head of the toke arrangement of input. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a macro expansion technique for source code described in a programming language or the like, and more particularly to a technique effective when applied to a macro expansion method and a preprocessor including a multi-stage macro expansion process.

  In general, a compiler performs lexical analysis, syntax analysis, semantic analysis, and the like on a source code described in a programming language, and then generates an object code through processing such as optimization. Here, in general, in a compiler, before performing lexical analysis on the source code, the macro expansion in which the macro call in the source code is expanded by the macro definition defined in the source code or the header file by the preprocessor. Preprocessing including processing is performed.

  Here, in the macro call, there are cases where the macro call is described in a multi-stage nested structure, for example, an input parameter in the macro call is further described by the macro call.

  At this time, when the preprocessor expands the macro call, generally, as described in Non-Patent Document 1, for example, after expanding the macro call, the macro is further expanded with respect to the newly expanded text. By scanning whether or not there is a call, it is possible to expand a macro call having a multi-level nested structure. In such a multistage macro call, it may differ depending on the compiler whether to give priority to expansion from the inside of the nest or from the outside.

“RAD Studio Online Help C ++ Reference”, [online], EMBARCADERO TECHNOLOGIES, [Search November 2, 2009], Internet <URL: http://docwiki.embarcadero.com/RADStudio/en/Define>

  Here, for example, when considering a code check tool that checks whether or not the source code description conforms to programming rules and rules, the code check tool is generally expanded in the same way as a compiler. In addition, processing such as lexical analysis and syntax analysis is performed, and a function of performing various analyzes on the processing results is provided. At this time, for example, the preprocessor of the code check tool desirably has the same processing specifications as the preprocessor of the compiler in the real environment used for generating an executable program from the source code.

  However, in the conventional preprocessor, for example, the priority order of the expansion of the multi-stage macro call as described above is not implemented so that it can be changed. Therefore, a preprocessor in a code check tool or the like needs to be mounted flexibly so that the priority order in the development of multi-stage macro calls can be changed as appropriate according to the specifications of the compiler in the real environment.

  At that time, for example, there is a method of switching the processing contents based on the priority order by specifying the processing option at the time of execution for the preprocessor. Considering the productivity of maintenance, it is desirable to be as simple as possible and to be efficiently implemented with less processing overlap.

  On the other hand, as for the processing method itself of the multistage macro expansion in the conventional preprocessor, as described above, after expanding the macro call, whether or not there is a further macro call for the expanded text (whether macro expansion is possible). Scan) is generally performed, and pre-processor implementation is poor in efficiency and flexibility.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a macro expansion method capable of flexibly and simply mounting macro expansion processing including expansion of multi-stage macro calls in a preprocessor, and a preprocessor equipped with the macro expansion method. The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  Of the inventions disclosed in this application, the outline of typical ones will be briefly described as follows.

  A macro expansion method according to a representative embodiment of the present invention is a macro expansion method for expanding a macro call in a source code by a macro definition by a computer program, and is characterized by executing the following steps. is there.

  That is, the computer program generates a token array by cutting out a first token from a character string of the source code by lexical analysis, and inputs the token array in order from the token array with the token array as an input. A step of reading into the buffer, and determining whether each of the first tokens in the buffer is a part corresponding to the macro call, and if not, the part in the buffer Outputting each first token as a processing result and clearing the buffer.

  Further, if each of the first tokens in the buffer corresponds to the macro call, whether or not all or part of the first tokens in the buffer are completed as the macro call. And when the macro call is completed, a step of replacing the portion completed as the macro call with the content of the corresponding macro definition and expanding, and a character string obtained by expanding the macro call A step of cutting out a second token by lexical analysis from the above, and a step of returning each cut out second token to the beginning of the token array of the input and clearing a portion where the macro call is expanded in the buffer It is characterized by doing.

  Among the inventions disclosed in the present application, effects obtained by typical ones will be briefly described as follows.

  According to the representative embodiment of the present invention, the character string generated by the macro expansion is returned to the input character string, thereby simplifying the macro expansion processing algorithm in the preprocessor, the flexibility of the macro expansion processing, the preprocessor Development and maintenance productivity can be improved.

It is the flowchart which showed the example of the macro expansion | deployment process in the preprocessor which is one embodiment of this invention. It is the figure which showed the example of a process of the multistage macro expansion | deployment in one embodiment of this invention. It is the figure which showed another processing example of the multistage macro expansion | deployment in one embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

  A preprocessor according to an embodiment of the present invention includes a macro expansion process for expanding a macro call in a source code by a macro definition defined in the source code, a header file, or the like, as in a general preprocessor. It is a software program that performs processing and outputs a preprocessed source code as a processing result.

[Macro expansion processing flow]
FIG. 1 is a flowchart showing an example of macro expansion processing in the preprocessor according to the present embodiment. First, the preprocessor generates a character array by cutting out characters from the source code (S101). Next, lexical analysis is performed on the source code (character array) to cut out tokens to generate a token array (S102). Note that the method of cutting out characters from the source code and tokenizing them by lexical analysis is the same as that of a general preprocessor, and thus description thereof is omitted here.

  Next, a loop process is started in which the tokens in the token array 110 cut out in step S101 are sequentially input and the process is repeated (S103). In the loop processing, first, the target token is read into the macro expansion processing buffer (S104). At this time, if a token already exists in the buffer, it is read in a form linked to the rear end. The buffer is a data area provided on the memory of a computer system in which the preprocessor operates during macro expansion processing.

  Next, the preprocessor determines whether or not the token in the buffer corresponds to the macro definition (S105). If the portion corresponds to the macro definition, it is determined whether or not the token in the buffer is completed as the macro definition (S106). If the macro definition is completed, the contents of the token in the buffer (macro definition) are output to the macro table and the buffer is cleared (S107). Note that the token content (macro definition) output to the macro table may be output as an extended character in the same manner as in step S109 described later for use in later analysis.

  If the token in the buffer is not completed as a macro definition in step S106, the processing proceeds to the next token (S113, S103). Since the macro definition needs to be described before the macro call in the source code, the macro definition is usually held in the macro table by the above process before the subsequent macro expansion process is executed. Will be.

  If the token in the buffer does not correspond to the macro definition in step S105, next, whether or not the token in the buffer is a part corresponding to the macro call, that is, whether or not a reference (call) to the macro definition is included. Is determined (S108). If it does not correspond to the macro call, the token in the buffer is output as the result of the macro expansion process, the buffer is cleared (S109), and the process moves to the next token (S113, S103). When a token in the buffer is output as a processing result, the token is output in a form linked to the trailing end of the already output result.

  If the token in the buffer corresponds to the macro call in step S108, then whether or not all or part of the token in the buffer is completed as a macro call and can be expanded by the macro definition. Is determined (S110).

  Here, when deciding whether or not macro expansion is possible, whether to expand from the inside of the nest or from the outside in the multistage macro call according to the instruction content to the preprocessor It is desirable to be able to switch. That is, when preferentially expanding from the outside, the tokens in the buffer are determined from the top, and when the connection of each token is completed as a macro call (that is, from the top in the buffer in the processing flow of FIG. 1). When all tokens up to the rear end are completed as macro calls), it is determined that macro expansion is possible.

  On the other hand, when preferentially expanding from the inside, the tokens in the buffer are judged from the back end, and the concatenation of tokens (which may be some tokens in the buffer) is completed as a macro call. If it is, it is determined that macro expansion is possible for the token portion. The preprocessor is instructed as an option at the time of execution of the preprocessor, or specified in advance in a setting file, etc. It can carry out by various methods.

  If all or some of the tokens in the buffer are not completed as a macro call and cannot be expanded in step S110, the process proceeds to the next token (S113, S103). On the other hand, if the macro expansion is possible, the token of the part completed as the macro call in the buffer is replaced with the token corresponding to the macro definition and expanded (S111). Further, the macro expanded portion token is returned to the head of the input token array, the macro expanded portion buffer is cleared (S112), and the processing of the next token is started (S113, S103).

  The above process is repeated until the input token array (including the token returned in step S112) becomes empty, and the macro expansion process ends. As a result, it is possible to obtain an output as a result of performing macro expansion on the source code (all tokens in the token array 110).

  As described above, in this embodiment, a character string obtained as a result of macro expansion in the buffer is not output as it is, but is returned to the input in consideration of multi-stage macro expansion. As a result, the macro expansion process can be implemented without requiring a process of re-scanning the character string that has been macro expanded in the buffer and determining the macro call. In addition, since it is possible to minimize different processing depending on whether priority is given to expansion from the inside of the nest or from the outside in multistage macro calls, the macro expansion processing must be implemented simply and flexibly. Can do.

[Specific examples of macro expansion]
FIG. 2 is a diagram showing an example of multi-stage macro expansion processing according to the present embodiment. In the example of FIG. 2, an example is shown in which multistage macro call is preferentially expanded from the outside of the nest. That is, when the token read into the buffer is determined from the top and the connection of each token is completed as a macro call (that is, all tokens from the top to the back end in the buffer are macros in this embodiment). When it is completed as a call), it is determined that macro expansion is possible.

  In the figure, a source code 100 is a source code having a multi-stage macro expansion described in C language. Note that the input source code 100 is not limited to that written in a programming language such as C language or a tag language such as XML (eXtensible Markup Language), but can be applied to any source code having macro definition and macro call. is there. The source code 100 may be composed of a plurality of files, and may include, for example, a header file having a macro definition.

  The right side of the third line of the source code 100 calls the macro definition “MUL (x, y)”. At this time, the macro definition “NN” is called as the input parameter “x”. have. This expression is divided into tokens as shown in the figure, for example, by the processing in steps S101 and S102 of FIG. The preprocessor performs macro expansion using the token array 110 as an input. At this time, the macro definitions (“MUL (x, y)” and “NN”) on the first and second lines of the source code 100 are stored in the macro table by the processes in steps S105, S106, and S107 in FIG. Shall be registered.

  First, the first token “MUL” of the input token array 110 is read into the macro buffer 10 which is a buffer for macro expansion processing ((1) in the figure). At this time, the token “MUL” in the macro buffer 10 is a portion corresponding to the macro call (that is, a portion corresponding to the macro definition “MUL (x, y)” registered in the macro table), and Since the macro call is not completed (the macro cannot be expanded), the processing of the next token is started without outputting the contents of the macro buffer 10 ((2) in the figure).

  In the subsequent processing, tokens are sequentially read from the input token array 110 and connected to the rear end of the macro buffer 10, but reading is continued because the connection of tokens in the macro buffer 10 is not completed as a macro call. After that, when the token “)” is read, the token concatenation “MUL (NN, 3)” from the beginning in the macro buffer 10 is completed as a macro call, and macro expansion is possible (( 3)).

  Therefore, the preprocessor expands the token concatenation “MUL (NN, 3)” with the content “x * y” of the macro definition “MUL (x, y)” to obtain the character string “NN * 3”. Thereafter, the character string “NN * 3” is divided into tokens “NN”, “*”, and “3” by the process of step S112 in FIG. The cut out tokens are not output, and are returned to the top of the input token array 110 in consideration of multi-stage macro expansion by the processing of step S113 in FIG.

  As processing for the next token, the preprocessor reads the first token “NN” of the input token array 110 into the macro buffer 10 ((4) in the figure). At this time, the token “NN” in the macro buffer 10 is a portion corresponding to the macro call, and is completed as the macro call and can be expanded ((5) in the figure). Therefore, the preprocessor expands the token “NN” by the macro definition to obtain the character string “10”, and cuts it out as a token. The cut out token is not output, and is returned to the top of the input token array 110 in consideration of multi-stage macro expansion.

  As processing for the next token, the preprocessor reads the first token “10” of the input token array 110 into the macro buffer 10 ((6) in the figure). At this time, since the token “10” in the macro buffer 10 does not correspond to the macro call, the preprocessor outputs the contents of the macro buffer 10 (token “10”) as a macro expansion result, and the next token is the macro. Read into the buffer 10 ((7), (8) in the figure). Since all subsequent tokens do not correspond to macro calls, they are output as they are, and after processing all tokens in the token array 110, a character string “10 * 3;” can be obtained as a result of macro expansion processing.

  On the other hand, FIG. 3 is a diagram illustrating another example of multi-stage macro expansion processing according to the present embodiment. The example of FIG. 3 shows an example in the case of preferential expansion from the inside of the nest in a multi-stage macro call. That is, the token in the macro buffer 10 is determined from the rear end, and when the token concatenation is completed as a macro call, it is determined that macro expansion is possible for the token portion.

  The processes up to (1) and (2) in the figure are the same as those shown in (1) and (2) of FIG. Thereafter, the tokens are sequentially read from the input token array 110 and connected to the rear end of the macro buffer 10. When the “NN” token is read, the rear end token “NN” in the macro buffer 10 is It is completed as a macro call and can be expanded ((3) in the figure). Therefore, the preprocessor expands the token “NN” by the macro definition to obtain the character string “10”, and cuts out the character string “10” as a token by the process of step S112 in FIG. 1 as in the example of FIG. . The cut out token is not output, and is returned to the top of the input token array 110 in consideration of multi-stage macro expansion in step S113 of FIG.

  As processing of the next token, the preprocessor reads the first token “10” of the input token array 110 into the macro buffer 10 ((4) in the figure). At this time, the token “MUL (10” in the macro buffer 10 is a portion corresponding to the macro call and is not completed as the macro call, so the next token is not output without outputting the contents of the macro buffer 10. (5 in the figure).

  In the subsequent processing, tokens are sequentially read from the input token array 110 and connected to the rear end of the macro buffer 10, but reading is continued because the connection of tokens in the macro buffer 10 is not completed as a macro call. Thereafter, when the token ")" is read, the token concatenation "MUL (10, 3)" from the rear end in the macro buffer 10 is completed as a macro call, and macro expansion is possible (in the figure). (6)).

  Therefore, the preprocessor expands the token concatenation “MUL (10, 3)” with the content “x * y” of the macro definition “MUL (x, y)” to obtain the character string “10 * 3”. Thereafter, the character string “10 * 3” is divided into tokens “10”, “*”, and “3” and cut out. The extracted tokens are not output, but are returned to the top of the input token array 110 in consideration of multi-stage macro expansion. The subsequent processes (7) to (10) are the same as those shown in (6) to (9) of FIG.

  As described above, according to the macro expansion method of the preprocessor according to the embodiment of the present invention, a multi-stage macro expansion is performed without outputting the character string expanded in the macro buffer 10 as it is. Therefore, it is not necessary to implement a process of performing re-scanning on the character string expanded in the macro in the buffer and determining the macro call.

  In addition, whether or not macro expansion is possible for a token in the macro buffer 10 (depending on whether the macro call is completed) depends on whether priority is given to expansion from the inside or the outside of the nest in multi-stage macro calls. Whether or not) is determined from the beginning or the rear end of the macro buffer 10 can be suppressed to the minimum necessary processing difference. As a result, the macro expansion processing can be implemented simply and flexibly in the preprocessor.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

  For example, the macro expansion method according to the present embodiment is not limited to the preprocessor described above, but can be applied to various software programs that perform macro expansion processing for expanding macro calls by macro definition.

  The present invention can be applied to a macro expansion method and a preprocessor including a multi-stage macro expansion process for a source code described in a programming language or the like.

  10 ... Macro buffer, 100 ... Source code, 110 ... Token array.

Claims (5)

A macro expansion method for expanding a macro call in a source code by a macro definition by a computer program,
The computer program is
Cutting out a first token from the character string of the source code by lexical analysis to generate a token array;
Reading the first token into the buffer in order from the token array, using the token array as an input;
It is determined whether each of the first tokens in the buffer is a part corresponding to the macro call, and if it is not a part corresponding to the macro call, the first tokens in the buffer are Outputting the processing result and clearing the buffer;
Whether or not all or a part of each first token in the buffer is completed as the macro call when each first token in the buffer is a portion corresponding to the macro call. Determining, if completed as the macro call, replacing the portion completed as the macro call with the content of the corresponding macro definition, and expanding,
Cutting out a second token from a string obtained by expanding the macro call by lexical analysis;
And a step of returning each cut-out second token to the beginning of the input token array and clearing a portion of the buffer where the macro call is expanded. The macro expansion method according to claim 1,
The computer program is
When determining whether all or a part of each first token in the buffer is completed as the macro call, all or a part of each first token from the top in the buffer The macro expansion method characterized by determining by. The macro expansion method according to claim 1,
The computer program is
When determining whether all or part of each first token in the buffer is completed as the macro call, all or one of the first tokens from the rear end in the buffer. A macro expansion method characterized in that determination is performed by a section. The macro expansion method according to claim 1,
The computer program is
When determining whether or not all or part of each of the first tokens in the buffer is completed as the macro call, each of the first tokens in the buffer is determined based on an instruction to the computer program. A macro expansion method characterized by switching between determination by all or part of a first token and determination by all or part of each first token from the rear end in the buffer. A preprocessor for executing a macro expansion process for expanding a macro call in a source code by a macro definition;
Cutting out a first token from the character string of the source code by lexical analysis to generate a token array;
Reading the first token into the buffer in order from the token array, using the token array as an input;
It is determined whether each of the first tokens in the buffer is a part corresponding to the macro call, and if it is not a part corresponding to the macro call, the first tokens in the buffer are Outputting the processing result and clearing the buffer;
Whether or not all or a part of each first token in the buffer is completed as the macro call when each first token in the buffer is a portion corresponding to the macro call. Determining, if completed as the macro call, replacing the portion completed as the macro call with the content of the corresponding macro definition, and expanding,
Cutting out a second token from a string obtained by expanding the macro call by lexical analysis;
And a step of returning each cut-out second token to the head of the input token array and clearing a portion of the buffer where the macro call is expanded.

Images