JPH02166522A - Data converting method - Google Patents

Abstract

PURPOSE:To improve the memory efficiency of a computer by performing a pattern matching job at each process for a syntax analysin which is applied to each part of an input symbol train and disusing the syntax analyzing trees that failed in the pattern matching jobs. CONSTITUTION:A recording means which records the data having a structure for each partial tree of a conventional syntax analyzing tree is provided. The pattern matching jobs are carried out based on the information which produces the data having structures and in each process for syntax analysis that is carried out to each part of a symbol train. Thus the data having structures are produced when the pattern matching jobs succeed. At the same time, the data having structures are recorded via the means which records the data having a structure for each partial tree of a syntax analyzing tree. Then the syntax analyzing tree is disused when a pattern matching job fails. In such a manner, a pattern matching job is carried out in each process for the syntax analysis which is performed to each part of a symbol train. Then the syntax analyzing trees which failed in the pattern matching jobs are disused. Thus the unnecessary syntax analyzing trees can be omitted and therefore the memory efficiency of a computer is improved.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a method for converting symbol strings by a data processing device, using conversion rules given from the outside. Concerning the conversion from symbol strings to structured data. Such conversion may be accomplished by machine translation or language processing, such as a compiler,
This is an essential function for interpreted programming language processing systems.

[Conventional technology]

Generally, in data conversion processing, intermediate data is provided during the conversion process, and conversion is performed in stages such as converting input data to intermediate data and converting intermediate data to target data. A tree structure is used as Using a tree structure as intermediate data has the advantage of ease of maintenance when converting data.

Generally, the process of generating intermediate data from input data is called parsing process, and the tree-structured data obtained as a result of parsing process is called parsing tree.

In syntactic analysis, one grammar is usually described by a set P of relations called production rules, a set T of terminal symbols, a set N of non-terminal symbols, and a starting symbol S. A production rule consists of a left side and a right side, and indicates that the symbol on the left side is rewritten by the symbol string on the right side.

A non-terminal symbol is a symbol that is rewritten by P,
A terminal symbol is a symbol that cannot be rewritten by P. S
is a non-terminal symbol, and if the input sentence conforms to the grammar, it can be rewritten into the input sentence by applying P to S.

One of the basic concepts of grammar is context-free grammar.

A context-free grammar is a grammar in which the left-hand side of each production rule is a non-terminal symbol, and the right-hand side is constrained to a string of 0 or more symbols. Each grammar rule in a 0-context-free grammar is A −+ a A E N , a E (N U
T ) Represented as a monkey. Here, () represents a sequence of zero or more arbitrary elements of the set.

There are two essentially different methods for parsing.

One method is to create a parser specifically for a given syntax rule. The configuration of the syntax analysis processing device using this method strongly reflects the given syntax rules, and in general, a more efficient and easy-to-use system can be constructed.

Another method is to design a general parser that is valid for all syntax rules. In this case, an individual syntax rule is input from the outside and the processing device operates on the basis of this syntax rule. Therefore, the parsing process using this method is characterized by a high degree of flexibility.

The latter data conversion method using the horizontal text analysis processing method is described in Japanese Patent Application No. 120201/1983, so an outline thereof will be explained here.

Associating the structure of output terms with terminal and non-terminal symbols (e.g. in the form of arguments) to the syntactic rules described in context-free grammars
The described conversion rules are received from the outside, and the structure of the input symbol string is parsed based on the conversion rules to generate a parse tree. Next, while separating the parse tree obtained as a result of the parsing process, output terms are generated according to the description of the structure of output terms related to terminal symbols and non-terminal symbols corresponding to the structure of the input symbol string.

Here, terms are defined as follows using constant symbols (function symbols) with arguments.

(1) A function symbol with 0 arguments is a term.

(2) f is a function symbol with n arguments, tl, t2. ...tn
If is a term, then f(t1.t2.-, tn) is also a term.

Further, in this specification, the description of the structure of the output term is referred to as information for generating the (output) term.

[Problem to be solved by the invention]

In the above conventional technology, first, a syntax analysis process is performed, and after all the syntax analysis processes are completed, a pattern match is performed using information that generates structured data while separating the syntax parse tree obtained as a result of the syntax analysis process. Generates structured data. Therefore, among the parsing trees generated during parsing processing, parsing trees that become unnecessary due to a failure in pattern matching after the parsing processing is completed are not included in the parsing process. It must be held until the tree pattern is matched, which impairs the efficiency of the computer's memory.

Means for Solving Problem C6] In the present invention, means for recording data having a structure in each subtree unit of the conventional syntax parsing tree is provided, and each step of the parsing process performed for each part of a symbol string is provided. Then, pattern matching is performed using the information to generate structured data, and if the pattern matching is successful, structured data is generated, and data with side paths for each subtree of the above parse tree is recorded. Data having the above structure is recorded by means, and if pattern matching fails, this parse tree is discarded.

[Effect]

The present invention performs pattern matching in each process of parsing processing performed on each part of a symbol string, and discards parsing trees that fail in pattern matching, thereby eliminating unnecessary inscription parsing trees. It disappears.

〔Example〕

FIG. 1 is a flowchart showing an overview of data conversion processing of the present invention.

First, input the conversion rule from the outside (101), input one symbol from the symbol string using the input section (102), and if it is the end of the symbol string (103), output the term (101).
4) End the process. If it is not the end of the symbol string, a parsing process is performed (105), a term is generated, the generated term is added to the term set (106), and the process returns to step 102.

FIG. 2(a) is an example of a hardware configuration for implementing the present invention. 21 is a control unit, 22 is an arithmetic unit, 23 is an input device, 24 is a storage device, and 25 is an output device. Further, dotted line arrows indicate the flow of control information, and solid line arrows indicate the flow of data.

FIG. 2(b) is a diagram showing the basic functional configuration of the present invention. In this case, 1 is the input symbol string, which is input from the input unit 2. Reference numeral 3 denotes an output term, which converts the symbol string input at the input unit 2 according to conversion rule 4, and outputs the generated term. Reference numeral 5 denotes a syntactic analysis unit, which is involved in syntactic analysis processing. The syntax analysis process is performed based on the conversion rules stored in the conversion rule storage section 8. The result of the parsing process is stored in the parsing tree storage section 9. The parse tree storage unit 9 also stores information for generating terms.

'Mt The sentence parse tree decomposition unit 6 decomposes the parse tree resulting from the parse process stored in the parse tree storage unit 9. The term generation unit 7 generates terms from the parse tree decomposed by the parse tree storage unit 6 using information for generating terms stored in the storage unit 9.

In this embodiment, the parsing process uses Ja
y Early) parsing algorithm is used.

Jay Earley's parsing algorithm is 19
Jay Barley's paper "An Efficient Concentration-Free Purging Algorithm J" published in CACM Vol. 13, No. 2, February 1970.
(An efficient Context F
ree Parsing AI Horithm), so an overview thereof will be explained here.

Using a marked production rule in which the symbol string on the right side of the production rule is divided by one mark, we introduce a state consisting of the above marked production rule and the symbol position of the input symbol string, and each time one symbol is read from the input symbol string, By organizing the generated states into sets, we manage the production rules that are being applied during the parsing process.

The state is that the symbol string to the left of the mark on the right side of the marked production rule is
It means that it is changed to a column of 1, and expresses the application status of the production rule.

The parse tree corresponding to this state has a string of pointers corresponding to the state, and corresponds to the symbol string on the left side of the mark on the right side of the marked production rule, and the state applied when rewriting each symbol. Expressed by pointing.

In context-free grammars, there are generally multiple production rules that are applied when rewriting a single symbol.

Collect pointers corresponding to the state. Collecting pointers corresponds to expressing them in an overlapping manner by sharing the nodes of multiple lock analysis books.

Earley's parsing algorithm allows parsing based on all context-free grammars, and when the grammar has ambiguity, it generates all parse trees that can be generated from a given input sentence without duplication of processing. do.

Furthermore, in this embodiment, the marked generation rule with information for generating structured data is called a conversion rule element.

FIG. 3 shows details of the conversion rule elements stored in the conversion rule placement section. A conversion rule element consists of a non-terminal symbol on the left side, a non-terminal symbol or terminal symbol to the right of the mark on the right side, a pointer to the next conversion rule element, information for generating an output term, and a pointer to an intermediate node collection. Become. Information for generating output terms is provided to intermediate nodes when they are generated in the process of parsing. The conversion rule element is static information that expresses the grammar, and is not generated, updated, or deleted during processing such as syntax analysis.

FIGS. 4(a) to 4(d) show details of the information stored in the parsing main storage unit.

401 is an intermediate node aggregator. An intermediate node aggregator is a set of an intermediate node connector and a pointer to an intermediate node aggregator belonging to another parse tree.

402 is an intermediate node connector. An intermediate node connector is a set of a pointer to an intermediate node or leaf and a pointer to an intermediate node aggregator belonging to the same parse tree.

403 is the details of the intermediate node. An intermediate node consists of the position of a symbol, a pointer to an intermediate node aggregator, information for generating an output term, and a pointer to a set of terms. The information for generating the output term is the information for generating the term of the conversion rule element that is provided in the intermediate node when generating the intermediate node in the process of syntactic analysis processing.

A pointer to a set of terms is a pointer to a set of terms whose elements are terms generated from each parse tree managed by the node.

404 is a leaf. The leaf is the position of the symbol.

It consists of information for generating output terms and a pointer to a set of terms. The set of terms pointed to by the pointer to the leaf term set is a set of terms that has only one term as an element.

FIG. 5 is a flowchart of the main routine of data conversion processing.

First, read the start symbol (501) and start procedure 1 with the start symbol and the position of the initial symbol as arguments (502
). Next, read one symbol from the input symbol string and the position of that symbol on the input symbol string (503). If it is the end of the input symbol string (504), start procedure 5 with the symbol position as an argument ( 505).

If it is not the end of the input symbol string, procedure 2 is started with the symbol and its position as arguments.

FIG. 6 is a flowchart of procedure 1.

First, find the conversion rule element that has the argument symbol on the left side6
01), if it does not exist (602), the process ends. Next, if there is an intermediate node with the position of the symbol of the argument (
603), return to step 601. If there is no intermediate node having the position of the symbol of the argument, generate the intermediate node (604).

Generate an empty intermediate node setter and add it to the set (605),
Invoke procedure 3 using the conversion rule element and intermediate node as arguments (606), and return to step 601.

FIG. 7 is a flowchart of Procedure 2.

First, a leaf is generated (701), and a conversion rule element having an argument symbol on the right side of the mark is searched for (702).

If it does not exist (703), the process ends.

Next, if the intermediate node symbol position does not connect to the argument symbol position (704), the process returns to step 702. If the position of the symbol of the intermediate node is connected to the position of the symbol of the argument, then
Obtain a pointer to the intermediate node aggregator (705), obtain the next conversion rule element (706), generate an intermediate node with the position of the symbol extended, and add it to the intermediate node set (70).
7).

Next, create an intermediate node connector by pairing the pointer to the leaf and the pointer to the intermediate node aggregator (708)
, the conversion rule element and the intermediate node are used as arguments to start procedure 3, and the process returns to step 702.

FIG. 8 is a flowchart of 1 procedure 3.

First, if the mark of the conversion rule element of the argument is at the left end of the right side (
801), 1 procedure 4 is started using the symbol on the left side and the intermediate node of the argument as an argument, and 802) the process ends. The mark of the argument conversion rule element is not at the right end of the right side, and the symbol to the right of the mark is not a terminal symbol! If it is -1 (803), start procedure 1 using the symbol on the right side of the mark as an argument (804) and end the process. If the symbol to the right of the mark is a terminal symbol, the process ends.

FIG. 9 is a flowchart of procedure 4.

First, search for the conversion rule element whose argument symbol is on the right side of the mark 901. ), if it does not exist (902), the process ends. Next, if the position of the intermediate node symbol is not connected to the position of the argument symbol (903), the process returns to step 901. If the position of the symbol of the intermediate node is connected to the position of the symbol of the argument, obtain the next conversion rule element (904,) and perform pattern matching (905) - If pattern matching fails (906), proceed to step 901. return. If the pattern match is successful, an intermediate node is generated by extending the position of the symbol and added to the set (907), a term is generated and added to the set of terms (908) - an intermediate node aggregator is generated and added to the set (908).
909). Next, start procedure 3 using the conversion rule element and intermediate node as arguments (91.0), and return to step 901.

Here, pattern matching is a process performed using information that generates structured data to check whether the data structures can be made equal. Structured data can have variables, and if the pattern match is successful, the variables are bound. For example, if a name starting with ``'' is used as a variable and a pattern match is performed for f(x, y) and f(a, b), the pattern match is successful and a is bound to X and b is bound to y.

If pattern matching is performed for f(-χ, -y) and g(c*d), the pattern matching will fail.

FIG. 10 is a flowchart of procedure 5.

First, search for a conversion rule element with the start symbol on the left side and the mark at the right end of the right side (1001).The position of the node symbol is
If the initial position and the position of the argument symbol are -1 (1002
) A syntax error is output (1003). If the position of the symbol of the node is minus 1 from the initial position of the symbol of the argument, the term of the node is output (1004), and the process ends.

This embodiment uses Jay Earlsy's parsing algorithm described above, so it can be applied to all context-free grammars.

Furthermore, the internal representation shown in FIG. 4 can also be implemented using another representation that has the same logical meaning, such as a table representation.

In this embodiment, a plurality of parse trees are expressed as 4Qj trees using node setters and node connectors, but other expression methods may also be used.

〔Effect of the invention〕

As described above, in the present invention, patterns are matched in each process of parsing processing applied to each part of an input symbol string, and parse trees that fail in pattern matching are discarded, thereby eliminating unnecessary syntax. It is no longer necessary to keep a parse tree, making it possible to improve the efficiency of computer memory.

[Brief explanation of the drawing]

FIG. 1 is a flowchart showing an overview of the processing procedure of an embodiment of the present invention. Figure 2(a) is. FIG. 3 is a hardware configuration diagram of the above embodiment of the present invention. FIG. 2(b) is a functional block diagram of the data conversion device according to the above embodiment of the present invention. FIGS. 3 and 4 are explanatory diagrams showing the logical structure of data in the above embodiment of the present invention. 5 to 10 are flowcharts showing the processing procedure of the above-described embodiment of the present invention. 2... Input section, 5... Text analysis section, 6... Syntactic analysis tree decomposition section, 7... Term production section, 8... Conversion rule storage section, 9... Syntax analysis section Tree memory department. Part (α) 1------4-Screaming Ellipse Section (b) Fig. Taku-ku Fig.■

Claims (1)

[Claims] 1. In a data processing device that converts a symbol string into structured data according to a conversion rule given from the outside, the structure of the symbol string is described in the notation of a context-free grammar, and the structure is a step of inputting a conversion rule that describes the structure of data in association with terminal symbols and non-terminal symbols used in the context-free grammar; a step of inputting a symbol string divided into parts; and a step of inputting a symbol string described in the conversion rule. parsing each part of the symbol string by a reduction process based on the structure of the string; and analyzing the structure of data having a structure associated with terminal symbols and non-terminal symbols corresponding to the structure of the symbol string. Data conversion comprising: generating data having a structure according to the description; and generating and outputting data having an overall structure by synthesizing the partially generated data having the structure. Method. 2. A data conversion method in which information for generating structured data is included in the conversion rule, and in the process of converting an input symbol string to structured data, syntax is applied to each part of the symbol string. Parsing processing, processing to provide each parse tree with information for generating data with the structure specified by the conversion rule, processing to decompose the parse tree, processing for each parse tree according to the information for generating data with structure. A data conversion method characterized in that an input symbol string is converted into data by performing a process of generating structured data from each part of the symbol string, and synthesizing structured data generated from each part of the symbol string. 3. The data conversion method according to claim 1, wherein the syntax analysis process is performed using Jay Earley's syntax analysis algorithm. 4. A data conversion system, which includes an input section for inputting symbol strings, a syntax analysis section for performing syntax analysis processing, a parsing tree decomposition section for disassembling the syntax parse tree, and a structure derived from the decomposed parse tree. For each part of the symbol string input in the input section, the syntax analysis section analyzes the syntax, the parse tree decomposition section performs parse tree decomposition processing, and the data generation section analyzes the structure of each part of the symbol string input in the input section. 1. A data conversion system characterized in that the symbol string is converted into structured data by performing processing to generate data having the following structure, and synthesizing structured data generated from each part of the symbol string. 5. The data conversion method according to claim 1, wherein a plurality of parse trees are expressed as one multiplex tree by sharing common nodes among the nodes of each parse tree. 6. An editing and translation system characterized in that an input sentence is analyzed and converted by using the data conversion method according to claim 1.