KR0133406B1 - Dynamic conversion method of retrieval statement concerning korean language information retrieval - Google Patents

Abstract

According to the present invention, when a Korean information retrieval system is implemented, a search query (query) inputted by a user to find desired information is analyzed internally in a system, and SQL, which is a DBMS query word, is input to a search statement inputted to actually retrieve information stored in a database. It is about how to convert dynamically with Structured Query Language.

In order to convert the search statement and provide the result of the query, the process is divided into six steps. In particular, the lexical analysis, the bottom-up parsing method based on the operator priority, and the intermediate code generation method are devised and implemented.

When using the information retrieval system, the search statements entered by the user are composed in various ways, and the search statements can be constructed in advance by considering the number of cases even in a static program. It is almost impossible to write a program in advance in consideration of search statements in all cases created by various inclinations.

In addition, when using a specific DBMS, there are many limitations in processing a static program in order to implement a search system so that the hit rate and reproducibility can be increased without depending on the DBMS.

According to the present invention, these problems can be solved and the inputted search statement is analyzed by the system at that time to dynamically convert to the corresponding query of the actual DBMS.

Search statement creation in the information retrieval system differs depending on the system configuration, but it is applied here only to the information retrieval system that is intended to use the DBMS and to create a search statement by a combination of logical operators (AND, OR, NOT, etc.). .

Description

Dynamic Coversion Method of Retrieval Statement Concering Korean Language Information Retrieval

1 is a block diagram illustrating a dynamic conversion and a search execution process of a search statement.

2 is a state transition diagram using a finite autometa.

3 is a schematic diagram of the soil stored in the stack after parsing.

4 is an operator priority relationship diagram.

5 is a diagram of the intermediate code stored in the stack in the form of a table.

6 is a diagram illustrating the keywords, keyword inverted, and prefix code relations.

In the present invention, when the command-type Hangul information retrieval system is implemented, SQL, which is a DBMS query word corresponding to the input search statement, can be analyzed to internally analyze a search statement inputted by a user to find desired information and to actually retrieve information stored in a database. (Structured Query Language) relates to dynamic conversion into statements.

In general, an information retrieval system is to store information after processing it using a file or a database, and to allow users to retrieve the desired information as needed. Depending on the method of use, a command or menu method, In addition, a mixture of them, and recently, a hypertext method (mouse or direction key), a touch screen method, and a voice recognition system using a graphical user interface under a window environment have been developed or under development.

In this section, we will focus on the command-based Hangul information retrieval system, and when users use these systems to find information, they will be able to make various search statements or make errors according to their inclination.

Therefore, the system must accurately interpret and analyze the entered search text and provide the information to be provided or rewritten immediately. To provide that information, the system must translate the search statement into the corresponding database query to get the information stored in the database.

In order to get the information, the system developer may perform a search by constructing a corresponding database query in anticipation of a search statement that can be input in advance, considering the number of cases even in a static program.

However, the information service provides information to a small number of groups or a large number of unspecified users. Therefore, the database is considered in advance in consideration of all cases of search written by various tendencies, such as the user's occupation, education, and experiences or experts using the system. There is a problem that it is almost impossible to write a query in a program.

Accordingly, an object of the present invention is to solve such a problem and to analyze the search text inputted by the system at the time of the input and to optimize the operation at the same time as the query query of the actual DBMS and to optimize the response time to shorten the response time. will be.

In order to achieve the above object, the present invention creates a search statement in an instruction type information retrieval system according to the configuration of the system, but uses a DBMS and creates a search statement by a combination of logical operators (AND, OR, NOT, etc.). The information retrieval system is composed of parsing step, semantic analysis step, intermediate code generation step, dynamic SQL generation step, SQL processing step, and search result output step. It is characterized by the method of outputting the search result after dynamic conversion of the search statement input by the user into a query.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention;

First, the present invention will be defined as follows.

Definition 1 lexical analysis

By separating the input search word by each word, Token type corresponding to each word is made into Tree structure of Postfix Expression type.

For example, the middle expression form with the operator * (mobile communication + computer) in the middle is formed into a tree structure in the form of the postfix expression so that the operator comes after the operator priority like mobile communication computer + United States *. ) To save.

Definition 2 Symantics Analysis

Parsing, a parsing stage, determines the presence or absence of logical operators by performing the necessary interpretation or analysis when moving from node to node along the tree structure. The processing is divided differently with and without logical operators.

Definition 3 Intermediate Code Generation

Intermediate code generation is limited to the case where there are logical operators, and the logical operators, subjects, and operators that are included in the search statement in the intermediate process before converting to the actual DBMS query form with the post-expression tree structure stored on the stack. Store information such as the operation order according to the rank on the stack.

Definition 4 Query Generation

Dynamic conversion with query information of actual DBMS with intermediate code information stored on the stack. Query optimization must be defined in the program in advance.

Definition 5 Perform search and provide results

Performs a search with the generated query and constructs a search set and outputs the result.

For example, mobile communication 329

Computer 534

American 232

S1 United States * (telecommunications + computers) 176

S1 is a search order, and the number after each subject or search statement indicates the number of records stored in the database.

According to the above definition, a detailed description of a method of dynamically converting a search sentence of an information retrieval system according to the present invention into a query language is as follows.

1 is a block diagram showing a dynamic change of a search statement and a process of performing a search. In the present invention, information retrieval is to find out how many data on a particular subject exist in a database. In other words, after converting the search statement entered by the user into a query through a parser, semantic parser, intermediate code generator, dynamic SQL generator, SQL processor, and search result outputter, it performs a search (composes a search SET). The result was output.

In FIG. 1, the first step, the parser, is a word or token consisting of a search word, a logical operator (AND, OR), a NOT operator, and parentheses operators that can change the operation priority. After checking the validity of the syntax of the search statement defined in the system, and checking the form of Infix Expression in which the operator is in the middle of the main words, Postfix Expression).

For example, the search for telecommunications AND US OR UK is converted to telecommunications US AND UK OR.

The parser is divided into lexical analysis module and parsing module. The function of lexical analysis module divides input search statements into basic units called tokens and passes them to the parsing module.

The types of tokens generated are the main word and the logical operators AND (*), OR (+), NOT (^), parentheses (()), ranges (:), adjacent words (ADJ), and prefix codes (year: PY, Low: AU), search SET number (SNO), and extended search term (ENO).

FIG. 2 is a state transition diagram using a finite automata. To generate a token, a token must be recognized. As shown in FIG. 2, a state transition diagram using a finite automata is described. Token recognition was used.

3 is a diagram of tokens stored in a stack after parsing. Tokens recognized by the state diagram according to the present invention are stored on the stack in a table form as shown in FIG. 3 and then transferred to a parsing module.

Also, the function of parsing module performs parsing by receiving token from lexical module to check whether input search statement conforms to the grammar defined by the system and analyzes it. Convert to form

The parsing method uses operator precedence parsing, a kind of bottom-up parsing method.

Operator priority parsing is a method of generating a parse tree by repeating reduction by using operation priority relations between terminals.

4 is an operator priority relation diagram. When a search statement is defined using an operator grammar, which is a kind of formal grammar, parsing is performed as follows.

E → E + E: E * E: id

Where E is a non-terminal, + is the OR operator, * is the AND operator, and id is the main word.

Using this grammar to define operator priority relationships is the same as in fourth.

In operator precedence, parentheses AND means that the parentheses have higher precedence than AND, OR parentheses mean that OR has lower precedence than the parentheses.

Where $ represents the end of the search statement rather than the token

The algorithm that checks the grammar of the search statement using this operator priority parsing and converts the search statement of the median expression to the postfix expression is as follows.

PROCEDURE Parser (S)

/ * Input: operator precedence relationship (4th degree), search statement (S) * /

/ * Output: post-expression search statement (Store_Parsing_Token module * /

/ * The search statement consists of A1A2A3 ...... An $ and there is only $ on the stack * /

/ * The Get_Token module passes the next token from the search statement * /

/ * Opeator_precedence is the operator priority table defined in the program * /

Current Token Type = '$' / * Mark the end of the search statement for the current token type * /

Call Search_Token / * Module to retrieve tokens * /

Initialize_Stack / * Initialize the stack to store tokens * /

Top of Stack = 0

Token Type, Token Type Index = '$'

Call Push_Stack / * Save token on stack * /

Token Counter = 0

Call Get_Token (Current Token Type, Curren Token Type Index)

Call Search_Token

Do While (^ (Token Type Index of Stack (Top of Stack) = '$'

Current Token Type Index = '$'))

End_Reduce = Top of Stack / * Set end of Reduction * /

Do I = Top of Stack To 1 By -1 Until (Token Type Index of

Stack (I) ^ = Non_Terminal) / * Search Terminal on Top of Stack * /

If Token Type Index of Token (I) ^ = Non_Terminal

Then

Index of Stack = I

End / * End of / Do Loop * /

If Operator_Precedence (Token Type Index of Stack (Index of Stack),

Current Token Type Index) = ''

Then / * Reduce * /

Do

Do Until (Operator_Precedence (Token Type Index of Stack (Top of

Stack), Terminal) = ''

Do Until (Token Type ^ = Non_Terminal Token Type ^ = ')'

Call Pop_Stack / * retrieve token from stack * /

If Token Type ^ = Non_Terminal Token Type ^ = ''

Then

Do;

Terminal = Token Type

If Token Tyep ^ = '('

Then

Call Store_Parsing_Token / * Saves the search statement in the table

module */

End

End / * Do Until * /

Do While (Token Type Index of Stack (Top of Stack) = Non_Terminal)

Call pop_stack

End

End / * Do Until * /

Call Valid_Reduction / * Module that checks for reduction * /

If ^ (Token Type Index of Stack (Top of Stack) = '$'

Current Token Type = '$'

Then

Do

Token Type, Token Type Index = Non_Terminal

Call push_stack

End

End / * Do * /

Else

If Operator_Precedence (Token Type Index of Stack (Index of Stack),

Current Token Type) = `` or

Operator_Precedence (Token Type Index of Stack (Index of Stack),

Current Token Tyep) = '='

Then

Do / * Push Current Token into Stack * /

Token Type = Current Token Type

Token Index = Current Token Index

Call push_stack

Call Get_Token (Current Token Type, Current Token Type Index)

Call Search_Token

End

Else

Call Syntax Error / * Search Statement Error Handling Module * /

End / * Do While * /

In relation to the present invention, the second step of the semantics analyzer (Symantics Analaysis) in FIG. To determine the existence of logical operators.

The presence or absence of such a logical operator makes the process of dynamically converting a search statement into a query to a query with or without the intermediate code generator as a next step to be processed differently.

In addition, the third step, the intermediate code generator, is a complete dynamic SQL for the search statement stored in the stack structure by converting the compound expression of the infix expression type to the postfix expression type by the parser. In order to create a statement, it is divided into operator units so that operations can be performed sequentially based on the operators defined in the system, and the intermediate level code is created and stored in temporary memory with a stack structure.

FIG. 5 is a diagram illustrating the intermediate code stored in the stack in the form of a table. Referring to FIG. 5, for example, a search statement of US AND mobile communication OR Japan AND mobile communication is performed by a parser. AND Japanese mobile communication AND OR-type search statement is converted into T1, US AND mobile communication method T1, Japanese AND mobile communication method T2 and T1 OR T2 operation method T3.

The algorithm for generating this intermediate code is as follows.

PROCEDURE Internediate_Code_Generation (E)

/ * Input: Postfix E * /

/ * Output: intermediate code stored in table * /

/ * Count_NextToken_Index: Find index of next token * /

/ * Search_NextToken (Token_Index): find operator * /

Top of Stack = 0 / * Initialize stack pointer * /

Intermediate_Code_Index = 0

Temp_Index = 0 / * Temporary storage location of intermediate code * /

Do While (token number = 0)

Token_Index = Count_NextToden_Index

Next_Token = Search_Next_Token (Token_Index)

If Next_Token = Operand / * Check if the next token is an operator * /

Then

Call Push_Stack / * Overland types (I: Token, T: Temp) and Tokens on the stack

Save location of index or temp * /

Else

Do

Intermediate_Code_Index = Intermediate_Code_Index + 1

Intermediate Code Operator (Intermediate_Code_Index) = Next_Token

Call Pop_Stack / * Take token (Operland 2) from stack * /

/ * Operand1_Type (I: Token, T: Temp) * /

/ * Operand1_Index: Index or Temp position of the token * /

IntermediateCodeOp2Type (Intermediate_Code_Index) = Operand2_

Type

IntermediateCodeOp2Type (Intermediate_Code_Index) = Operand2_

Index

Call Pop_Stack / * Retrieve token (Operland 1) from stack * /

/ * Operand1_Type (I: Token, T: Temp) * /

/ * Operand1_Index: Index or Temp position of the token * /

IntermediateCodeOp1Type (Intermediate_Code_Index) = Operand2_

Type

IntermediateCodeOp1Type (Intermediate_Code_Index) = Operand2_

Index

Temp_Index = Temp_Index = + 1

IntermediateCodeResType (Intermediate_Code_Index) = 'T'

IntermediateCodeResType (Intermediate_Code_Index) = Temp_Index

Call Push_Stack / * Save intermediate code on stack: 'T' and Temp_Index * /

End

End / * Do While * /

After the intermediate code is generated by the intermediate code generator, the next step is to convert the SQL statement, which is a query word of the DBMS used by the system, through the SQL generator to get the information in the database.

SQL, a query language of DBMS, is a DBMS standard adopted by ISO and ANSI, which combines the relational algebra of relational DBMS and components and features of relational interpretation.

In addition to these query functions, SQL also supports data definition language (DDL) functions that define database structures, data manipulation language (DML) functions that modify data in the database, and security and control of the database. It is a powerful database language with features such as Data Control Language (DCL). FIG. 6 is a diagram showing keywords, keyword inverted, and prefix code relations in the context of the present invention. The keywords and logical operators AND, OR, NOT, and range search, truncation search, prefix code, and search set by the SQL generator are shown in FIG. The following is the algorithm and method for dynamically converting a search statement consisting of number and extended search terms into a SELECT statement in SQL, assuming that there are three relations as shown in Figure 6.

First, the SELECT statement provides union, intersect, and minus, which are relational algebraic unions, unions, intersections, and difference sets. The basic structure of a SELECT statement consists of three clauses: SELECT, FROM, and WHERE.

SELECT clause: Corresponds to the extraction operation of relational algebra and enumerates the attributes (fields) that you want to appear in the query results.

-FROM clause: This clause expresses the relational algebraic selection predicate, and predicates on the constraints of the attributes of the relations listed in the FROM clause.

SELECT A1, A2, .........., An

FORM R1, R2, .........., Rn

WHERE P

Where A1, A2, .........., An represents an attribute, R1, R2, .........., Rn represents a relation, P represents a predicate, and DBMS represents a SELECT statement. To process, get Cartesian products of the relations listed in the FROM clause, and then perform the select operation with the predicates listed in the WHERE clause. Finally, extract the values for the attributes listed above in the SELECT clause.

PROCEDURE SQL_Generation (C)

/ * Input: intermediate search form C * /

/ * Output: SELECT statement in SQL * /

Do While (middle code ^ = '$') / * $: end of middle code * /

Count_IntermediateCode_Index / * Calculate the index of the intermediate code * /

If index 2 of the intermediate code / * If there is one operator in the search statement * /

Then

Do

Call Query_Generation / * Generate Query (SQL) * /

Return

End

Else / * If the operator is 2 or more in the search statement * /

Do

Call Query_Generation / * Generate Query (SQL) * /

Store_Intermediate_SQL / * Store intermediate SQL statements in Temporay * /

End

Query_Generation / * A circular procedure that generates a query (SQL) statement * /

Switch (Operator) / * Operator * /

Case 1 (AND): Generate SELECT statement with 'INTERCT' function

Case 2 (OR): Generate SELECT statement with 'UNION' function

Case 3 (NOT): Generate SELECT statement with 'NOT' function

Case 4 (cutting): Generate SELECT statement with 'cutting' function

Case 5 (Range): Generate SELECT statement with 'Range' function

Case 6 (prefix code): Generate SELECT statement with 'prefix code' function

Otherwise:

End / * End of Switch (Operator) * /

End / * End of Query_Generation PROCEDURE * /

CompleteOptimize_SQL_Statement / * Final SQL Statement Generation and Optimization * /

End / * End of Do While (middle code = '$') * /

-Keyword search

The following example shows how to convert a search statement to find a computer related information into a SELECT statement.

Select Record Count

From keyword relation

Where keyword = computer

-Prefix Code Search

Count is a built-in function that calculates the total number of records when converting a search statement to find information containing a subject in the subject computer. T1 is a system-defined prefix code among the words in the subject. When

Select Count (record number)

From prefix relation

Where T1 = Computer

-Cut (?) Search

When converting a query to find SELECT information whose main word starts with communication into a SELECT statement, LIKE is a parameter among query words that handle truncation function.

Select Count (record number)

From keyword relation

Where keyword LIKE? = Communication

AND operation

The query to find records related to both computers and semiconductors is computer AND semiconductor.

If the keyword number (or synonym number) of the computer is 1 and the keyword number (or synonym number) of the semiconductor is 2 in the relations of FIG. 6, the search statement is expressed as a SELECT statement as follows.

(Select record number

From keyword Inverted relation

Where keyword number (or synonym number) = 1)

INTERSECT

Select record number

From keyword Inverted relation

Where keyword number (or synonym number) = 2)

In this case, the INTERSECT operation may or may not be provided depending on the DBMS. If not, the INTERSECT operation may be converted using the JOIN operation as follows.

Select A. Record Number

From Keyword Inverted Relation A, Keyword Inverted Relation B

Where A. Keyword Number (or Synonym Number) = 1

AND B. Keyword Number (or Synonym Number) = 2

OR operation

The query to find records related to both computers and semiconductors is computer OR semiconductors.

In the relations of FIG. 6, if the keyword number (or synonym number) of the computer is 1 and the keyword number (or synonym number) of the semiconductor is 2, the search statement is expressed as a SELECT statement as follows.

(Select record number

From keyword Inverted relation

Where keyword number (or synonym number) = 1)

UNION

Select record number

From keyword Inverted relation

Where keyword number (or synonym number) = 2)

NOT (^) operation

When searching all the data of each country except the US among the data related to mobile communication, the query would be mobile communication → USA.

When the keyword number (or synonym number) related to mobile communication is 1 and the keyword number (or synonym number) of USA is 2 and this search statement is expressed as DL2 SQL statement, there is no direct SQL statement that actually provides NOT operation. It was created by combining the functions.

Select Distinct Record Number

From keyword Inverted relation

Where Keyword Number (or Synonym Number) In (1)

And A. Record Number NOT In (Select A. Record Number

From Keyword Inverted Relation A, Keyword Inverted Relation B

Where A. Keyword Number (or Synonym Number) = 1

And B. Keyword Number (or Synonym Number) = 2

And A. Record Number = B. Record Number

Range (:) operation

The range operation (:) is PY = 1991 AND mobile communication OR PY = 1992 AND only when AND and OR combinations are used to express the query when searching for data related to mobile communication from 1991 to 1993. Mobile search OR PY = 1993 AND If the number of years (search key) you want to find like mobile communication becomes more complicated, the search statement becomes more complicated.However, if you use range operation, you can easily configure mobile communication AND PY = 1991: 1993 do.

Here, suppose that the keyword number (or synonym number) for mobile communication is 1 and the prefix code of the year search key is PY.

Select Distinct Record Number

From Keyword Inverted Relation A, Search Key Relation B

Where A. Keyword Number (or Synonym Number) In (1)

And B. PY Between '1991' And '1993'

And A. Record Number = B. Record Number

-Parentheses (())

When you want to find a query about US mobile communication or computer, you can create a query with US AND mobile OR OR US AND computer with only AND and OR combinations. As in the case of computers).

Here, assume that the keyword number (or synonym number) for the United States is 1, the keyword number for mobile communication is 2, and the keyword number for computer is 3.

Select Distinct Record Number

From Keyword Inverted Relation A, Keyword Inverted Relation B

Where Keyword Number (or Synonym Number) In (1)

And (keyword number (or synonym number) = 2

Or keyword number (or synonym number) = 3

And A. Record Number = B. Record Number

As above, when the query word corresponding to the search statement is generated, the SQL processor takes information from the database and constructs a search set for the result.

The search set is an order in which a search is performed, and this information may be used as a kind of search statement.

In the final step, the search result displayer displays the search results on the screen.

For example, in the case where the first inputted search sentence is a US AND (mobile communication OR computer), assuming that only the main term computer is the equivalent, the search result has the following form.

239 Mobile Communications -------- [1]

157 @Computer -------- [2]

37 US AND (Mobile Communication OR Computer) -------- [3]

[1] means that the total number of records related to the mobile communication of the main word is 239. In the [2], the @ symbol means that the main word has been treated as an equivalent and the total number of records about the computer is 157.

And [3] means that the search statement US AND (Mobile Communication OR Computer) is composed of the first set (S1) and a total of 37 records are stored in the expert information database in the system.

As described above, the search statement input by the present invention can be analyzed in the system at the time of the input, and the corresponding query of the actual DBMS can be dynamically transformed and simultaneously optimized to shorten the response time.

Claims (3)

In a method for dynamically transforming and executing a search statement of an information retrieval system, A parsing step of separating the input search statement by each word, generating a token corresponding to each word, checking whether it is suitable for the grammar of the search statement defined in the system, and converting it; A semantics analysis step of determining whether a logical operator exists by performing an analysis or analysis required when moving from node to node along a tree structure in the parsing module; Only when there is a logical operator in the semantic analysis step, the intermediate code generation step of decomposing the search statement stored in the memory of the stack structure into operator units to form an intermediate code and storing the intermediate code in the temporary memory having the stack structure Wow ; A query word generation step of dynamically converting intermediate code information stored in the stack into a query word (SQL) of an actual DMS; A query processing step of retrieving information from a corresponding database after the query SQL generation step and constructing a search set; And performing a search using the generated query word and outputting a search result for the result. The method of claim 1, The division analysis step includes a lexical analysis step of separating the input search sentence by each word and separating the token into a token having a word form corresponding to each word by using a finite autometa. Dynamic conversion of search statements. The method of claim 1, The division analysis step performs bottom-up parsing on the tokens output after the lexical analysis step according to the priority of the operator in order to check whether the search statement defined in the system is suitable for the grammar. A parsing method that converts an Infix Expreesion type search statement existing in the middle of a main word into a tree structure of a rear expression type placed after a main word or a search key. Dynamic conversion of search statements for information retrieval.