JP2008287394A - Document retrieval method and apparatus, and computer program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document retrieval method, a document retrieval apparatus, and a computer program therefor, which improves accuracy in retrieval and absorbs orthographic variations in a retrieval result. <P>SOLUTION: The document retrieval apparatus 10 receives, an input search character string as a query via an input section 1, uses a formal name/regular expression dictionary 31, a conversion list 41, and an insertion list 51 to create a regular expression for the received query, uniquely identifies the formal name of a word from the regular expression, creates a query group by using the created regular expression to expand the query, uses the expanded query group to retrieve a document via a document retrieval section 6, extracts document data including the query group, uses the identified formal name and regular expression corresponding to the formal name to unify the notation of a query included in the extracted document data, and outputs information on the document data unified in notation via an output section 8. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention creates a regular expression dictionary by generating a regular expression of the same meaning from a query as a character string to be searched that is input to the search device, performs question expansion and standardization, improves search accuracy, and The present invention relates to a document search method and apparatus, and a computer program thereof, characterized in that it can simultaneously absorb fluctuations in search results.

  Conventionally, in a document search apparatus, when a searcher inputs a query as a character string to be searched, a document including the query is searched from a document database. In a conventional document search apparatus, when a searcher inputs a query, a function for searching not only for a document having the corresponding query but also for a document having a notation of the corresponding query and preventing omission of document search is added. There are many things. In order to absorb the notation fluctuation, the search device uses a synonym dictionary. The dictionary is required not only to expand the query but also to absorb the query notation included in each retrieved document.

For example, in the question answering system described in Non-Patent Document 1, the difference between the text of the user's question sentence and the knowledge database is absorbed using a synonym dictionary in which synonyms and synonyms are grouped. . This question answering system is equivalent to the search system in the task of searching for a text that matches the keyword because the text that is the basis of the answer to the question keyword is searched from the document set. In addition, when this question answering system uses a synonym dictionary to absorb the deviation of the expression in the text including the question sentence and the answer, the entire set of expressions including the query and its notation is expanded. Need to be registered in the dictionary. Reference 1 manually creates a synonym dictionary in which 919 groups of synonymous expressions, 3512 words, and 217 phrases are registered.
Automatic Question Answering Based on Large-scale Text Knowledge Base-Dialog Navi-Yoji Kiyota, Ikuo Kurohashi, Toko Kido, Natural Language Processing Vol.10 No.4 2003

  However, there are the following problems in creating a dictionary including a written expression.

  -Since there are an infinite number of expression patterns including notation fluctuation, the dictionary size becomes very large when all of them are registered.

  ・ Since new words and notation fluctuations continue to be generated every day, it may not be sufficient to collect expressions at a certain point in time. Also, speedy response is required.

  ・ It takes a lot of time and cost to create manually.

  The problem as described above is more conspicuous in product names composed of character strings including alphanumeric symbols.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a document search method, a document search apparatus, and a computer program thereof that can improve accuracy during search and absorb notation fluctuations in search results. .

  In order to achieve the above-mentioned object, the present invention provides a database of formal name / regular expression dictionaries having information on pairs of formal names of words and regular expressions of the formal names as entries, and character strings to be searched for From a computer device comprising a database of conversion lists that can be arbitrarily registered used as a conversion rule when creating a regular expression of a query, and a database of insertion lists that can arbitrarily register characters and symbols to be inserted when creating a regular expression of the query When a query is input to the document search device, the document search device extracts document data including a query group obtained by expanding the query from a plurality of document data stored in a document database. A document search method for outputting a result, wherein the document search device uses an input character string to be searched as a query. Accept, create a regular expression of the accepted query using the formal name / regular expression dictionary, the conversion list, and the insertion list, and uniquely identify the formal name of the word from the regular expression, Using the created regular expression, the query is expanded to create a query group, a document search is performed using the expanded query group, document data including the query group is extracted, and the identified formal A document search method is proposed that uses a name and a regular expression corresponding to the formal name to unify the notation of a query included in the extracted document data, and output information of the document data with the unified notation.

  According to the document search method of the present invention, a notation fluctuation pattern described in a regular expression is automatically generated from a query including an input alphanumeric symbol string, and the regular expression and a formal name are paired and registered in a dictionary. As a result, the query at the time of search is expanded and the notation of search results is unified.

That is, in the present invention: Expressions including various notations of the input query are described as regular expressions and registered in the dictionary.

  2. A regular expression including all possible notation fluctuations is automatically generated by fully expanding the regular expression.

3. If a new notation is found, the regular expression in the dictionary is automatically updated.
Is performed on a character string including alphanumeric symbols.

  By using regular expressions in this way, the number of registered words in the dictionary can be reduced, and one dictionary can be used for both query expansion and notation fluctuation absorption, and the dictionary size can be reduced.

  In addition, by automatically generating expressions that include all possible notation fluctuations, it is possible to improve both the speed and completeness of correspondence to new words, and at the same time reduce costs.

  Furthermore, since both the input query and the formal name are described by regular expressions, there is a high possibility that the query can be identified even if a query different from the formal name is input.

  The present invention also provides a document search apparatus to which the above document search method is applied, that is, when a query as a character string to be searched is input, the query is expanded from a plurality of document data stored in a document database. A document search apparatus that extracts document data including a group of queries and outputs the extraction result, an input means for receiving the input character string to be searched as a query, a formal name of the word, and the formal A database of formal names / regular expression dictionaries having information paired with regular expressions of names as entries, a database of conversion lists that can be arbitrarily registered to be used as conversion rules when creating regular expressions of the query, and a normality of the query A database of an insertion list in which characters and symbols to be inserted at the time of expression creation can be arbitrarily registered, the formal name / regular expression dictionary, Using the conversion list and the insertion list, the regular expression of the accepted query is created, and the formal name identifying means for uniquely identifying the formal name of the word from the regular expression, and using the created regular expression Query expansion means for expanding a query to create a query group, document search means for performing a document search using the expanded query group, and extracting document data including the query group, and specifying the formal name Using the formal name specified by the means and the regular expression corresponding to the formal name, the notation unifying means for unifying the notation of the query included in the extracted document data, and the information of the document data with the unified notation A document search apparatus provided with output means for outputting.

  Furthermore, the present invention provides a computer program for causing a computer device to operate as the document search device, as a formal name / regular expression dictionary having as an entry information obtained by pairing a formal name of a word and a regular expression of the formal name. Database, database of conversion list that can be arbitrarily registered used as a conversion rule when creating a regular expression of a query as a character string to be searched, and insertion that can arbitrarily register characters and symbols to be inserted when creating a regular expression of the query When a query is input to a document search device comprising a computer device having a list database, the document search device expands the query from a plurality of document data stored in the document database. The document retrieval apparatus that extracts the document data including the data and outputs the extraction result is operated. The received query using the step of accepting the inputted character string to be searched as a query, the formal name / regular expression dictionary, the conversion list, and the insertion list Creating a regular expression, and uniquely identifying the formal name of the word from the regular expression, expanding the query using the created regular expression, creating a query group, and the expanded Performing a document search using a query group, extracting document data including the query group, and using the specified formal name and a regular expression corresponding to the formal name, in the extracted document data A computer program including the step of unifying the notation of a query included in the document and the step of outputting document data information in which the notation is unified Proposed.

  According to the document search method and document search apparatus of the present invention, the number of registered words in the dictionary can be reduced by using regular expressions, and one dictionary can be used for both query expansion and notation fluctuation absorption. The dictionary size can be reduced. In addition, automatic generation of regular expressions including all possible notation fluctuations can improve the speed and coverage of new words, and at the same time reduce costs. Furthermore, since both the input query and the formal name are described by regular expressions, there is an excellent effect that the possibility that the query can be specified even if a query different from the formal name is input increases.

  In addition, by using the computer program of the present invention, it is possible to easily configure a document search apparatus that implements the document search method of the present invention using an arbitrary computer apparatus.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram of a document search apparatus according to an embodiment of the present invention. In the figure, reference numeral 10 denotes a document search device comprising a computer device that operates according to a predetermined computer program. An input unit 1, a question expansion unit 2, a database 3 in which a formal name / regular expression dictionary 31 is stored, and a conversion list 41 are provided. A database 4 stored, a database 5 storing an insertion list 51, a document search unit 6, a notation unification unit 7, and an output unit 8 are provided. Each of these components is configured by both hardware and software of the computer device.

  As will be described later in detail, the outline will be described first here.

  The input unit 1 accepts a query for searching in an arbitrary format. In the query expansion unit 2, if the query includes two or more of numbers, alphabets, symbols, and other character types, the query is expanded to a plurality of synonymous expressions and the expanded query group, otherwise the document is left as is. Output to the search unit 6. The document search unit 6 searches for a document including the output of the question expansion unit 2 and passes a document set as a result to the notation unification unit 7. The notation unifying unit 7 unifies the notation of the input query included in the document data set returned from the document search unit 6 only when the query is expanded, and passes the document data set to the output unit 8. The output unit 8 outputs the processing result in an arbitrary form.

  The input / output method in the input unit 1 and the output unit 8 and the search method in the document search unit 4 are well-known existing technologies (for example, Non-Patent Document 2: Information Science and Technology Vol.54 2004 No.2, Special Feature The Internet search engine, the search engine algorithm (Kanemune Susumu) (p78 to p83), and the search engine architecture (Yaman Hayato) (technology disclosed in p84 to p89) are used.

  The regular expression here is explained using Perl as an example. Therefore, when using other expressions, the method is followed. Also, regular expressions that include uncertain elements such as “. *” And “. +” That repeat “any number of characters any number” are not used. In addition, although different from the Perl expression, “” is used for convenience in expressing a space symbol.

  When using this apparatus 10, it is necessary to perform a prior setting (dictionary registration function).

  That is, when using the search device 10, the user can select a “dictionary registration function” for whether or not to update the dictionary (new word registration and regular expression expansion) using a setting file or the like. .

  When it is necessary to create the formal name / regular expression dictionary 31 when the apparatus 10 is used for the first time, the “dictionary registration function” must be turned on.

  When this apparatus 10 is used only for query expansion, or when a notation fluctuation expression that is difficult to register in a dictionary is used as a query, the “dictionary registration function” can be turned off.

  Also, conversion rules and symbols used for query expansion can be set in advance in the conversion list 41 and the insertion list 51.

  Next, the operation of each component of the apparatus 10 will be described in detail.

  The input unit 1 acquires a search target character string input by the user in an arbitrary format, receives the character string as a query, and passes the received query to the question expansion unit 2. The query is composed of a character string.

  For example, as shown in FIG. 2, the question expansion unit 2 includes an input determination unit 21, a formal name specifying unit 22, and a query expansion unit 23, and a formal name / regular expression dictionary 31, a conversion list 41, and an insertion list 51. Process using.

  The official name / regular expression dictionary 31 is a dictionary that expresses a set of synonymous expressions of words as a regular expression and has one entry as a pair with the official name of the word. FIG. 3 shows an example of the formal name / regular expression dictionary 31.

  The conversion list 41 is a list in which brand names having a large number of synonymous expressions and symbols that do not appear to change in meaning even if deleted are registered. An entry such as a brand name having many synonymous expressions registers the synonymous expression set as one entry in the conversion item. For example, in entries 1 to 4 in FIG. 4, synonymous expressions of conversion items are separated by “, (comma)” as (expression 1, expression 2, expression 3). Also, an entry that can only be deleted is registered in the form of one entry and one as shown by (&) in entry 5.

  The expression registered in the entry of the insertion list 5 is used when the query is expanded to an expression including notation fluctuation. Therefore, in the insertion list 5, symbols to be inserted at the time of query expansion, such as blank symbols and hyphens that are frequently used for expression fluctuation expressions, are registered. An example of the insertion list 5 is shown in FIG. In the insertion list 5 of FIG. 5, ““ (blank symbol) ”is registered as the insertion item of entry 1, and“-(hyphen) ”is registered as the insertion item of entry 2.

  The input discriminating unit 21 queries the formal name specifying unit 22 when the query input from the input unit 1 includes any of (alphabet, number, symbol) and is composed of two or more types of characters. Is output. Otherwise, the query is output to the document search unit 6.

  The formal name identification unit 22 performs the process shown in FIG. 6 using the formal name / regular expression dictionary 31, the conversion list 41, and the insertion list 51. Since the processing here requires ON / OFF information of the “dictionary registration function” performed in advance, the information is captured in advance and used in the processing of steps (22s26) and (22s27).

  Here, the details of the process performed by the formal name specifying unit 22 will be described with reference to the flowcharts of FIGS. 6 and 7.

  The official name identification unit 22 compares the query received from the input determination unit 21 with the entry of the formal name / regular expression dictionary 31 and whether there is a match with the regular expression of the entry of the formal name / regular expression dictionary 31. (22s1) and if there is a match, the process proceeds to step (22s3), and the regular expression of the matched entry is output to the query expansion unit 23. If there is no match, the process proceeds to step (22s2) to create a regular expression.

  The detailed processing of creating a regular expression in step (22s2) is as shown in FIG. 7. First, the query is divided at different boundaries of character types to form array elements (22s21). At this time, the space symbol is not an element and is deleted. Next, rule application processing is performed (22s22). The details of the rule application process (22s22) are as shown in FIG. 8, and the conversion list 41 is used for this process.

  That is, in the rule application process (22s22), the process from step (22s221) to step (22s226) is repeated for each element of the array created in step (22s21), and the repetition is finished when all elements are examined.

  In step (22s222), it is determined whether or not the array element matches the expression of the conversion list. If they match, the process proceeds to step (22s223), and if not, the process proceeds to step (22s224).

  In step (22s223), the array element is changed to (matched entry,?), And the process proceeds to step (22s224).

  In step (22s224), it is determined whether or not the array element includes an alphabet. If the array element includes an alphabet, the process proceeds to step (22s225), and if not, the process proceeds to step (22s226).

  In step (22s225), the array element is converted to (as it is, all capital letters, all lower case letters, and only the first letter is capital letters), and if duplicates appear in the elements, the duplicate parts are deleted. Thereafter, the process proceeds to step (22s226).

  In step (22s226), when all the elements of the array have been examined, the process is terminated, and the process proceeds to step (22s23) of the symbol insertion process in step (22s2).

  In the symbol insertion processing step (22s23), the insertion list 51 is delimited by “, (comma)”, and a character “?” Is added to the end to form one element, which is then inserted between query array elements. ).

  In the regular expression creation process step (22s24), array elements are connected to create a regular expression. Details of this processing are shown in the flowchart of FIG.

  That is, in the processing of step (22s24), the processing from steps (22s241) to (22s248) is repeated for each element of the array.

  In step (22s242), it is determined whether or not all the characters delimited by “, (comma)” in one element are one character. If it is one character, the process proceeds to step (22s243). Advances to the processing of step (22s244).

  In step (22s243), “, (comma)” is deleted, the character “()” is converted to the character “[]”, and the process proceeds to step (22s245).

  In step (22s244), “, (comma)” is converted to “| (pipe)” and the process proceeds to step (22s245).

  In step (22s245), it is checked whether or not the character “?” Is included at the end of the element. If the character “?” Is included at the end of the element, the process proceeds to step (22s246). If the character “” is not included, the process proceeds to step (22s247).

  In step (22s246), the character “?” Is moved to the right outside the parenthesis outside the element, and the process proceeds to step (22s247).

  In step (22s247), when all the elements have been processed, the process proceeds to step (22s248).

  In step (22s248), all elements are connected to form a character string, and the process ends. Thereafter, the process proceeds to step (22s25).

  In step (22s25), it is checked whether or not a formal name that matches the created regular expression exists in the formal name / regular expression dictionary 31. As a result, if the official name matching the created regular expression is found in the official name / regular expression dictionary 31, the process proceeds to step (22s26), and if not, the process proceeds to step (22s27).

  In step (22s26), the regular expression corresponding to the matched formal name is expanded to include the newly created regular expression, and this is registered when the dictionary registration function is ON. Details of the processing of this step (22s26) will be described with reference to the flowchart of FIG. In the process of step (22s26), as shown in FIG. 10, the processes of steps (22s161) to (22s2610) are performed.

  In step (22s261), the two regular expressions are separated in pairs of parenthesis expressions and stored in the array. At this time, the part without parentheses is appended with the character “()” and stored. Thereafter, the process proceeds to step (22s262).

  In step (22s262), an element that completely matches between two regular expressions is searched, and a group of elements that do not match the boundary are compared from the front. Thereafter, the process proceeds to step (22s263).

  In step (22s263), a partially matching portion is searched for among the group of elements that do not match between the two regular expressions extracted in step (22s262). Thereafter, the process proceeds to step (22s264).

  In step (22s264), it is determined whether or not one part in the element is different. If only one part in the element is different and the other parts match, the process proceeds to step (22s265). Otherwise, the process proceeds to step (22s267). ).

  In step (22s265), it is determined whether or not the different part in the element is a single character. If the different part in the element is a single character, the process proceeds to step (22s266). Otherwise, the process in step (22s268) is performed. Proceed to processing.

  In step (22s266), when the parenthesis surrounding the element is a character “[]”, an unmatched character is added in the middle, and when the character “()” is used, a character “| (pipe)” is connected. Thereafter, the process proceeds to step (22s269).

  Step (22s267) is processing when there is no matching element as a result of the determination in step (22s264). Therefore, a character “?” Is put outside the parentheses of each element and then inserted into the original position. Thereafter, the process proceeds to step (22s269).

  In step (22s268), when the character is “[]”, it is replaced with the character “()” and connected with the character “| (pipe)”. Thereafter, the process proceeds to step (22s269).

  In step (22s269), when all the non-matching parts detected in the process of step (22s262) (all of the different parts) have been compared, the iterative process is terminated, and the process proceeds to step (22s2610).

  In step (22s2610), if the above-mentioned “dictionary registration function” is ON, the regular expression of the corresponding entry in the formal name / regular expression dictionary 31 shown in FIG. 3 is re-registered in the newly created regular expression. To step (22s26).

  When the process of step (22s26) is finished, the process of step (22s2) is finished, and the process proceeds to step (22s23).

  In step (22s27), if the dictionary registration function is ON, the entered query is registered as a formal name and registered as a new entry in the formal name / regular expression dictionary 31 shown in FIG. To do. When the process of step (22s2) is finished, the process proceeds to step (22s3).

  In step (22s3), when the regular expression creation process in step (22s2) is finished, the pair of the formal name and regular expression registered in the dictionary or newly created is output to the query expansion unit 23, and the process is terminated. .

  The query expansion unit 23 expands a query using the created regular expression to create a query group. The details of the processing of the query expansion unit 23 are as shown in the flowchart of FIG. 11, and the processing of steps (23s1) to (23s9) is performed.

  In step (23s1), all of the regular expressions output by the official name specifying unit 23 are expanded except for the symbol part, and the process proceeds to step (23s2).

  In steps (23s2) to (23s8), all these processes are performed for each symbol part before expansion in the regular expression.

  In step (23s3), it is determined whether the character before or after the symbol part being processed is a symbol, and if the character before or after the symbol part being processed is a symbol Copies the information being processed, and advances the processing of step (23s4) and step (23s6) in parallel. Otherwise, the process of step (23s7) is performed.

  In step (23s4), the symbols before and after the symbol part being processed are deleted, and the process returns to step (23s3).

  In step (23s 5), it is determined whether any character before or after the symbol part to be processed is the head or end of the query, and before or after the symbol part to be processed. The process of step (23s6) is performed when any of the following characters is the head or end of the query, and the process of step (23s7) is performed otherwise.

  In step (23s6), the symbol part being processed is deleted without being expanded, and the process proceeds to step (23s2), and the process proceeds to the next symbol part.

  In step (23s7), the symbol part being processed is expanded, and the process proceeds to step (23s8).

  In step (23s8), when all the symbol parts included in the regular expression have been processed, the iterative process is terminated, and the process proceeds to step (23s9).

  In step (23s9), among the queries that have been expanded, those that have overlapped with the previously generated query are deleted, and the process is terminated.

  The query extension unit 23 passes the synonymous expression set of the expanded query, its formal name, and regular expression to the document search unit 6. The query expansion unit 23 passes only the query to the document search unit 6 when query expansion is not performed.

  Thus, the process of the question expansion unit 2 ends.

  The document search unit 6 receives [query only] or [formal name, regular expression, synonym expression set] of the query from the question expansion unit 2. In the former case, a query is used, and in the latter case, a synonymous expression set of queries is used to perform a document search process using a well-known existing technique, and a plurality of document data stored in the document database 100 are stored. The corresponding document data is extracted from the inside. Then, if there is a list of document data as a search result, a formal name, and a regular expression, they are also output to the notation unifying unit 7.

  As shown in FIG. 12, the notation unifying unit 7 includes an output discriminating unit 71 and a unifying unit 72. When the question expansion unit 2 performs the question expansion, the search result, the official name of the query, and its normal name An expression is received from the document search unit 6. When the question expansion unit 2 does not perform the question expansion, only the search result is received from the document search unit 6.

  If only the search result is received from the document search unit 6, the output determination unit 71 does nothing and passes the search result to the output unit 8 as it is. In addition to the search result from the document search unit 6, the output determination unit 71 passes the formal name and regular expression to the unification unit 72 to request processing.

  The unifying unit 72 uses a regular expression to search for an expression including a query notation included in a set of document data as a search result, and rewrites the expression with an official name. Then, the result is passed to the output unit 8.

  The output unit 8 outputs document data information in which the query notation is unified in an arbitrary format.

  Next, an example using a specific example regarding the above embodiment will be described in detail.

  Here, the regular expression follows the Perl expression. The symbol ““ ”represents a blank symbol (space).

  In this embodiment, the “dictionary registration function” is set to ON as a prior setting (dictionary registration function). In the present embodiment, a description will be given assuming that a query “FOMA P902i” is input to the input unit 1 of the document search apparatus 10 as a character string to be searched. In this case, as the formal name / regular expression dictionary 31, the conversion list 41, and the insertion list 51 used in the question expansion unit 2, those shown in FIGS.

  The input determination unit 21 of the question expansion unit 2 receives the query “FOMA P902i” received by the input unit 1 as an input. Since the query “FOMA P902i” is composed of alphabets, numbers, and space symbols, the query is output to the official name identification unit 22.

  The official name identification unit 22 performs the process shown in FIG. That is, the formal name identification unit 22 determines that the query “FOMA P902i” received from the input determination unit 21 in step (22s1) matches the regular expression of the entry of the formal name / regular expression dictionary 31 shown in FIG. Since there is no, go to step (22s2) and create a regular expression.

  In the regular expression creation process in step (22s2), the conversion list 41, the insertion list 51, and the formal name / regular expression dictionary 31 are used. In step (22s21), the query “FOMA P902i” is first divided at the boundaries of different character types. , [FOMA, P, 902, i]. Delete the “” (blank) between FOMA and P.

  Next, rule application processing is performed in step (22s22). The conversion list 41 is used for this processing.

  In step (22s221), the processing in steps (22s221) to (22s226) is repeated for each element of the array [F0MA, P, 902, i] created in the processing in step (22s21). At this time, the processing starts from [F0MA].

  In step (22s222), it is determined whether or not the array element [FOMA] matches the entry in the conversion list 41. Since the array element [FOMA] matches entry 1, the process proceeds to step (22s223).

  In step (22s223), the array element [FOMA] is changed to [(FOMA, former,?)], And the process proceeds to step (22s224).

  In step (22s224), since the changed array element (FOMA, former,?) Includes an alphabet, the process proceeds to step (22s225).

  In the process of step (22s225), (FOMA, Forma,?) → (FOMA, FOMA, foma, Foma, Forma,?) → (FOMA, foma, Foma, Forma,?), And go to the process of Step (22s226) move on.

  In step (22s226), not all elements have been examined, so the process returns to step (22s221).

  In step (22s221), the next array element [P] is processed.

  In step (22s222), since array element [P] does not match the conversion list, the process proceeds to step (22s224).

  In step (22s224), since the array element [P] includes alphabets, the process proceeds to step (22s225).

  In the process of step (22s225), P → (P, P, p, P) → (P, p), and the process proceeds to step (22s226).

  In step (22s226), not all elements have been examined, so the process returns to step (22s221), and the remaining [902] and [i] are processed in the same manner, and all elements are processed. The array elements are [(FOMA, foma, Foma, former,?), (P, p), 902, (i, I)]. At that point, the process ends, and the process proceeds to the regular expression creation process step (22s23) in step (22s2).

  In step (22s23), the insertion list 51 shown in FIG. 5 is separated by “, (comma)” [“”,-], and a “?” Character is added to the end to add one element [“”,-,? ] Between query array elements as [FOMA, foma, Foma, former,?), (“”,-,?), (P, p), (“”,-,?), 902, (“ ”,-,?), (I, I)], then the process proceeds to step (22s24).

  In step (22s24), array elements are connected to create a regular expression. That is, for each element of the array, steps (22s241) to (22s248) (FOMA, foma, Foma, former,?), (“”,-,?), (P, p), (“”,-,? ), 902, (“”,-,?), (I, I), and the process is repeated seven times. First of all, the processing of (FOMA, foma, Foma, former,?) Is performed.

  In the process of step (22s242), since (FOMA, foma, Foma, former,?) Are not all one character delimited by “, (comma)” in one element, the process proceeds to step (22s244). .

  In step (22s244), “, (comma)” is converted to “| (pipe)” to (FOMA | foma | Foma | former |?), And the process proceeds to step (22s245).

  In the process of step (22s245), since the array element (FOMA | foma | Foma | former |?) Has the character “?” At the end of the array element, the process proceeds to step (22s246).

  In step (22s246), the character “?” Is moved to the right outside the parenthesis outside the array element, and (FOMA | foma | Foma | former)? The process proceeds to step (22s247).

  In step (22s247), all elements have not been examined yet, so the process returns to step (22s241).

  Next, in step (22s241), processing of array elements ("",-,?) Is performed.

  In the process of step (22s242), since all the characters delimited by “, (comma)” in one element are one character in (“”,-,?), The process proceeds to step (22s243).

  In step (22s243), ", (comma)" is deleted from the array element ("",-,?), the character "()" is converted to the character "[]", and the array element ["" -?], Then proceed to step (22s245).

  In step (22s245), since the array element [“”-?] Has the character “?” At the end of the element, the process proceeds to step (22s246).

  In step (22s246), the character “?” Is moved to the right outside of the parenthesis outside the element and [“”-]? Then, the process proceeds to step (22s247).

  In step (22s247), all elements have not been examined yet, so the process returns to step (22s241), and the remaining (P, p), (“”,-,?), 902, (“”,-,? ), (i, I) are similarly processed. When all the elements have been processed, the process proceeds to step (22s248).

  If it is determined in step (22s247) that all elements have been processed, the iterative process is terminated and the process proceeds to step (22s28).

  In step (22s248), when all the elements have been processed, all the elements are connected and a character string (F0MA | Foma | foma)? [“”-]? [Pp] [“”-]? 902 [“”-]? [iI] is set, and the process proceeds to step (22s25).

  In step (22s25), since the official name “DoCoMo P902i” that matches the created regular expression is in entry 3 of the formal name / regular expression dictionary 31 in FIG. 3, the process proceeds to step (22s26).

  In step (22s26), the regular expression corresponding to the matched formal name is expanded to include the newly created regular expression.

  In step (22s261), a regular expression (DoCoMo | Docomo | docomo) [“”-] in the formal name / regular expression dictionary 31? [Pp] [“”-] 902 [“”-]? i and the newly created regular expression (FOMA | Foma | foma)? [“”-]? [Pp] [“”-]? 902 [“”-]? The two regular expressions [iI] are delimited in pairs of parenthesis expressions and stored in an array. At this time, the part without parentheses is stored after adding (), so [(DoCoMo | Docomo | docomo), [""-]? , [Pp], [“”-], (902), [“”-]? , (i)] and [(FOMA | Foma | foma)? , [“”-]? , [Pp], [“”-]? , (902), [“”-]? , [iI]]. Thereafter, the process proceeds to step (22s262).

  In step (22s262), an element that matches between the two regular expressions is searched. As a result,[[""-]? , [Pp], [“”-], (902), [“”-]? ] Match, so the group of elements before and after the match [[DoCoMo | Docomo | docomo)] and [(FOMA | Foma | foma)], [(i)] and [[iI]] The following steps (22s263) to (22s268) are repeated and compared from the previous. In that case, is the newly created array (F0MA | Foma | foma) changed such as insertion? [“”-]? [Pp] [“”-]? 902 [“”-]? Do this for the element in [iI]. First, the processing proceeds to step (22s263) in order to perform processing from [(DoCoMo | Docomo | docomo)] and [(FOMA | Foma | foma)].

  In step (22s263), a partially matching portion is searched for among a group of elements that do not match between the two regular expressions extracted in step (22s262). As a result, there is no partial match between the group of elements [(DoCoMo | Docomo | docomo)] and [(FOMA | Foma | foma)], and the process proceeds to step (22s264).

  In the process of step (22s264), since there is no matching part in the element, the process proceeds to step (22s267).

  In the process of step (22s267), add the character “?” Outside the parentheses of each element and connect them to [(DoCoMo | Docomo | docomo)? (FOMA | Foma | foma)? The process proceeds to step (22s269).

  In the process of step (22s269), since not all elements have been compared yet, the process returns to the process of step (22s262) and the comparison process is repeated for the next different part.

  In step (22s262), a matching part is searched for between the two regular expressions [(i)] and [[iI]], and the process proceeds to step (22s263).

  In the process of step (22s263), a part that partially matches is found among a group of elements that do not match between the two regular expressions extracted in the process of step (22s262). Here, only i is equal in the two regular expressions [(i)] and [[iI]]. Thereafter, the process proceeds to step (22s264).

  In the process of step (22s264), the two regular expressions [(i)] and [[iI]] differ by only one part in the element, and the other parts match, so the process proceeds to step (22s265).

  In the process of step (22s265), since the different part in the elements of the two regular expressions [(i)] and [[iI]] is one character, the process proceeds to the process of step (22s266).

  In the process of step (22s266), when the parenthesis surrounding the element is [], an unmatched character is added in the middle, and the process proceeds to step (22s269) as [[iI]].

  In the process of step (22s269), after comparing the different parts, the newly created sequence is connected to the character string [(DoCoMo | Docomo | docomo)? (FOMA ｜ Foma ｜ foma)? , [“”-]? , [Pp], [“”-]? , (902), [“”-]? , [iI]], the repetition ends, and the process proceeds to step (22s2610).

  In step (22s2610), the entry 3 of the formal name / regular expression dictionary 31 shown in FIG. 3 is changed to DoCoMo P9O2i / (DocoMo | Docomo | docomo) [""-]? [Pp] [“”-] 902 [“”-]? i to DoCoMo P902i / [(DoCoMo | Docomo | docomo)? (F0MA | Foma | foma)? [“”-]? [Pp] [“”-]? 902 [“”-]? [iI]], the process of step (22s26) is terminated, the process returns to step (22s2), the process of step (22s2) is also terminated, and the process proceeds to step (22s3).

  In step (22s3), when the regular expression creation process in step (22s2) is finished, DoCoMo P902i / [(DoCoMo | Docomo | docomo)? (FOMA ｜ Foma ｜ foma)? [“”-]? [Pp] [“”-]? 902 [“”-]? [iI]] is output to the query expansion unit 23, and the process ends.

  The query expansion unit 23 performs the process shown in FIG. That is, in step (23s1), the query expansion unit 23 performs all expansion except for the symbol part in the regular expression output by the formal name identification unit 22, and (Example 1) FOMA [“”-]? P [“”-]? 902 [“”-]? i, (Example 2) [""-]? p [“”-]? 902 [“”-]? Proceed to step (23s2) as i etc.

  In steps (23s2) to (23s8), the process proceeds for each symbol part before expansion in the regular expression.

In the case of the above (Example 1), the processing of steps (23s3) to (23s6) is not performed, and the processing of step (23s7) is performed with FOMA [“”-]? P [“”-]? 902 [“”-]? Expand i as it is,
FOMA P 902 i
FOMA P 902-i
FOMA P 902i
FOMA P-902 i
FOMA P-902-i
FOMA P-902i
FOMA P902 i
FOMA P902-i
FOMA P902i
FOMA-P 902 i
FOMA-P 9O2-i
FOMA-P 9O2i
FOMA-P-902 i
FOMA-P-902-i
FOMA-P-902i
FOMA-P902 i
FOMA-P902-i
FOMA-P902i
FOMAP 902 i
FOMAP 902-i
FOMAP 902i
FOMAP-902 i
FOMAP-902-i
FOMAP-902i
FOMAP902 i
FOMAP902-i
FOMAP902i
Then, the process proceeds to step (23s8).

In the case of the above (example 2), the result of the determination in step (23s5) is that the part before the symbol part is the head of the query, so the process of step (23s6) is performed and p [“”-]? 902 [“”-]? Expand all i's in step (23s7)
p 9O2i
p 9O2-i
p 902i
P-902 i
P-902-i
p-9O2i
p902 i
p902-i
p902i
Then, the process proceeds to step (23s8).

  In step (23s8), when all the symbol parts included in the expanded regular expression have been processed, the iterative process is terminated and the process proceeds to step (23s9).

  In the process of step (23s9), among the queries that have been expanded, those that have overlapped with the previously generated query are deleted. Since there is no duplication here, the processing is terminated and the synonym expression set FOMA P-902i, FOMA-P902I, p902i, etc. of the query that has been expanded 1728 ways, the formal name DoCoMo P902i, the regular expression [(DoCoMo | Docomo | docomo )? (F0MA | Foma | foma)? [""-]? [Pp] [""-]? 902 [""-]? [II]] is passed to the document search unit 6. Thus, the process of the question expansion unit 2 ends.

  The document search unit 6 receives the [formal name, regular expression, synonym expression set] of the query from the question expansion unit 2, and performs a search using existing techniques using the synonym expression set of the query. Then, a list of document data as a search result, a formal name, and a regular expression are output to the notation unifying unit 7.

The notation unit 7 performs the process shown in FIG. That is, because the query unification unit 7 expands the query in the question expansion unit 2 ・ Search results ・ Formal name DoCoMo P902i
Regular expression [(DoCoMo | Docomo | docomo)? (FOMA | Foma | foma)? [""-]? [Pp] [""-]? 902 [""-]? [II]]
Receive.

  In the output discriminating unit 71 of the notation unifying unit 7 that has received these, the formal name and the regular expression are input, so that they are passed to the unifying unit 72 and the processing in the unifying unit 72 proceeds.

  In the unifying unit 72, the regular expression [(DoCoMo | Docomo | docomo)? (FOMA | Foma | foma)? [""-]? [Pp] [""-]? 9O2 [""-]? [II]] Is used to find an expression including the query notation included in the set of document data as a search result, and rewritten with the official name “DoCoMo P902i”. The result is passed to the output unit 8.

  The output unit 8 presents to the user a set of document data in which the query notation is unified in an arbitrary format.

  As described above, according to the document search apparatus 10 of the present embodiment to which the document search method of the present invention is applied, the number of registered words in the dictionary can be reduced by using regular expressions, and one dictionary can be queried. It can be used for both expansion and notation fluctuation absorption, and the dictionary size can be reduced. In addition, by automatically generating a regular expression including all possible notation fluctuations, it is possible to improve the speed and coverage of new words, and at the same time reduce costs. Furthermore, since both the input query and the formal name are described in regular expressions, the possibility that the query can be specified even if a query different from the formal name is input is increased. Furthermore, by using the computer program of the present invention applied to the document search apparatus 10 of the present embodiment, a document search apparatus that implements the document search method of the present invention can be easily configured by an arbitrary computer apparatus. it can.

Functional block diagram of a document search apparatus in an embodiment of the present invention The functional block diagram which shows the detailed structure of the question expansion part in one Embodiment of this invention The figure which shows an example of a formal name / regular expression dictionary in one Embodiment of this invention The figure which shows an example of the conversion list in one Embodiment of this invention The figure which shows an example of the insertion list in one Embodiment of this invention The flowchart explaining the processing operation of the formal name specific | specification part in one Embodiment of this invention. The flowchart explaining operation | movement of the regular expression creation process in one Embodiment of this invention. The flowchart explaining operation | movement of the rule application process in one Embodiment of this invention. The flowchart explaining operation | movement of the regular expression creation process in one Embodiment of this invention. The flowchart explaining the operation | movement of the regular expression expansion process in one Embodiment of this invention. The flowchart explaining the processing operation of the query expansion part in one Embodiment of this invention. The functional block diagram which shows the detailed structure of the description unification part in one Embodiment of this invention

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Input part, 2 ... Question expansion part, 3, 4, 5 ... Database, 6 ... Document search part, 7 ... Notation unification part, 8 ... Output part, 21 ... Input discrimination | determination part, 22 ... Formal name identification part, 23 ... Query expansion part, 31 ... Regular name / regular expression dictionary, 41 ... Conversion list, 51 ... Insertion list, 71 ... Output discrimination part, 72 ... Unification part, 100 ... Document database.

Claims (6)

A database of formal name / regular expression dictionaries having as entries information that pairs the formal name of the word with the regular expression of the formal name, and an arbitrary conversion rule used when creating a regular expression for a query as a character string to be searched When a query is input to a document search device comprising a computer having a conversion list database that can be registered in a database and an insertion list database that can arbitrarily register characters and symbols to be inserted when creating a regular expression of the query, The document search method is a document search method for extracting document data including a query group obtained by expanding the query from a plurality of document data stored in a document database, and outputting the extraction result.
The document search device includes:
Accept the entered search string as a query,
Create a regular expression of the accepted query using the formal name / regular expression dictionary, the conversion list, and the insertion list, and uniquely identify the formal name of the word from the regular expression,
Using the created regular expression, expand the query to create a query group,
A document search is performed using the expanded query group, and document data including the query group is extracted.
Using the identified formal name and a regular expression corresponding to the formal name, unifying the notation of the query included in the extracted document data,
A document search method characterized by outputting information on document data with unified notation. The document search device updates the formal name / regular expression dictionary using a regular expression created when a new query is input and information on a formal name of a word specified from the regular expression. The document search method according to claim 1. A document that, when a query as a character string to be searched is input, extracts document data including a query group obtained by expanding the query from a plurality of document data stored in a document database, and outputs the extraction result A search device,
Input means for receiving the input character string to be searched as a query;
A database of formal name / regular expression dictionaries having as entries information that is pairs of formal names of words and regular expressions of the formal names;
A database of conversion lists that can be arbitrarily registered and used as conversion rules when creating regular expressions of the query;
A database of an insertion list that can arbitrarily register characters and symbols to be inserted when creating a regular expression of the query;
Create a regular expression of the accepted query using the formal name / regular expression dictionary, the conversion list, and the insertion list, and identify a formal name of the word uniquely from the regular expression Means,
Query expansion means for expanding a query using the created regular expression to create a query group,
A document search means for performing a document search using the expanded query group and extracting document data including the query group;
Using a formal name identified by the formal name identifying means and a regular expression corresponding to the formal name, a notation unifying means for unifying the notation of a query included in the extracted document data;
A document retrieval apparatus comprising: output means for outputting information of document data in which notation is unified.   The said formal name identification means has a means to create the said formal name / regular expression dictionary using the created regular expression and the information of the formal name of the word identified from this regular expression. Document retrieval device. A database of formal name / regular expression dictionaries having as entries information that pairs the formal name of the word with the regular expression of the formal name, and an arbitrary conversion rule used when creating a regular expression for a query as a character string to be searched When a query is input to a document search device comprising a computer having a conversion list database that can be registered in a database and an insertion list database that can arbitrarily register characters and symbols to be inserted when creating a regular expression of the query, A computer that operates the document search apparatus that extracts document data including a query group obtained by expanding the query from a plurality of document data stored in a document database and outputs the extraction result. A program,
Receiving the input character string to be searched as a query;
Creating a regular expression for the accepted query using the formal name / regular expression dictionary, the conversion list, and the insertion list, and uniquely identifying the formal name of the word from the regular expression;
Using the created regular expression to expand the query to create a set of queries,
Performing a document search using the expanded query group, and extracting document data including the query group;
Unifying the notation of the query included in the extracted document data using the identified formal name and a regular expression corresponding to the formal name;
And a step of outputting document data information in which notation is unified. A step of updating the formal name / regular expression dictionary using a regular expression created when a character string to be newly searched is input and information on a formal name of a word identified from the regular expression. The computer program according to claim 5.

Images