JP5225021B2 - Full-text search method, apparatus and program - Google Patents

Description

  The present invention relates to a full-text search method, apparatus, and program, and more particularly to high-speed partial matching of description contents called full-text search in the field of storing and searching a large number of XML and HTML documents that are structured data. The present invention relates to a full-text search method, apparatus, and program for a character string straddling XML data tags.

  There is a full-text search technique for quickly searching for a document containing a search keyword from an accumulated document group. This is realized by cutting out appearance words from flatly described text (character string) and indexing in which document each word appears (full text index). As a method of extracting words, a method using morphological analysis or a method of extracting a fixed number of mechanical character strings called N-grams while shifting each character one by one is used.

On the other hand, XML is one of markup languages, and data described in XML is structured and meaningful in structure. FIG. 12 shows an XML document and its structure.
XML data can be represented by a tree structure, and each section is called a “node”. In particular, the root node is “root element” (“book”), the value (description content) is “text node” (“mystery of the universe”), and what is described in the tag is “attribute” (“no = 1”) ").

  In the full-text search for the XML document (s), it is necessary to be able to target a specific range of the document in addition to the entire document. In other words, by designating a path, only a necessary part can be searched from a large XML document. A part of the target document is referred to herein as a “partial document”. FIG. 13 shows an example of a partial document. FIG. 6A shows a partial document when / book / chapter is designated, and FIG. 5B shows a partial document when / book / chapter / section is designated.

When trying to achieve full-text search for such partial documents at high speed, it is efficient to specify the partial documents and whether or not the search keywords are included in the description contents of the range (and also to calculate the fitness). There is a need (see, for example, Patent Document 1).
JP 2008-146424 A

  An XML document has a structure called “Mixed Content” that allows a mixture of text and tags. An example is shown in FIG. In the example, the <b> tag is mixed for highlighting in the text. Thus, even if it is a semantically continuous character string, it may be divided by a tag and become separate text nodes. In the conventional method including the above-mentioned Patent Document 1, since processing is performed in units of text nodes, for example, when a full-text search is performed using “Mercury”, the full-text index is indexed separately as “water” and “star”. , Can't search.

  In order to solve this, it is necessary to index the words across the text nodes. However, if the text nodes are simply connected, the structure cannot be maintained. As the simplest method, there is a method of extracting a character string obtained by connecting text nodes for every partial document and constructing a full-text index group for it. However, this method is not practical because the index size becomes too large due to duplication of text nodes.

  The present invention has been made in view of the above points. A full-text search capable of realizing a full-text search designating an arbitrary structure for an XML document including mixed contents without constructing a huge full-text index group. An object is to provide a method, an apparatus, and a program.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a full-text search method for performing a full-text search on an XML document group in which text and tags as structured data are mixed.
In an apparatus having node management means for managing a structural relationship between XML nodes, full-text index means for constructing an index on a word in the XML document, and search means for executing a search,
The node management means assigns a document ID to each document in the input XML document group, assigns a range label that allows the tags to be straddled to all nodes of the document, and stores them in the range label storage means. range labeling step (step 1),
The full-text index means cuts out a word that straddles a word and a tag from the text of the XML document, and stores the appearance position information including the appearance position of the word and the document ID in the XML document in the transposition table of the transposition table storage means. A transposition table storage step (step 2) ,
A storage process consisting of:
A range label acquisition step (step 3) in which the search means extracts a search path and a search keyword from the input search query, and acquires a document ID and a range label corresponding to the search path from the range label storage means;
An appearance position information acquisition step (step 4) of acquiring appearance position information matching the search keyword from the transposition table storage means;
A search process comprising: a node information selection step (step 5) for selecting node information in which the appearance position information of the search keyword acquired in the appearance position information acquisition step exists from the document ID and the range label acquired in the range label acquisition step. and, it was carried out,
In the range labeling step (step 1) of the storage process,
The document ID of the XML document, the preorder value and postorder value of each node, the tag name, the start character offset when all the text nodes are concatenated, and the description content are stored in the range label storage means as a range label,
In the transposition table storage step (step 2) of the storage process,
All text nodes of an XML document are concatenated, divided into character strings of a certain length, and words are extracted by the N-gram method. If the extracted words straddle the text nodes, the end of the text node is reached. A word cutting step of cutting out a word of M characters (where M is an integer of 1 or more and less than N);
For each extracted word, the node identifier preorder value including the start position, the node identifier postorder value including the end position, and the start position in the concatenated text are acquired, and the preorder value, the postorder value, and the start position Then, an appearance position information storage step of storing the document ID as appearance position information in the transposition table of the transposition table storage means is performed.

Further, the present invention (Claim 2 ), in the range label acquisition step (Step 3) of the search process,
N-gram decomposes the search keyword, gets the search word,
Obtaining the appearance position information of the search word from the transposition table of the transposition table storage means based on the search word,
The start position of the search word appearance position information is added to obtain the search keyword appearance position information.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 3 ) is a full-text search device for performing a full-text search on an XML document group in which text and tags as structured data are mixed,
A node management unit 120 that manages a structural relationship between XML nodes; a full-text index unit 130 that builds an index on a word in the XML document; and a search unit 140 that executes a search.
The node management means 120
A range label assigning unit that assigns a document ID to each document of the input XML document group, assigns a range label that allows the tags to be straddled to all nodes of the document, and stores the range label in the range label storage unit 121 122,
The full text index means 130
A transposition table storage that cuts out a word across a word and a tag from the text of the XML document and stores appearance position information including a pair of the appearance position of the word and the document ID in the XML document in the transposition table of the transposition table storage unit 131 Means 132,
The search means 140 is
A range label acquisition unit 141 that extracts a search path and a search keyword from the input search query and acquires a document ID and a range label corresponding to the search path from the range label storage unit 121;
Appearance position information that matches the search keyword is acquired from the transposition table storage means 131, and the appearance position information acquisition means 142;
Possess the range label document ID and ranges label obtained by the obtaining unit 141, a node information selecting means 143 for selecting the node information acquired occurrence position information of the search keywords occurrence position information obtaining unit 142 is present, a
The range label assigning means 122 of the node management means 120
A document ID of the XML document, a preorder value and a postorder value of each node, a tag name, a start character offset when all the text nodes are concatenated, and a means for storing the description content as a range label in the range label storage unit 121;
The transposed table storage means 132 of the full-text index means 130 is
All text nodes of an XML document are concatenated, divided into character strings of a certain length, and words are extracted by the N-gram method. If the extracted words straddle the text nodes, the end of the text node is reached. A word cutout means for cutting out a word of M characters (where M is an integer of 1 or more and less than N);
For each extracted word, the node identifier preorder value including the start position, the node identifier postorder value including the end position, and the start position in the concatenated text are acquired, and the preorder value, the postorder value, and the start position And an appearance position information storage means for storing the document ID in the transposition table of the transposition table storage means 131 as the appearance position information,
The range label acquisition unit 141 of the search unit 140 includes:
N-gram decomposition of a search keyword, acquisition of a search word, acquisition of appearance position information of the search word from the transposition table of the transposition table storage means 131 based on the search word, and start of the appearance position information of the search word Means for associating positions and obtaining appearance position information of search keywords;

Further, according to the present invention (claim 7), in the range label assigning means 122 of the node management means 120,
A means for storing the document ID of the XML document, the preorder value and postorder value of each node, the tag name, and the description content in the range label storage means as a range label;
In the transposed table storage means 132 of the full-text index means 130,
A morpheme analysis unit that divides an XML document into character strings of a certain length and extracts words straddling words and tags by performing morphological analysis on the character strings;
For each extracted word, the preorder value of the node in which the first character of the word appears and the postorder value of the node in which the last character of the word appears are obtained, and the preorder value, the postorder value, and the document ID are represented as the appearance position. Means for storing in the transposition table of the transposition table storage means 131 as information,
In the range label acquisition means 141 of the search means 140,
A morphological analysis is performed on the search keyword, a search word is acquired, and appearance position information of the search word is acquired from the transposition table of the transposition table storage unit based on the search word.

Further, according to the present invention (claim 8), in the range label assigning means 122 of the node management means 120,
A document ID of the XML document, a preorder value and a postorder value of each node, a tag name, a start character offset when all the text nodes are concatenated, and a means for storing the description content as a range label in the range label storage unit,
In the transposed table storage unit 132 of the full-text index storage unit 130,
All the text nodes of the XML document are concatenated, divided into character strings of a certain length, words are extracted by the N-gram method, and when the extracted words straddle the text nodes, the end of the text node is reached. Word extraction means for extracting M characters (where M is an integer of 1 or more),
For each extracted word, the node identifier preorder value including the start position, the node identifier postorder value including the end position, and the start position in the concatenated text are acquired, and the preorder value, the postorder value, and the start position A second range label assigning unit that stores the document ID as appearance position information in the transposition table of the transposition table storage unit 131,
In the range label acquisition means 141 of the search means 140,
The search keyword is decomposed into N-grams, the search word is obtained, the appearance position information of the search word is obtained from the transposition table of the transposition table storage means based on the search word, and the start position of the appearance position information of the search word And a means for acquiring appearance position information of the search keyword.

The present invention (Claim 4 ) is a full-text search program for causing a computer to function as each means constituting the full-text search device according to Claim 3 .

  As described above, according to the present invention, it is possible to perform a full-text search even for a word that straddles a tag (strands a text node) by creating a transposition table in which a range value assigned to a text node is expanded. In addition, it is possible to realize a full text search specifying an arbitrary structure for an XML document including mixed contents without constructing a huge full text index group.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 is a configuration diagram of the full-text search apparatus according to the embodiment of the present invention.

  The full-text search apparatus 100 shown in the figure includes a path index unit 110, a node management unit 120, a full-text index unit 130, and a search unit 140.

  The path index unit 110 is not essential for realizing the function, but is usually used for realizing high-speed search.

  The node management unit 120 identifies documents that appear at each node, assigns an ID to each document, and manages the structural relationship between the nodes. In the present invention, a “range label” is used for managing the structural relationship, and a range label is assigned to each node and stored in the range level storage unit 121 as a node table. Range labels are, for example, “S. Al-Khalifa, HV Jagadish, N. Koudas, JM Patel, D. Drivastava, and Y. Wu:“ Structural joins: A primitive for efficient XML query pattern matching, ”Proc. ICDE, p. 141, 2002 ”, preorder and postorder values are assigned to all nodes, and child nodes' preorder and postorder values are included in the parent node's preorder and postorder values. Labeled. An example of a range label is shown in FIG. For example, the “desc” node of the range label (9, 60) has a child relationship with the “chapter” node of the range label (6, 61), and the range label (9, 60) also becomes (6, 61). include. In addition, the “b” node of the range label (12, 15) is also included in (6, 61), and it is possible to discriminate at high speed not only directly in the parent-child but also in the ancestor-descendant relationship (the combination of this discrimination is This is called “structural join”). FIG. 5 shows a storage example of the node table in the range level storage unit.

  The full-text index unit 130 cuts out a word from the text obtained by connecting all the text nodes of the XML document, and records the appearance position of the word in the transposed table of the transposed table storage unit 131. The word appearance position records not only the identification of the document in which the word appears, but also the structural position of the partial document in which the word appears in the document, that is, the range label of the above-described appearing node.

  The search unit 140 receives the search query, specifies a node that matches the specified search path and search keyword from the path index unit 110, the node management unit 120, and the full-text index unit 130, and returns the node.

  The operation of the system composed of the “storage phase” and “search phase” of the XML document will be described below.

  In the storage phase, various indexes are created from the given XML document group.

  FIG. 6 shows the overall flow of the storage phase in one embodiment of the present invention.

  In the following, each of the path index unit 110, the node management unit 120, and the full-text index unit 130 performs processing on the XML document group.

  Step 110) The node management unit 120 retrieves the XML document from the XML document storage device 10.

  Step 120) Further, a document ID is assigned to the extracted document and stored in a memory (not shown).

  Step 130) The detailed operation of the node management unit 120 is shown in FIG.

  The node management unit 120 extracts the node from the memory (not shown) (step 131), identifies the path, and transmits it to the path index unit 110 (step 132). A range label is assigned to each node (step 133), and node information including a document ID, a range label, a start character offset, and the like is stored in the node table of the range label storage unit 121 (step 134). The above process is repeated for all the extracted nodes (step 135).

  As described above, the node management unit 120 stores the XML document shown in FIG. 4 in the node table of the range level storage unit 121 as shown in FIG. Stores document ID (docID), range label (pre, post), tag name (tag), start character offset (offset) when all text nodes are connected, and description content (value). However, “offset” is not essential for the morphological analysis method.

Step 140) The path index 110 stores the relationship to the node information (group) of the node management unit 120 corresponding to the path for each path. In the present invention, the specific method is not limited.
(A) A method of assigning an ID to a path and holding the path in common between the path index unit 110 and the node management unit 120;
(B) A method of holding the address (pointer / offset) stored in the node management unit 120 in the path index unit 110;
And so on.

  Step 150) The full-text index unit 130 concatenates all text nodes, cuts out words from the text by the morphological analysis method or the N-gram method, and uses the transposition table to store the appearance position in the transposition table storage unit 131. Record. FIG. 8 shows an operation flow of the full-text index unit 130.

  All the text nodes stored in the memory (not shown) in step 120 are connected (step 151), and words are cut out using the morphological analysis method or the N-gram method (step 152). The node management unit 120 is inquired based on the extracted word, and the position information of the appearance location of the word is acquired (step 153). When the word is not an existing word (No at Step 154), the word is registered in the transposition table of the transposition table storage unit 131 (Step 155). Appearance position information such as a document ID and a range label is additionally stored in the transposition table of the transposition table storage unit 131 (step 156). The processing from step 153 onward is repeated for all the words stored in the memory (not shown) (step 157).

  Next, the search phase will be described.

  FIG. 9 shows the flow of the search phase in one embodiment of the present invention.

  Step 210) The search unit 140 extracts a search path and a search keyword from the input search query.

  Step 220) The relationship to the node information (group) matching the search path is acquired from the path index unit 110.

  Step 230) The document ID and range label of the node information (group) corresponding to the above are acquired from the node management unit 120.

  Step 240) The appearance position information (group) matching the search keyword is acquired from the full-text index unit 130.

  Step 250) A structure join is performed for the document ID and range label of the node information (group) and the document ID and range label of the appearance position information (group) of the search keyword (node information (group) in which the appearance position information exists). select).

  Step 260) If necessary, the node management unit 120 assembles and returns a partial document corresponding to the selected node information (group).

[First embodiment]
<Storage phase>
In the present embodiment, an example will be described in which the morphological analysis method is used when the full-text index unit 130 extracts words in the storage phase.

  In step 152 of FIG. 8, the full-text index unit 130 performs the following processing.

  (1) The text is divided into fixed lengths (usually sentence by sentence), and the following processing is performed sequentially.

    a) Morphological analysis is performed to cut out a word (group). The following processing is performed for each word.

      i. Get the preorder value of the node containing the start position.

      ii. Get the postorder value of the node containing the end position.

      b) Record the document ID, preorder value, and postorder value in the transposition table.

  An example of creating a transposition table is shown in FIG. The transposition table of the transposition notation part 131 is a table in which appearance positions (groups) are grouped for each word. In the example shown in the figure, “Mercury” and “Venus” straddle the tags, so the range labels are also (13, 17), (19, 23), and straddle the tags.

<Search phase>
The search unit 140 acquires appearance position information (group) that matches the search keyword in the following procedure.

  (1) A morphological analysis is performed on the search keyword to obtain a search word.

  (2) Based on the search word, appearance position information (group) of the search word is obtained from the transposed table storage unit 131 of the full-text index unit 130.

  As a specific example, the processing in the case of a search query in which the search path “/ book / chapter” and the search keyword “Mercury / Venus” are written in the node table of FIG. 5 and the transposition table of FIG. 11 is shown.

  Step 1: Retrieve the search path “/ book / chapter” and the search keyword “Mercury / Venus” from the search query.

  Procedure 2: The relationship to the node information corresponding to the search path “/ book / chapter” is acquired from the path index unit 110.

  Procedure 3: The document ID and range label are acquired from a plurality of lines including the second line in FIG. 6 as the corresponding node information group from the node management unit 120.

(001, 6, 61), ...
Procedure 4: Appearance position information (group) that matches the search keyword “Mercury / Venus” is acquired from the full-text index unit 130 by the following procedure.

    (A) Morphological analysis of “Mercury / Venus” to obtain the search words “Mercury” and “Venus”.

(B) From the transposition table of FIG.
Appearance position information group of “Mercury”: “(001, 13, 17),…”
Appearance position information group of “Venus”: “(001, 19, 23), ...” is obtained.

  Step 5: For the document ID and range label group corresponding to the search path “/ book / chapter” obtained in step 3 and the appearance position information group matching the search keyword “Mercury / Venus” obtained in step 4 , Do structural joins.

In this example, at least as the correct combination,
(001,6,61) and (001,13,17) and (001,19,23)
Is selected.

  It should be noted that when there are a plurality of search words as in this example, the node information that is combined with the appearance position information of all the search words is correct.

  Procedure 6: When a partial document is required as a search result, the partial document is assembled from the node information selected by the structure join.

  In the case of (001, 6, 61), the node table (from FIG. 5, the document ID is 001 and the range label is included in (6, 61) (lines 2 to 9 in FIG. 5 and subsequent lines) is extracted. Assemble the partial document as the return result.

Similarly, when the search path is “/ book / chapter / desc / b”, the node information obtained in step 3 is
(001, 12, 15), (001, 18, 21), ...
Thus, none of the appearance position information obtained in the procedure 4 is combined. As described above, even if a part of the search keyword is included in the search path, the operation is performed correctly so that the search keyword is not included in the result unless the relation includes all the keywords.

[Second Embodiment]
In the present embodiment, a case where a word is cut out by the N-gram method is shown. As a specific example, a case where N = 2, that is, a Bi-gram is shown.

<Storage phase>
The full text index unit 130 performs the following processing.

  Procedure 1: All text nodes are connected (step 152).

  Procedure 2: Divide into character strings of a certain length (usually sentence by sentence) and perform the following processing in sequence.

(A) A word (group) is cut out by the N-gram method. However, in addition to normal N-gram clipping, clipping is performed in consideration of the unit of the text node. That means
i. Mechanically cut out N characters from the concatenated text.

    ii. When the extracted word straddles the text node, the M characters up to the end of the text node are also extracted in duplicate. This is for dealing with search words with fewer than N characters without omission.

  (B) The following processing is performed for each extracted word.

     i. Get the preorder value of the node containing the start position.

    ii. Get the postorder value of the node containing the end position.

   iii. Record the document ID, preoder value, postorder value, and start position in the transposition table.

  An example of creating a transposition table is shown in FIG. Unlike the example of the morphological analysis method, the start position (position) in the character string that concatenates all the text nodes is also stored as the appearance position information, and the two-character word across the tags such as “Mercury” and “Water” , One character up to the end of the text node is stored in duplicate.

<Search phase>
The search unit 140 acquires appearance position information (group) that matches the search keyword in the following procedure.

  Procedure 1: The search keyword is decomposed into N-grams to obtain a search word (group).

  Procedure 2: Appearance position information (group) is obtained for each search word from the transposition table of the transposition notation part 131.

  Procedure 3: The search word appearance start position is matched, and the search keyword appearance position information is obtained.

  As a specific example, the search path “/ book / chapter” and the search keyword “Mercury” are stored in the node table of the range label storage unit 121 of the node management unit 120 of FIG. 5 and the transposition table of the transposition table storage unit 131 of FIG. The processing for a search query with “Venus” is shown.

  Procedure 1: (similar to the first embodiment) The search path “/ book / chapter” and the search keyword “Mercury / Venus” are extracted from the search query.

  Procedure 2: (similar to the first embodiment) The relationship to the node information corresponding to the search path “/ book / chapter” is acquired from the path index unit 110.

  Procedure 3: (similar to the first embodiment) The document ID and the range label are acquired from the node management unit 120 from a plurality of rows including the second row of FIG.

(001, 6, 61), ...
Procedure 4: Appearance position information (group) that matches the search keyword “Mercury / Venus” is acquired from the full-text index unit 130 by the following procedure.

  (A) Bi-gram decomposition of “Mercury / Venus” to obtain the search words “Mercury”, “• Gold”, “Venus”. At the same time, when the start position of “Mercury” is k, “Venus” has a relationship of k + 2, and “Venus” has a relationship of k + 3.

(B) From the transposition table of FIG.
Appearance position information group of "Mercury": "(001, 13, 17, 9), ..."
Appearance position information group of "・ Gold": "(001, 16, 20, 11), ..."
Appearance position information group of “Venus”: “(001, 19, 23, 12),.

  (C) A combination of the appearance position information is found that satisfies the relationship of the start position (position) with the same document ID. In this example, each first column is filled.

  (D) A search keyword appearance position information group is created from the above combinations.

(001, 13, 23), ...
The document ID is common, the preoder value is “Mercury”, and the postorder value is “Venus”.

  Step 5: For the document ID and range label group corresponding to the search path “/ book / chapter” obtained in step 3 and the appearance position information group matching the search keyword “Mercury / Venus” obtained in step 4 , Do structural joins.

In this example, at least as the correct combination,
(0061, 6, 61) and (001, 13, 23)
Is selected.

  Procedure 6: (Similar to the first embodiment) When a partial document is required as a search result, the partial document is assembled from the node information selected by the structure join.

  In the case of (001,6,61), the node table (from FIG. 5, the document ID is 001, and the range label is included in (6,61) (lines 2-9 in the figure and beyond) is extracted. The partial document is assembled and used as the return result.

Similarly, when the search path is “/ book / chapter / desc / b”, the node information obtained in step 3 is
(001, 12, 15), (001, 18, 21), ...
Thus, none of the appearance position information obtained in the procedure 4 is combined. Thus, even if a part of the search keyword is included in the search path, it operates correctly so that it is not included in the result unless the relationship includes all the keywords.

  Note that the operation of the components of the full-text search apparatus 100 described in FIG. 3 can be constructed as a program and installed in a computer used as the full-text search apparatus, or distributed via a network. .

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiments and examples, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to structured data, particularly XML full-text search technology.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the full text search apparatus in one embodiment of this invention. It is an example of the range label in one embodiment of this invention. It is a storage example of the node table of the range level memory | storage part in one embodiment of this invention. It is the whole storage phase flow in one embodiment of the present invention. It is a flow (storage phase) of the node management part in one embodiment of this invention. It is a flow of the storage phase of the full-text index part in one embodiment of this invention. It is a flow of the search phase in one embodiment of the present invention. It is a creation example (morpheme analysis method formula) of the transposition table of 1st Example of this invention. It is a creation example (Bi-gram system) of the first embodiment of the present invention. It is a figure which shows an XML document and its structure. It is an example of the partial document which a path | pass shows. It is an XML example of mixed contents.

Explanation of symbols

10 XML document storage device 100 Full-text search device 110 Path index unit 110
120 node management means, node management section 121 range label storage means, range label storage section 122 range label assigning means 130 full-text index means, full-text index section 131 transposed table storage means, transposed notation billion section 132 transposed table storage means 140 search means, Search unit 141 Range label acquisition unit 142 Appearance position information acquisition unit 143 Node information selection unit

Claims (4)

A full-text search method for performing a full-text search on an XML document group in which text and tags as structured data are mixed,
In an apparatus having node management means for managing a structural relationship between XML nodes, full-text index means for constructing an index on a word in the XML document, and search means for executing a search,
The node management means assigns a document ID to each document of the input XML document group, assigns a range label that allows the tags to be straddled to all nodes of the document, and stores it in the range label storage means A range labeling step to be performed ;
The full-text index means cuts out a word that straddles a word and a tag from the text of the XML document, and uses the transposition table of the transposition table storage means as the appearance position information composed of the appearance position of the word and the document ID in the XML document. A transposition table storage step to store in
A storage process consisting of:
A range label acquiring step in which the search means extracts a search path and a search keyword from the input search query, and acquires a document ID and a range label corresponding to the search path from the range label storage means;
An appearance position information acquisition step of acquiring appearance position information that matches the search keyword from the transposition table storage means;
A node information selection step of selecting node information in which the appearance position information of the search keyword acquired in the appearance position information acquisition step exists from the document ID and the range label acquired in the range label acquisition step;
A search process consisting of
And
In the range labeling step of the storing process,
The document ID of the XML document, the preorder value and postorder value of each node, the tag name, the start character offset when concatenating all the text nodes, and the description content are stored in the range label storage means as a range label,
In the transposition table storage step of the storage process,
All the text nodes of the XML document are concatenated, divided into character strings of a certain length, words are cut out by the N-gram method, and when the cut out words straddle the text nodes, the end of the text node A word extracting step of cutting out a word of up to M characters (where M is an integer of 1 or more and less than N);
For each extracted word, the node identifier preorder value including the start position, the node identifier postorder value including the end position, and the start position in the concatenated text are acquired, and the preorder value, the postorder value, and the start position , An appearance position information storage step of storing the document ID as the appearance position information in the transposition table of the transposition table storage means ,
Full-text search method comprising rows Ukoto. In the range label acquisition step of the search process,
N-gram decomposition of the search keyword to obtain a search word,
Obtaining the appearance position information of the search word from the transposition table of the transposition table storage means based on the search word,
The full-text search method according to claim 1, wherein the start position of the search word appearance position information is associated with each other and the appearance position information of the search keyword is acquired. A full-text search apparatus that performs a full-text search on an XML document group in which text and tags that are structured data are mixed,
Node management means for managing a structural relationship between XML nodes, full-text index means for constructing an index on a word in the XML document, and search means for executing a search,
The node management means includes
A range label assigning unit that assigns a document ID to each document of the input XML document group, assigns a range label that allows the tags to be straddled to all nodes of the document, and stores the range label in the range label storage unit. Have
The full-text index means is:
A transposition table storage that cuts out a word that spans a word and a tag from the text of the XML document, and stores appearance position information including a pair of the appearance position of the word and the document ID in the XML document in a transposition table of a transposition table storage unit Having means,
The search means includes
A range label acquisition unit that extracts a search path and a search keyword from the input search query, and acquires a document ID and a range label corresponding to the search path from the range label storage unit;
From the transposition table storage means, to obtain appearance position information that matches the search keyword, appearance position information acquisition means,
From the document ID and the range label acquired by the range label obtaining means, have a, and node information selecting means occurrence position information of the search keyword selects a node information existing acquired by the occurrence position information obtaining unit ,
The range label giving means of the node management means is
A document ID of the XML document, a preorder value and a postorder value of each node, a tag name, a start character offset when all the text nodes are concatenated, and a description content stored in the range label storage unit as a range label,
The transposition table storage means of the full-text index means is
All the text nodes of the XML document are concatenated, divided into character strings of a certain length, words are cut out by the N-gram method, and when the cut out words straddle the text nodes, the end of the text node A word segmentation means for segmenting a word of up to M characters (where M is an integer greater than or equal to 1 and less than N);
For each extracted word, the node identifier preorder value including the start position, the node identifier postorder value including the end position, and the start position in the concatenated text are acquired, and the preorder value, the postorder value, and the start position , Appearance position information storage means for storing the document ID as the appearance position information in the transposition table of the transposition table storage means,
The range label acquisition unit of the search unit includes:
The search keyword is decomposed into N-grams, a search word is obtained, the appearance position information of the search word is obtained from the transposition table of the transposition table storage means based on the search word, and the appearance position information of the search word A full-text search apparatus comprising means for associating start positions and acquiring appearance position information of the search keyword . A full-text search program for causing a computer to function as each means constituting the full-text search device according to claim 3 .

Images