WO2015025467A1 - Text character string search device, text character string search method, and text character string search program - Google Patents

Text character string search device, text character string search method, and text character string search program Download PDF

Info

Publication number
WO2015025467A1
WO2015025467A1 PCT/JP2014/003817 JP2014003817W WO2015025467A1 WO 2015025467 A1 WO2015025467 A1 WO 2015025467A1 JP 2014003817 W JP2014003817 W JP 2014003817W WO 2015025467 A1 WO2015025467 A1 WO 2015025467A1
Authority
WO
WIPO (PCT)
Prior art keywords
prefix
character string
search
score
specified
Prior art date
Application number
PCT/JP2014/003817
Other languages
French (fr)
Japanese (ja)
Inventor
穣 岡嶋
康高 山本
Original Assignee
Necソリューションイノベータ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Necソリューションイノベータ株式会社 filed Critical Necソリューションイノベータ株式会社
Priority to EP14838200.5A priority Critical patent/EP3037986A4/en
Priority to JP2015532688A priority patent/JP6072922B2/en
Priority to CN201480046496.4A priority patent/CN105474214A/en
Priority to US14/909,793 priority patent/US20160196303A1/en
Publication of WO2015025467A1 publication Critical patent/WO2015025467A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/90335Query processing
    • G06F16/90344Query processing by using string matching techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/205Parsing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/279Recognition of textual entities

Definitions

  • the present invention relates to a character string search device, a character string search method, and a character string search program for searching for a key that partially matches an input character string.
  • a search keyword of a search candidate is displayed on an input form of a search engine, or a URL as a candidate is displayed on a URL (Uniform ResourceatorLocator) input form of a Web browser.
  • a URL Uniform ResourceatorLocator
  • displaying conversion candidates at the time of prediction conversion of IME (Input Method Editor) or displaying correct spelling candidates in the spell checker are examples of input support.
  • Such input support is realized as a dictionary search.
  • a character string that the user is likely to input is registered in advance in the dictionary as a key.
  • the dictionary is searched using the character string input by the user as a search query, an appropriate key is acquired as an input candidate, and displayed on the screen.
  • search keyword recommendation search keywords previously input by the user are registered in the dictionary in advance and used as input candidates.
  • Top-k search Topic-k dictionary search
  • Non-Patent Document 1 describes a data structure that obtains a high-order key at a high speed from keys that coincide with each other by using a RMQ Trie and a RMQ (Ranged Minimum Query) structure. .
  • FIG. 9 is an explanatory diagram showing the RMQ Trie.
  • the node v having the search query P as a prefix is found, and the key range [a, b] under the node v is obtained. All keys included in [a, b] have the search query P as a prefix.
  • the top k keys having the search query P as a prefix are obtained. .
  • Non-Patent Document 1 describes two other types of data structures that are used to acquire a high-order key at a high speed from keys that coincide with each other in the same manner as RMQ Trie.
  • Non-patent document 2 describes Top-k search in document search. This method realizes Top-k search by adding additional data necessary for Top-k search to the data structure based on the data structure for document search.
  • Non-Patent Document 1 by using each data structure described in Non-Patent Document 1, it is possible to acquire a key candidate that matches forward at high speed, but it is difficult to acquire a key candidate that partially matches. is there.
  • Top-k search in document search can be realized.
  • the data used for document search is large in size, there is a problem that when the search method used for document search is directly applied to a dictionary, the target data size becomes large.
  • an object of the present invention is to provide a character string search device, a character string search method, and a character string search program that can perform partial match search of a character string at high speed while reducing the amount of data.
  • a character string search device is a character string that searches for a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched.
  • a prefix for identifying a set of prefixes ending with an input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string, in the search device From the set identification part and the set of prefixes ending with the input character string, each prefix is defined with the largest character string score among the character string scores associated with the search candidate character string beginning with that prefix.
  • Character string specifying the search candidate character string with the highest character string score from the prefix specification part that specifies the prefix with the highest prefix score and the search candidate character string starting with the specified prefix With department And wherein the door.
  • the character string search method is a character string that searches for a search candidate character string that includes an input character string from a set of search candidate character strings that are associated with a character string score that indicates the degree to be searched with priority
  • a prefix for identifying a set of prefixes ending with an input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string in the search method From the set identification step and a set of prefixes ending with the input string, each prefix is defined with the largest string score among the string scores associated with the search candidate string starting with that prefix.
  • a prefix identification step that identifies the prefix with the largest prefix score, and a string identification that identifies the search candidate string with the largest string score from among the search candidate strings that begin with the identified prefix Characterized in that it comprises a step.
  • a character string search program provides a computer for searching for a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched.
  • An applied character string search program wherein a prefix ending with an input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string
  • Prefix set identification processing that identifies a set of words, out of a set of prefixes that end with the input character string, out of the character string score associated with the search candidate character string that starts with that prefix for each prefix
  • a prefix identification process that identifies the prefix with the largest prefix score defined by the largest string score, and a search string that begins with the identified prefix, and a string Core characterized in that to execute a string specifying process for specifying the maximum search candidate character strings.
  • a partial match search of a character string can be performed at high speed while reducing the amount of data.
  • the present invention realizes a data structure for searching for a high-order key that partially matches an inputted character string at high speed by extending XBW, which is a dictionary data structure, to Top-k search. is there.
  • each key that is a search candidate character string is assigned a score (hereinafter referred to as a character string score) indicating the degree of search priority, and a set of keys is represented by a trie structure. .
  • all the prefixes of the keys included in the key set are represented by the XBW structure used for dictionary search.
  • the character string search device of the present invention uses this XBW structure to specify a prefix range ending with the input character string.
  • Each prefix is associated with a maximum score (hereinafter referred to as a prefix score) among keys starting with the prefix. Therefore, the character string search device identifies the prefix with the largest prefix score within the identified prefix range.
  • the RMQ structure is used to specify the maximum prefix score within the specified prefix range.
  • the RMQ structure used to represent the relationship between the prefix and the prefix score is referred to as a first RMQ structure.
  • the character string search device uses the first RMQ structure to specify the prefix with the largest prefix score within the specified prefix range.
  • the character string search device identifies the key with the largest character string score among the keys that start with the identified prefix.
  • the identified prefix corresponds to one node in the trie tree. Therefore, in order to specify the maximum character string score in the range of keys existing under each node, the RMQ structure is used as in the case of specifying the maximum prefix score.
  • the RMQ structure used to represent the relationship between the key and the character string score is referred to as a second RMQ structure.
  • the character string search device uses the second RMQ structure to specify the key with the maximum character string score from the range of keys starting with the specified prefix.
  • the character string search device After specifying the key with the maximum character string score, the character string search device performs a process of searching for a key having a character string score of the second or later in order to apply this character string search to the Top-k search.
  • the position where the second and subsequent keys exist is the second and subsequent keys starting with the specified prefix, or the first and subsequent prefixes that have not been specified yet.
  • the character string search device holds the prefix score of the specified prefix and the character string score of the specified key.
  • the character string search device selects a key or prefix that is the maximum score among the character string scores and prefix scores it holds. If it is a key, the key with the next highest string score is searched for among the keys that start with the same prefix. If it is a prefix, a prefix having the next highest prefix score is searched for. By repeating this, it is possible to efficiently search for keys having a higher character string score among keys including the input character string.
  • FIG. FIG. 1 is a block diagram showing a configuration example of a first embodiment of a character string search device according to the present invention.
  • the character string search device according to the present embodiment includes an input unit 10, a prefix set specifying unit 20, a search management unit 30, a prefix specifying unit 31, a character string specifying unit 32, an output unit 40, and search information. And a storage unit 50.
  • the input unit 10 inputs a character string of one character or more.
  • the character string search device searches for a key that partially matches an input character string.
  • an input character string is referred to as a search query (or simply a query) P.
  • the search information storage unit 50 stores a set of keys that are search candidate character strings.
  • the character string score is associated with the key used in this embodiment. That is, the character string search device of the present embodiment preferentially searches for a key having a higher character string score from the set of keys.
  • the search target key is represented using the structure of the trie tree.
  • FIG. 2 is an explanatory diagram illustrating an example of a trie tree corresponding to a key.
  • the trie tree is configured to place a common character at a common node.
  • the search information storage unit 50 may store the key itself represented by the trie tree, or may store only the structure of the trie tree, as will be described later.
  • each leaf node represented by a tree structure corresponds to each key. Accordingly, the search information storage unit 50 stores the score (character string score) of each key illustrated in FIG. 2 in association with each leaf node. Thereby, when the trie tree is searched and the leaf node is reached, the character string score corresponding to the key represented by the leaf node can be acquired.
  • the search information storage unit 50 stores a set of prefixes p so that a character string ending with the query P can be searched.
  • the prefix p is a character string of one or more consecutive characters extracted from the first character of each key. This set of prefixes p may be sorted lexicographically from the end.
  • XBW a structure called XBW is used to represent such a set of prefixes.
  • XBW is a data structure that can efficiently represent a labeled tree structure. By expressing the trie tree using this XBW structure, a range search of the prefix p ending with the query P becomes possible.
  • the XBW is known to be able to realize a data structure that realizes an equivalent operation by two types of methods.
  • the first XBW has a structure in which characters representing child nodes are associated with each prefix of the dictionary at a node on the trie tree corresponding to the prefix.
  • the second XBW has a structure in which each prefix of the dictionary is associated with an ID of a prefix that becomes a parent node in a node on the trie tree corresponding to the prefix.
  • FIG. 3 is an explanatory diagram showing an example of the first XBW.
  • prefixes corresponding to the nodes of the trie tree are arranged in lexicographic order from the end, and characters representing child nodes are associated with the respective prefixes.
  • FIG. 4 is an explanatory diagram showing an example of the second XBW.
  • the prefixes corresponding to the nodes of the trie tree are arranged in lexicographic order from the end, and an ID is assigned to each prefix.
  • Each prefix is associated with its parent ID.
  • Such a structure makes it possible to move to the next parent node.
  • a range search can be performed for the prefix p ending with the query P.
  • the second XBW since it has only the parent ID, it is difficult to search for a child node. However, even when the second XBW is used, a range search of the prefix p ending with the query P is possible. In this embodiment, any XBW can be used.
  • the first XBW is described in Reference Document 1, and the second XBW is described in Reference Document 2.
  • ⁇ Reference 1 > Paolo Ferragina, Fabrizio Luccio, Giovanni Manzini and S. Muthukrishnan, "Structuring labeled trees for optimal succinctness, and beyond", FOCS '05 Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science, Pages 184-196
  • a score (that is, a prefix score) is defined for each prefix.
  • the prefix score is defined by the largest character string score among character string scores associated with keys starting with the prefix. When this score is expressed by an expression, it becomes as shown in Expression 1.
  • the score on the right side in Expression 1 represents a character string score, and the score on the left side in Expression 1 represents a prefix score.
  • Score (p) max ⁇ Score (key starting with prefix p) ⁇ (Formula 1)
  • the prefix score is the largest character string score among the keys existing under the node.
  • a first RMQ structure is added to the XBW structure, and a prefix score corresponding to each node can be specified using the first RMQ structure.
  • the prefix score of each prefix is stored in an array used in RMQ.
  • the array in which the prefix score is stored is referred to as a prefix score column R p . Since the prefixes are sorted based on the end, prefixes that end with the same string are specified in a single range. Therefore, by using the first RMQ structure, the maximum value in any range of the prefix score string R p can be specified.
  • the character string score of each key can be specified using the second RMQ structure.
  • the character string score of each key is stored in an array used in RMQ.
  • an array in which character string scores are stored is referred to as a character string score string R k . Since each key is sorted from the top, keys starting with a certain prefix are specified in a range of connections. Therefore, the maximum value in an arbitrary range of the character string score string R k can be specified by using the second RMQ structure.
  • FIG. 5 is an explanatory diagram illustrating an example of a data structure stored in the search information storage unit.
  • the XBW structure in the present embodiment is represented by a set S having a set of three elements for each node of the trie tree.
  • S last is a binary flag, and is 1 when the node is the last child of the node's parent node, and 0 otherwise.
  • S ⁇ is a character represented by the node.
  • S ⁇ is a prefix corresponding to the parent node of the node, and is a character string obtained by sequentially combining characters from the root to the parent node. Note that S ⁇ does not include the characters of the node itself.
  • the set of three elements is sorted in lexicographic order by comparing from the last character of the prefix included in each element to the first character.
  • row numbers are assigned to the sorted groups (S ⁇ , S ⁇ , S last ) in order from the top.
  • $ indicates the beginning of the key
  • # indicates the end of the key.
  • a prefix score Rp is defined for each prefix. Since the prefix score Rp is calculated from the character string score associated with each key as described above, the prefix score does not need to be explicitly retained.
  • the prefix IDs illustrated in FIG. 5 are assigned in the order in which all prefixes included in the dictionary are sorted from the end. Therefore, the order of prefix IDs matches the order of prefixes for which 1 is set in S last .
  • a range of prefixes ending with the query P can be specified.
  • the line corresponding to the prefix ending with the query “ab” corresponds to the line numbers 7 to 9 (that is, lines corresponding to “$ ab” and “$ cab”).
  • the ID of the prefix having the maximum score in the range can be acquired with the first RMQ structure. Furthermore, by using the first RMQ structure recursively, it is possible to acquire a prefix ID having a score of second or lower.
  • the prefix set specifying unit 20 specifies a set of prefixes including the input character string from the set of prefixes stored in the search information storage unit 50. Specifically, the prefix set specifying unit 20 specifies a set of prefixes ending with the input character string. For example, when the search information storage unit 50 stores a set of prefixes illustrated in FIG. 5, when “ab” is input as a character string, the prefix set specifying unit 20 sets the line numbers 7 to 9. A prefix existing in the range (ie, “$ ab”, “$ cab”) is specified as a set of prefixes.
  • the prefix specifying unit 31 specifies a prefix having a higher prefix score from the set of prefixes specified by the prefix set specifying unit 20.
  • the prefix identification unit 31 may identify a prefix having the largest prefix score or a prefix corresponding to the top n prefix scores (n is an arbitrary natural number).
  • the character string specifying unit 32 specifies a key having a higher character string score among keys starting with the specified prefix.
  • the character string specifying unit 32 may search for a key having the largest character string score or a key corresponding to the top m characters of the character string score (m is an arbitrary natural number).
  • the prefix identifying unit 31 identifies the prefix as “$ ab”.
  • the keys starting with the identified prefix “$ ab” are “aba” and “abcc”.
  • the character string score of “abba” is 3, and the character string score of “abcc” is 9.
  • the character string specifying unit 32 may select “abcc” as a key.
  • the search management unit 30 specifies a prefix range to be searched by the prefix specifying unit 31.
  • the search management unit 30 specifies a key range to be searched by the character string specifying unit 32, and specifies the key specified by the character string specifying unit 32 as a search target key.
  • the search management unit 30 first specifies the prefix range specified by the prefix set specifying unit 20 as the prefix range searched by the prefix specifying unit 31. Then, the search management unit 30 specifies a key starting with a prefix within the specified range as a key range to be searched by the character string specifying unit 32. Then, the search management unit 30 specifies the key specified by the character string specifying unit 32 as the search target key.
  • the search management unit 30 specifies a range excluding the already specified key from the keys starting with the key prefix specified by the character string specifying unit 32. Further, the search management unit 30 specifies a range obtained by removing the prefix specified by the prefix specifying unit 31 from the set of prefixes specified by the prefix set specifying unit 20.
  • the search management unit 30 causes the prefix specifying unit 31 and the character string specifying unit 32 to execute each process.
  • the prefix specifying unit 31 specifies the prefix having the maximum prefix score from the prefix range specified by the search management unit 30.
  • the character string specifying unit 32 specifies the key having the maximum character string score from the key range specified by the search management unit 30.
  • the search management unit 30 compares the prefix score specified from the prefix range with the character string score of the key specified from the key range. As a result of the comparison, if the highest score is a character string score, a key starting with the same prefix as that key is searched for a key having the next highest character string score. Specifically, the search management unit 30 excludes the key from the key range used when the key is specified, and bisects the key range to specify the two ranges.
  • the character string specifying unit 32 specifies a key having the maximum character string score from the two ranges.
  • the search management unit 30 excludes the prefix from the prefix range used when specifying the prefix and bisects the two ranges to specify the two ranges.
  • the prefix specifying unit 31 specifies a prefix having the maximum prefix score from the range.
  • the output unit 40 outputs the key specified by the search management unit 30 as a search result.
  • the prefix set specifying unit 20, the search managing unit 30, the prefix specifying unit 31, and the character string specifying unit 32 are realized by a CPU of a computer that operates according to a program (character string search program).
  • the program is stored in a storage unit (not shown) of a character string search device, and the CPU reads the program, and according to the program, a prefix set specifying unit 20, a search management unit 30, a prefix specifying unit 31 and The character string specifying unit 32 may be operated.
  • each of the prefix set specifying unit 20, the search managing unit 30, the prefix specifying unit 31, and the character string specifying unit 32 may be realized by dedicated hardware.
  • FIG. 6 is a flowchart showing an operation example of the character string search apparatus according to the present embodiment.
  • k keys are selected as candidates.
  • the search management unit 30 holds a priority queue (prefix and prefix score pair specified by the prefix specifying unit 31 and a key and character string score pair specified by the character string specifying unit 32). (Not shown).
  • This priority queue is a queue for holding candidate information. In the following description, a priority queue is simply referred to as a queue.
  • the input unit 10 inputs a character string to be searched (step S11).
  • the prefix set specifying unit 20 refers to the search information storage unit 50 and specifies a set of prefixes including the input character string (step S12).
  • the prefix identification unit 31 identifies a prefix having the largest prefix score from the prefix set identified by the prefix set identification unit 20, and holds the identified prefix / prefix score pair in the queue. (Step S13).
  • the character string specifying unit 32 specifies the key having the highest character string score from the keys starting with the specified prefix, and holds the specified key / character string score pair in the queue (step S14).
  • the search management unit 30 specifies the prefix or key of the maximum score among the prefix scores or character string scores held in the queue (step S15). Then, the search management unit 30 determines whether the maximum score is a prefix score or a character string score (step S16).
  • the search management unit 30 specifies the key of the character string score as an output target and excludes it from the queue (step S17). . Then, the character string specifying unit 32 specifies the key having the next highest character string score after the excluded key in the key range used when specifying the excluded key, and the specified key and character string The score pair is held in the queue (step S18).
  • the search management unit 30 excludes the prefix of the prefix score from the queue (step S19). Then, the prefix identification unit 31 identifies and identifies a prefix having the next largest prefix score after the excluded prefix in the range of prefixes used to identify the excluded prefix. A pair of prefix and prefix score is held in the queue (step S20).
  • the character string specifying unit 32 specifies the key having the largest character string score from the keys starting with the prefix specified in step S20, and holds the specified key and character string score pair in the queue ( Step S21).
  • step S22 If the queue is empty or the maximum score in the queue is lower than the kth largest string score found so far (Yes in step S22), the search management unit 30 will The key found in (1) is output as the upper key (step S23). On the other hand, when the key is not empty and the maximum score in the queue is not lower than the k-th largest character string score that has been found so far (No in step S22), the processing after step S15 is performed. Repeated.
  • the prefix / prefix score pair specified by the prefix specifying unit 31 and the key / string score pair specified by the character string specifying unit 32 are put in the same priority queue. By setting it, it is possible to take out the pair with the largest score from the prefix score or the character string score.
  • FIG. 7 is an explanatory diagram illustrating an example of processing for selecting a key having a large character string score.
  • the list illustrated in the left frame of FIG. 7 is a list schematically showing the XBW structure, where a number represents a prefix score and a letter represents a prefix.
  • the list illustrated in the right frame of FIG. 7 is a list schematically showing a trie tree, where a number represents a character string score and a character represents a key.
  • the prefix set specifying unit 20 uses “aggres”, “congres”, and “progress” as candidates from the set of prefixes represented by the XBW structure for the key range that partially matches the character string “gres”. Identify. If a prefix is specified, keys that begin with that prefix can be specified.
  • the prefix specifying unit 31 selects a prefix having the highest prefix score from prefixes ending with the input character string “gres” from the determined set of prefixes.
  • FIG. 7 shows a state in which the selected prefixes are arranged in descending order of prefix score.
  • the prefix score of “congres” is 45, which is the largest. Therefore, the prefix specifying unit 31 specifies “congres” as a prefix.
  • the character string specifying unit 32 selects a key having the largest character string score from keys starting with the selected prefix.
  • a key having the largest character string score is “congress”. Therefore, the character string specifying unit 32 specifies “congress” as the first key, and the search management unit 30 specifies the specified “congress” as a search target key.
  • the search management unit 30 includes a priority queue (not shown) for holding candidate information, and holds the prefix and key found so far in the queue together with the score. Keep it.
  • the search management unit 30 refers to the queue and selects the one with the highest score from the prefixes and keys held in the queue.
  • the character string specifying unit 32 searches for the key having the next highest character string score within the same key range as that when the key is searched. If the selected one is a prefix, the prefix specifying unit 31 selects a prefix having the next highest prefix score after the prefix within the same prefix range as that when the prefix is searched. Search for.
  • the prefix “congres” of the prefix score 45 and the key “congress” of the character string score 45 are held in the queue.
  • the search management unit 30 first pops the key “congress” and removes it from the queue.
  • the character string specifying unit 32 searches for a key having the next highest character string score after the key “congress” among keys starting with the same prefix “congres” as when the key “congress” was obtained.
  • the search management unit 30 bisects the key range searched for when the key “congress” is obtained, this time excluding “congress”, and the character string specifying unit 32 determines the two ranges.
  • the search management unit 30 When searching for a prefix, the search management unit 30 first pops the prefix “congres” and removes it from the queue. Then, the prefix specifying unit 31 searches for a prefix having a prefix score that is higher than the prefix “congres”. Specifically, the search management unit 30 bisects the range of the prefix searched when the prefix “congres” is obtained, excluding “congres”, and the prefix identification unit 31 Find the prefix with the highest prefix score in the range. At this time, prefixes having a large prefix score in the two ranges divided into two parts by excluding the prefix “congres” are prefix “aggres” of prefix score 12 and prefix “progres” of prefix score 21, respectively. It is. Therefore, the search management unit 30 newly holds these two prefixes in the queue.
  • the character string specifying unit 32 acquires the key having the maximum character string score starting with each prefix for the prefix “aggres” and the prefix “progres”. Thereby, the key “aggressive” of the character string score 12 and the key “progress” of the character string score 21 are obtained. Thereby, it can be confirmed that the prefix score of the prefix “aggres” is 12 and the prefix score of the prefix “progres” is 21 of the two prefixes acquired earlier.
  • the RMQ structure only the RMQ structure is retained, and the prefix score itself is not retained.
  • the RMQ structure alone can find which prefix has the largest prefix score, but cannot calculate its specific prefix score. Therefore, after obtaining the prefix with the largest prefix score in the range, in order to determine the specific value of the prefix score, the largest character among the keys starting with that prefix Need to get column score.
  • Prefix “progres” of prefix score 21 prefix “aggres” of prefix score 12, key “progress” of character string score 21, key “congressmen” of character string score 13, key “aggressive” of character string score 12 ".
  • the score is highest because the prefix “progress” of the prefix score 21 or the key “progress” of the character string score 21, and it is sufficient to search for either of these prefixes or keys.
  • the search management unit 30 does not register the prefix in the queue. This is because a prefix having the highest prefix score after that prefix has a smaller score. Similarly, when the score of a newly found key is lower than the kth largest character string score found so far, the search management unit 30 does not register the key in the queue. Thereby, it is possible to avoid searching for a prefix having a small prefix score or a key having a small character string score, and the top k keys can be efficiently collected.
  • the search ends when the queue is empty or the maximum score in the queue falls below the kth largest string score found so far.
  • the prefix set specifying unit 20 specifies a set of prefixes ending with the input character string from the set of prefixes, and the prefix specifying unit 31 is input.
  • the prefix with the highest prefix score is identified from the set of prefixes ending with the specified string.
  • specification part 32 specifies the key with the largest character string score from the keys which begin with the specified prefix.
  • the dictionary size can be reduced as compared with the case where the index is created for all the partial character strings.
  • the prefix specifying unit 31 specifies a prefix having a large prefix score
  • the character string specifying unit 32 searches for a key having a large character string score from the prefix. If the key is searched from the largest, the top k keys can be efficiently searched. Therefore, it is possible to perform a partial match search for character strings at high speed while reducing the amount of data.
  • the trie tree is used as a data structure capable of collecting common prefixes, so that the data size can be reduced.
  • the case where the key is represented using the data structure of the trie tree is illustrated, but the data structure may be a Patricia tree. By using the Patricia tree, the data size can be reduced more than the tree structure of the trie tree.
  • the character string search device of the present embodiment includes a search management unit 30 that manages the search range.
  • the search management unit 30 specifies a range excluding the already specified key from the keys starting with the key prefix specified by the character string specifying unit 32, and the prefix set specifying unit 20 A range excluding the prefix specified by the prefix specifying unit 31 is specified from the specified prefix set.
  • the prefix specifying unit 31 specifies the prefix having the largest prefix score from the prefix range specified by the search managing unit 30, and the character string specifying unit 32 is specified by the search managing unit 30.
  • the key with the highest string score is identified from the key range.
  • XBW used as a data structure for a dictionary can be expanded to a Top-k search, and therefore, space saving and high-speed processing can be realized when a partial match search is performed for the top k candidates.
  • Embodiment 2 a second embodiment of the character string search device according to the present invention will be described.
  • the configuration of the character string search device of this embodiment is the same as that of the first embodiment.
  • the character string search device according to the second embodiment can reduce the amount of data to be retained more than the character string device according to the first embodiment.
  • Two data structures are generated from the trie tree T described in the first embodiment.
  • One is a data structure related to a prefix, and the other is a data structure related to a key.
  • the data structure related to the prefix includes xbw which is the XBW representation of the trie tree T and the accompanying first RMQ structure. As described in the first embodiment, on xbw, the prefixes are arranged in the order sorted from the end.
  • a first RMQ structure is generated for the prefix score string R p shown in the first embodiment.
  • the search information storage unit 50 a prefix score column R p, may not be explicitly held, be held only the first RMQ structure calculated from the prefix score column R p Good.
  • the data structure relating to the key includes a Patricia tree T c generated from the trie tree T, a second RMQ structure, and a character string score string R k .
  • Tree structure of a Patricia tree T c is represented by DFUDS. Further, in the Patricia tree Tc , in order to identify only the leaf nodes of the tree structure, the same number of bit strings as the number of nodes are prepared.
  • a general Patricia tree holds a character string corresponding to each node.
  • the search information storage unit 50 of this embodiment removes the character string corresponding to each node and stores only the tree structure representing the parent-child relationship between the nodes. The reason for storing only such a tree structure will be described later.
  • each key is sorted in lexicographic order from the first character, and each key is assigned a key ID in that order.
  • Each prefix is also sorted in lexicographic order from the end, and each prefix is assigned a prefix ID in that order. Further, the range of the prefix ID is written as [s p , e p ], and the range on the set S representing the prefix is written as [s s , e s ].
  • the prefix set specifying unit 20 specifies a range [s p , e p ] of prefix IDs ending with the input character string. Specifically, the prefix set identifying unit 20 uses the XBW, range that is a character string end is entered prefix [s s, e s] to identify. However, since this range [s s , e s ] is a range on the set S, it is necessary to convert it to a prefix ID range [s p , e p ]. Therefore, the prefix set identification unit 20 identifies [s p , e p by identifying what number the first 1 and the last 1 included in [s s , e s ] are on S last. ] Is specified. This is because the elements that become 1 on S last correspond one-to-one in the same order as the prefix ID.
  • the prefix specifying unit 31 specifies the prefix having the maximum prefix score from the range [s p , e p ] of the specified prefix ID. Specifically, the prefix specifying unit 31 uses the first RMQ structure to specify the position of the prefix having the maximum prefix score within the range of [s p , e p ]. Incidentally, denoted here identified position of the prefix and i p.
  • Search management unit 30 from the position i p of the identified prefix identifies a range of keys that start with the prefix.
  • a range of keys starting with the specified prefix is denoted as [s k , e k ].
  • the retrieval management section 30 first as the corresponding position i s on S, identifies the last node having the prefix which corresponds to the position i p prefix.
  • the search management unit 30 restores a character string representing this prefix in xbw. Specifically, the retrieval management unit 30, by combining the characters when went following the parent from a node representing i s th row in XBW, restores the character string. The number of moves from node to parent is equal to the prefix length.
  • the search management unit 30 moves the target position from the parent node to the child node according to the order stored in the array d in the Patricia tree Tc . However, when the corresponding value of the array d is 1, the search management unit 30 ignores the value and performs processing for the next value.
  • Search management unit 30 uses the DFUDS, range of keys corresponding to the descendants of node u c has been reached [s k, e k] specifying the. Since the keys included in [s k , e k ] are all children of u c , it can be said that [s k , e k ] indicates a range of keys starting with the specified prefix.
  • the character string specifying unit 32 specifies the key ID (hereinafter referred to as i k ) having the maximum character string score from the specified key range [s k , e k ]. Specifically, the character string specifying unit 32 uses the second RMQ structure to specify the key position i k having the maximum prefix score within the range of [s k , e k ].
  • Character string specifying unit 32 from the position i k of the specified key ID, and identifies the string of keys. i k corresponds to the i k th leaf node u i on the Patricia tree T c . Therefore, the character string specifying unit 32 traces from u i to the parent node of the Patricia tree T c and stores the numbers of the child nodes in the array d in the reverse order of the order traced toward the parent.
  • the character string specifying unit 32 can specify the position of the node on xbw corresponding to u i by tracing xbw sequentially from the root according to this array d.
  • the character string specifying unit 32 can accurately restore the key by following a single chain.
  • the top second key is the second key with the same prefix or the first key with another prefix.
  • the data size when the data structure described in this embodiment is used will be described. If a trie tree T and a score array R k are given, the number of nodes t> the number of keys l holds. In general, the number of nodes t is about 10 times the number of keys l.
  • the data size is expressed by the following equation 2.
  • T c (Patricia tree) is generated from the trie tree T, and the tree structure is expressed in DFUDS.
  • the same number of bit strings as the number of nodes are prepared, and each bit of this bit string is used to identify whether or not it is a leaf node.
  • the Patricia tree of this embodiment is represented only by a tree structure from which character strings are removed. This is because character string information is obtained from xbw as described above.
  • a second RMQ structure (key) is generated for the key character string score array R k (score).
  • is represented by 2l + o (l) bits.
  • the calculation amount is calculated by O (k (log (k) +
  • indicates the length of the query
  • h indicates the average length of the keys registered in the dictionary.
  • FIG. 8 is a block diagram showing an outline of a character string search apparatus according to the present invention.
  • a character string search device includes a search candidate character string including an input character string from a set of search candidate character strings (for example, keys) associated with a character string score indicating a degree to be preferentially searched.
  • the prefix score defined by the largest character string score among the character string scores associated with the search candidate character string starting with the prefix is the largest.
  • the prefix specifying unit 82 specifies a prefix having a large prefix score
  • the character string specifying unit 83 searches for a search candidate character string having a large character string score from the prefix. If the search is started from the larger one, the top k search candidate character strings can be efficiently searched.
  • the character string search device may include a search management unit (for example, the search management unit 30) that manages the search range.
  • the search management unit specifies the range of the search candidate character string excluding the already specified search candidate character string from the search candidate character strings starting with the prefix of the search candidate character string specified by the character string specifying unit 83.
  • the prefix range excluding the prefix specified by the prefix specifying unit 82 from the set of prefixes specified by the prefix set specifying unit 81 may be specified.
  • the prefix specifying unit 82 specifies the prefix having the maximum prefix score from the prefix range specified by the search management unit
  • the character string specifying unit 83 is the search candidate specified by the search management unit.
  • a search candidate character string having the maximum character string score may be specified from the character string range.
  • the search management unit holds a pair of prefix and prefix score specified by the prefix specifying unit 82 and a pair of search target character string and character string score specified by the character string specifying unit 83 (for example, a priority queue may be included. Then, the search management unit identifies the prefix or the search target string with the maximum score from the prefix score or the string score from the pairs held in the queue, and the maximum score is the string score. If the maximum score is a prefix score, the search target character string of the character string score is excluded from the queue and specified as the output target, and the prefix of the prefix score may be excluded from the queue .
  • the prefix specifying unit 82 specifies a prefix with the next largest prefix score after the prefix excluded from the queue, and the string specifying unit 83 sets the maximum score. If the score is a string score, the excluded search target string in the search target string that starts with the same prefix used to identify the search target string excluded from the queue If the next highest character string score is specified and the largest score is the prefix score, the search with the highest character string score from the search target character strings starting with the prefix specified by the prefix specifying unit 82 The target character string may be specified.
  • the prefix score or character that is retained in the queue by retaining both the prefix-prefix score pair and both the search target string-string score pair pair in one queue.
  • the prefix specifying unit 82 and the character string specifying unit 83 repeat the above-described processing, whereby the search target character string having the higher character string score can be efficiently specified.
  • the character string search device generates a set of prefixes (for example, xbw) having a XBW data structure generated from a set of search candidate character strings represented by a trie data structure, and a trie tree data structure.
  • An information storage unit (for example, search information storage unit 50) may be provided.
  • the prefix identification unit 82 identifies the position of the prefix having the largest prefix score from the set of prefixes having the XBW data structure, and the search management unit identifies the Patricia from the position of the identified prefix.
  • the position of the corresponding node in the tree eg, u c ) may be specified. With such a configuration, it is possible to reduce the amount of data to be stored used for search.
  • the character string specifying unit 83 determines the position (for example, u i ) of the search candidate character string having the maximum character string score from the search candidate character strings existing under the position of the node specified by the search management unit.
  • the search candidate character string corresponding to the specified position may be specified from the set of prefixes having the XBW data structure.
  • the prefix specifying unit 82 uses the first RMQ structure based on the relationship between the prefix represented by the first RMQ structure and the prefix score, and sets the prefixes specified.
  • the prefix of the largest prefix score may be specified by performing a range search.
  • the character string specifying unit 83 uses the second RMQ structure based on the relationship between the search candidate character string represented by the second RMQ structure and the character string score, and identifies the prefix specified.
  • the search candidate character string having the maximum character string score may be specified by performing a range search on the search candidate character string starting with “.”
  • the present invention is preferably applied to a character string search device that searches for a key that partially matches an input character string.
  • the character string search device according to the present invention can be used, for example, when providing a search service.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

A prefix set specification unit (81) specifies, from a set of prefixes which are text character strings of one or more continuous text characters which are extracted from lead text characters of each search candidate text character string, a set of prefixes which end with an inputted text character string. A prefix specification unit (82) specifies the prefix with the highest prefix score from the set of prefixes which end with the inputted text character string, a prefix score being defined for each prefix by the highest text character string score among text character string scores associated with search candidate text character strings beginning with the said prefix. A text character string specification unit (83) specifies, from among the search candidate text character strings which begin with the specified prefix, the search candidate text character string with the highest text character string score.

Description

文字列検索装置、文字列検索方法および文字列検索プログラムCharacter string search device, character string search method, and character string search program
 本発明は、入力された文字列に部分一致するキーを検索する文字列検索装置、文字列検索方法および文字列検索プログラムに関する。 The present invention relates to a character string search device, a character string search method, and a character string search program for searching for a key that partially matches an input character string.
 人間のテキスト入力を支援する方法が普及し、我々の生活に欠かせないものになっている。入力支援として、例えば、検索エンジンの入力フォームに検索候補の検索キーワードを表示したり、WebブラウザのURL(Uniform Resource Locator)入力フォームに候補とするURLを表示したりすることが挙げられる。他にも、IME(Input Method Editor )の予測変換時に変換候補を表示したり、スペルチェッカーにおいて正しいスペルの候補を表示したりすることも入力支援の一例である。 The method of supporting human text input has become widespread and has become indispensable for our lives. As input support, for example, a search keyword of a search candidate is displayed on an input form of a search engine, or a URL as a candidate is displayed on a URL (Uniform ResourceatorLocator) input form of a Web browser. In addition, displaying conversion candidates at the time of prediction conversion of IME (Input Method Editor) or displaying correct spelling candidates in the spell checker are examples of input support.
 このような入力支援は、辞書の検索として実現される。ユーザが入力しそうな文字列を、あらかじめ辞書にキーとして登録しておく。ユーザが新たに文字列の入力を開始したとき、ユーザが入力した文字列を検索クエリとして辞書を検索し、入力候補として適切なキーを取得して、画面上に表示する。例えば、検索キーワードの推薦では、過去にユーザが入力した検索キーワードをあらかじめ辞書に登録しておき、入力の候補として用いる。 Such input support is realized as a dictionary search. A character string that the user is likely to input is registered in advance in the dictionary as a key. When the user newly starts inputting a character string, the dictionary is searched using the character string input by the user as a search query, an appropriate key is acquired as an input candidate, and displayed on the screen. For example, in search keyword recommendation, search keywords previously input by the user are registered in the dictionary in advance and used as input candidates.
 実際の場面では、候補に該当する全てのキーを列挙する必要はない。例えば、検索キーワードを推薦する場面では、入力頻度が高い上位k個を候補として推薦すればよい。このように、スコアが大きい上位k個のキーを検索する問題は、Top-k検索(Top-k辞書検索)と呼ばれる。 In actual situations, it is not necessary to enumerate all keys that are candidates. For example, in a scene where a search keyword is recommended, the top k items with the highest input frequency may be recommended as candidates. The problem of searching for the top k keys having a large score in this way is called a Top-k search (Top-k dictionary search).
 非特許文献1では、RMQ Trieという、トライ(trie)とRMQ(Ranged Minimum Query)構造を利用することで、前方一致するキーの中から上位のキーを高速に取得するデータ構造が記載されている。 Non-Patent Document 1 describes a data structure that obtains a high-order key at a high speed from keys that coincide with each other by using a RMQ Trie and a RMQ (Ranged Minimum Query) structure. .
 図9は、RMQ Trieを示す説明図である。図9に示す例では、検索クエリPを接頭辞として持つノードvを見つけ、ノードvの配下のキーの範囲[a,b]を得る。[a,b]に含まれるキーは、全て検索クエリPを接頭辞として持つ。このとき、各キーに対応づけて並べられたスコアの配列Rのうち範囲[a,b]の範囲のスコアを検索することで、検索クエリPを接頭辞として持つスコア上位k個のキーを得る。 FIG. 9 is an explanatory diagram showing the RMQ Trie. In the example shown in FIG. 9, the node v having the search query P as a prefix is found, and the key range [a, b] under the node v is obtained. All keys included in [a, b] have the search query P as a prefix. At this time, by searching the score in the range [a, b] from the score array R arranged in association with each key, the top k keys having the search query P as a prefix are obtained. .
 非特許文献1には、RMQ Trieと同様に、前方一致するキーの中から上位のキーを高速に取得するために用いられるデータ構造が、他に2種類記載されている。 Non-Patent Document 1 describes two other types of data structures that are used to acquire a high-order key at a high speed from keys that coincide with each other in the same manner as RMQ Trie.
 また、非特許文献2には、文書検索におけるTop-k検索について記載されている。この手法は、文書検索用のデータ構造を基礎として、そのデータ構造にTop-k検索に必要な付加データを追加することで、Top-k検索を実現する。 Non-patent document 2 describes Top-k search in document search. This method realizes Top-k search by adding additional data necessary for Top-k search to the data structure based on the data structure for document search.
 辞書に含まれるキーが大規模化すると、入力文字列に該当するキーの数も多くなるため、検索に時間がかかってしまう。そのため、候補とするキーを高速に取得することが求められる。 If the keys included in the dictionary become large, the number of keys corresponding to the input character string will increase, and search will take time. Therefore, it is required to obtain a candidate key at high speed.
 一方、非特許文献1に記載された各データ構造を利用することで、前方一致するキーの候補を高速に取得することは可能であるが、部分一致するキーの候補を取得することは困難である。 On the other hand, by using each data structure described in Non-Patent Document 1, it is possible to acquire a key candidate that matches forward at high speed, but it is difficult to acquire a key candidate that partially matches. is there.
 また、非特許文献2に記載されたデータ構造を用いることで、文書検索におけるTop-k検索を実現することができる。しかし、文書検索の際に用いられるデータはサイズが大きいため、文書検索に用いられる検索方法をそのまま辞書向けに適用した場合、対象とするデータサイズが大きくなってしまうという問題がある。 Further, by using the data structure described in Non-Patent Document 2, Top-k search in document search can be realized. However, since the data used for document search is large in size, there is a problem that when the search method used for document search is directly applied to a dictionary, the target data size becomes large.
 そこで、本発明は、データ量を削減しつつ高速に文字列の部分一致検索を行うことができる文字列検索装置、文字列検索方法および文字列検索プログラムを提供することを目的とする。 Therefore, an object of the present invention is to provide a character string search device, a character string search method, and a character string search program that can perform partial match search of a character string at high speed while reducing the amount of data.
 本発明による文字列検索装置は、優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列の集合から、入力された文字列を含む検索候補文字列を検索する文字列検索装置であって、各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定する接頭辞集合特定部と、入力された文字列で終わる接頭辞の集合の中から、接頭辞ごとにその接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコアが最大の接頭辞を特定する接頭辞特定部と、特定された接頭辞で始まる検索候補文字列の中から、文字列スコアが最大の検索候補文字列を特定する文字列特定部とを備えたことを特徴とする。 A character string search device according to the present invention is a character string that searches for a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched. A prefix for identifying a set of prefixes ending with an input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string, in the search device From the set identification part and the set of prefixes ending with the input character string, each prefix is defined with the largest character string score among the character string scores associated with the search candidate character string beginning with that prefix. Character string specifying the search candidate character string with the highest character string score from the prefix specification part that specifies the prefix with the highest prefix score and the search candidate character string starting with the specified prefix With department And wherein the door.
 本発明による文字列検索方法は、優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列の集合から、入力された文字列を含む検索候補文字列を検索する文字列検索方法であって、各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定する接頭辞集合特定ステップと、入力された文字列で終わる接頭辞の集合の中から、接頭辞ごとにその接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコアが最大の接頭辞を特定する接頭辞特定ステップと、特定された接頭辞で始まる検索候補文字列の中から、文字列スコアが最大の検索候補文字列を特定する文字列特定ステップとを含むことを特徴とする。 The character string search method according to the present invention is a character string that searches for a search candidate character string that includes an input character string from a set of search candidate character strings that are associated with a character string score that indicates the degree to be searched with priority A prefix for identifying a set of prefixes ending with an input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string in the search method From the set identification step and a set of prefixes ending with the input string, each prefix is defined with the largest string score among the string scores associated with the search candidate string starting with that prefix. A prefix identification step that identifies the prefix with the largest prefix score, and a string identification that identifies the search candidate string with the largest string score from among the search candidate strings that begin with the identified prefix Characterized in that it comprises a step.
 本発明による文字列検索プログラムは、優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列の集合から、入力された文字列を含む検索候補文字列を検索するコンピュータに適用される文字列検索プログラムであって、コンピュータに、各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定する接頭辞集合特定処理、入力された文字列で終わる接頭辞の集合の中から、接頭辞ごとにその接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコアが最大の接頭辞を特定する接頭辞特定処理、および、特定された接頭辞で始まる検索候補文字列の中から、文字列スコアが最大の検索候補文字列を特定する文字列特定処理を実行させることを特徴とする。 A character string search program according to the present invention provides a computer for searching for a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched. An applied character string search program, wherein a prefix ending with an input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string Prefix set identification processing that identifies a set of words, out of a set of prefixes that end with the input character string, out of the character string score associated with the search candidate character string that starts with that prefix for each prefix A prefix identification process that identifies the prefix with the largest prefix score defined by the largest string score, and a search string that begins with the identified prefix, and a string Core characterized in that to execute a string specifying process for specifying the maximum search candidate character strings.
 本発明によれば、データ量を削減しつつ高速に文字列の部分一致検索を行うことができる。 According to the present invention, a partial match search of a character string can be performed at high speed while reducing the amount of data.
本発明による文字列検索装置の第1の実施形態の構成例を示すブロック図である。It is a block diagram which shows the structural example of 1st Embodiment of the character string search apparatus by this invention. キーに対応するトライ木の例を示す説明図である。It is explanatory drawing which shows the example of the trie tree corresponding to a key. 第1のXBWの例を示す説明図である。It is explanatory drawing which shows the example of 1st XBW. 第2のXBWの例を示す説明図である。It is explanatory drawing which shows the example of 2nd XBW. 検索情報記憶部が記憶するデータ構造の例を示す説明図である。It is explanatory drawing which shows the example of the data structure which a search information storage part memorize | stores. 第1の実施形態の文字列検索装置の動作例を示すフローチャートである。It is a flowchart which shows the operation example of the character string search apparatus of 1st Embodiment. 文字列スコアの大きいキーを選択する処理の例を示す説明図である。It is explanatory drawing which shows the example of the process which selects a key with a big character string score. 本発明による文字列検索装置の概要を示すブロック図である。It is a block diagram which shows the outline | summary of the character string search apparatus by this invention. RMQ Trieを示す説明図である。It is explanatory drawing which shows RMQ Trie.
 まず初めに、本発明の文字列検索装置の概略を説明する。本発明は、辞書向けデータ構造であるXBWをTop-k検索に拡張することにより、入力された文字列に部分一致する上位のキーを検索するデータ構造を、省スペースかつ高速に実現するものである。 First, an outline of the character string search device of the present invention will be described. The present invention realizes a data structure for searching for a high-order key that partially matches an inputted character string at high speed by extending XBW, which is a dictionary data structure, to Top-k search. is there.
 本発明では、検索候補文字列であるキーごとに、優先的に検索すべき度合いを示すスコア(以下、文字列スコアと記す。)が割り当てられ、キーの集合は、トライ木の構造で表わされる。 In the present invention, each key that is a search candidate character string is assigned a score (hereinafter referred to as a character string score) indicating the degree of search priority, and a set of keys is represented by a trie structure. .
 また、キーの集合に含まれるキーの全ての接頭辞は、辞書検索に用いられるXBW構造で表される。本発明の文字列検索装置は、このXBW構造を利用して、入力された文字列で終わる接頭辞の範囲を特定する。また、各接頭辞には、その接頭辞で始まるキーの中で最大のスコア(以下、接頭辞スコアと記す。)が対応付けられる。そこで、文字列検索装置は、特定した接頭辞の範囲の中で、最大の接頭辞スコアの接頭辞を特定する。 In addition, all the prefixes of the keys included in the key set are represented by the XBW structure used for dictionary search. The character string search device of the present invention uses this XBW structure to specify a prefix range ending with the input character string. Each prefix is associated with a maximum score (hereinafter referred to as a prefix score) among keys starting with the prefix. Therefore, the character string search device identifies the prefix with the largest prefix score within the identified prefix range.
 本発明では、特定された接頭辞の範囲内で最大の接頭辞スコアを特定するために、RMQ構造が用いられる。以下、最大の接頭辞スコアを特定するため、接頭辞と接頭辞スコアとの関係を表わすために用いられるRMQ構造を、第一のRMQ構造と記す。文字列検索装置は、この第一のRMQ構造を用いて、特定された接頭辞の範囲内で最大の接頭辞スコアの接頭辞を特定する。 In the present invention, the RMQ structure is used to specify the maximum prefix score within the specified prefix range. Hereinafter, in order to specify the maximum prefix score, the RMQ structure used to represent the relationship between the prefix and the prefix score is referred to as a first RMQ structure. The character string search device uses the first RMQ structure to specify the prefix with the largest prefix score within the specified prefix range.
 さらに、文字列検索装置は、特定された接頭辞で始まるキーの中で最大の文字列スコアのキーを特定する。このとき、特定された接頭辞は、トライ木における一つのノードに対応する。そこで、各ノード配下に存在するキーの範囲の中で最大の文字列スコアを特定するため、最大の接頭辞スコアと特定する場合と同様、RMQ構造が用いられる。以下、キーと文字列スコアとの関係を表わすために用いられるRMQ構造を、第二のRMQ構造と記す。文字列検索装置は、この第二のRMQ構造を用いて、特定された接頭辞で始まるキーの範囲の中から最大の文字列スコアのキーを特定する。 Furthermore, the character string search device identifies the key with the largest character string score among the keys that start with the identified prefix. At this time, the identified prefix corresponds to one node in the trie tree. Therefore, in order to specify the maximum character string score in the range of keys existing under each node, the RMQ structure is used as in the case of specifying the maximum prefix score. Hereinafter, the RMQ structure used to represent the relationship between the key and the character string score is referred to as a second RMQ structure. The character string search device uses the second RMQ structure to specify the key with the maximum character string score from the range of keys starting with the specified prefix.
 最大の文字列スコアのキーを特定した後、文字列検索装置は、この文字列検索をTop-k検索に適用するため、文字列スコアが2番目以降のキーを検索する処理を行う。2番目以降のキーが存在する位置は、既に特定された接頭辞で始まるキーの2番目以降か、まだ特定されていない接頭辞の1番目以降である。 After specifying the key with the maximum character string score, the character string search device performs a process of searching for a key having a character string score of the second or later in order to apply this character string search to the Top-k search. The position where the second and subsequent keys exist is the second and subsequent keys starting with the specified prefix, or the first and subsequent prefixes that have not been specified yet.
 そこで、文字列検索装置は、特定された接頭辞の接頭辞スコアと特定されたキーの文字列スコアを保持しておく。文字列検索装置は、保持する文字列スコアと接頭辞スコアのうち、最大のスコアとなるキーまたは接頭辞を選択する。それがキーであれば、そのキーと同じ接頭辞で始まるキーの中で、次に文字列スコアが大きいキーを検索する。また、それが接頭辞であれば、その接頭辞の次に接頭辞スコアが大きい接頭辞を検索する。これを繰り返すことで、入力された文字列を含むキーのうち、文字列スコアが上位となるものを効率的に検索できる。 Therefore, the character string search device holds the prefix score of the specified prefix and the character string score of the specified key. The character string search device selects a key or prefix that is the maximum score among the character string scores and prefix scores it holds. If it is a key, the key with the next highest string score is searched for among the keys that start with the same prefix. If it is a prefix, a prefix having the next highest prefix score is searched for. By repeating this, it is possible to efficiently search for keys having a higher character string score among keys including the input character string.
 以下、本発明の文字列検索装置の実施形態を、図面を参照してより具体的に説明する。 Hereinafter, embodiments of the character string search device of the present invention will be described more specifically with reference to the drawings.
実施形態1. 
 図1は、本発明による文字列検索装置の第1の実施形態の構成例を示すブロック図である。本実施形態の文字列検索装置は、入力部10と、接頭辞集合特定部20と、検索管理部30と、接頭辞特定部31と、文字列特定部32と、出力部40と、検索情報記憶部50とを備えている。
Embodiment 1. FIG.
FIG. 1 is a block diagram showing a configuration example of a first embodiment of a character string search device according to the present invention. The character string search device according to the present embodiment includes an input unit 10, a prefix set specifying unit 20, a search management unit 30, a prefix specifying unit 31, a character string specifying unit 32, an output unit 40, and search information. And a storage unit 50.
 入力部10は、一文字以上の文字列を入力する。本実施形態の文字列検索装置は、入力された文字列に部分一致するキーを検索する。以下の説明では、入力される文字列のことを検索クエリ(または、単にクエリ)Pと記す。 The input unit 10 inputs a character string of one character or more. The character string search device according to the present embodiment searches for a key that partially matches an input character string. In the following description, an input character string is referred to as a search query (or simply a query) P.
 検索情報記憶部50は、検索候補文字列であるキーの集合を記憶する。本実施形態で用いられるキーには、上述するように文字列スコアが対応づけられる。すなわち、本実施形態の文字列検索装置は、キーの集合から、より文字列スコアが大きいキーを優先的に検索する。 The search information storage unit 50 stores a set of keys that are search candidate character strings. As described above, the character string score is associated with the key used in this embodiment. That is, the character string search device of the present embodiment preferentially searches for a key having a higher character string score from the set of keys.
 本実施形態では、データ量を削減するため、検索対象とするキーは、トライ木の構造を利用して表わされる。図2は、キーに対応するトライ木の例を示す説明図である。例えば、図2に例示する4つの単語(aba,abcc,cab,cac)が存在する場合、トライ木は、共通する文字を共通のノードに配置するように構成される。検索情報記憶部50は、トライ木で表わされたキーそのものを記憶してもよく、後述するように、トライ木の構造のみを記憶してもよい。 In this embodiment, in order to reduce the amount of data, the search target key is represented using the structure of the trie tree. FIG. 2 is an explanatory diagram illustrating an example of a trie tree corresponding to a key. For example, when there are four words (aba, abcc, cab, cac) illustrated in FIG. 2, the trie tree is configured to place a common character at a common node. The search information storage unit 50 may store the key itself represented by the trie tree, or may store only the structure of the trie tree, as will be described later.
 また、木構造で表わされた各葉ノードが、各キーに対応する。そこで、検索情報記憶部50は、図2に例示する各キーのスコア(文字列スコア)を各葉ノードに対応付けて記憶する。これにより、トライ木を探索して葉ノードに到達したとき、その葉ノードが表すキーに対応する文字列スコアを取得できる。 In addition, each leaf node represented by a tree structure corresponds to each key. Accordingly, the search information storage unit 50 stores the score (character string score) of each key illustrated in FIG. 2 in association with each leaf node. Thereby, when the trie tree is searched and the leaf node is reached, the character string score corresponding to the key represented by the leaf node can be acquired.
 さらに、検索情報記憶部50は、クエリPで終わる文字列を検索できるように、接頭辞pの集合を記憶する。ここで、接頭辞pとは、各キーの先頭文字から抽出される連続する1文字以上の文字列である。この接頭辞pの集合は、末尾から辞書式順序にソートされていてもよい。 Further, the search information storage unit 50 stores a set of prefixes p so that a character string ending with the query P can be searched. Here, the prefix p is a character string of one or more consecutive characters extracted from the first character of each key. This set of prefixes p may be sorted lexicographically from the end.
 本実施形態では、このような接頭辞の集合を表わすため、XBWという構造を利用する。XBWは、ラベルつきの木構造を効率的に表現することができるデータ構造である。トライ木をこのXBW構造を用いて表現することで、クエリPで終わる接頭辞pの範囲検索が可能になる。 In this embodiment, a structure called XBW is used to represent such a set of prefixes. XBW is a data structure that can efficiently represent a labeled tree structure. By expressing the trie tree using this XBW structure, a range search of the prefix p ending with the query P becomes possible.
 XBWは、同等の操作を実現するデータ構造を2種類の方法で実現できることが知られている。第1のXBWは、辞書の各接頭辞について、その接頭辞に対応するトライ木上のノードにおいて子ノードを表す文字を対応付ける構造を有する。第2のXBWは、辞書の各接頭辞について、その接頭辞に対応するトライ木上のノードにおいて親ノードとなる接頭辞のIDを対応付ける構造を有する。以下、それぞれのXBWの内容を説明する。 XBW is known to be able to realize a data structure that realizes an equivalent operation by two types of methods. The first XBW has a structure in which characters representing child nodes are associated with each prefix of the dictionary at a node on the trie tree corresponding to the prefix. The second XBW has a structure in which each prefix of the dictionary is associated with an ID of a prefix that becomes a parent node in a node on the trie tree corresponding to the prefix. Hereinafter, the contents of each XBW will be described.
 図3は、第1のXBWの例を示す説明図である。図3に例示する第1のXBWでは、トライ木の各ノードに対応する接頭辞が末尾から辞書式順序で並べられ、それぞれの接頭辞について、子ノードを表す文字が対応づけられる。このような構造により、各接頭辞から特定の文字を表す子ノードに移動することが可能になり、トライ木と同等の動作を実現できる。また、クエリPで終わる接頭辞pを範囲検索することができる。 FIG. 3 is an explanatory diagram showing an example of the first XBW. In the first XBW illustrated in FIG. 3, prefixes corresponding to the nodes of the trie tree are arranged in lexicographic order from the end, and characters representing child nodes are associated with the respective prefixes. With such a structure, it is possible to move from each prefix to a child node representing a specific character, and an operation equivalent to a trie tree can be realized. Further, it is possible to perform a range search for the prefix p ending with the query P.
 図4は、第2のXBWの例を示す説明図である。図4に例示する第2のXBWでは、トライ木の各ノードに対応する接頭辞が末尾から辞書式順序で並べられ、各接頭辞にIDが付与される。そして、それぞれの接頭辞について、その親のIDが対応づけられる。このような構造により、次の親ノードに移動することが可能になる。また、第1のXBWと同様に、クエリPで終わる接頭辞pを範囲検索することができる。 FIG. 4 is an explanatory diagram showing an example of the second XBW. In the second XBW illustrated in FIG. 4, the prefixes corresponding to the nodes of the trie tree are arranged in lexicographic order from the end, and an ID is assigned to each prefix. Each prefix is associated with its parent ID. Such a structure makes it possible to move to the next parent node. Similarly to the first XBW, a range search can be performed for the prefix p ending with the query P.
 なお、第2のXBWでは、親のIDしか持たないため、子ノードを探索することは困難である。ただし、第2のXBWを利用する場合でも、クエリPで終わる接頭辞pの範囲検索は可能である。本実施形態では、いずれのXBWも利用可能である。 In the second XBW, since it has only the parent ID, it is difficult to search for a child node. However, even when the second XBW is used, a range search of the prefix p ending with the query P is possible. In this embodiment, any XBW can be used.
 第1のXBWについては参照文献1に、第2のXBWについては参照文献2に記載されている。
 <参照文献1>Paolo Ferragina, Fabrizio Luccio, Giovanni Manzini and S. Muthukrishnan, "Structuring labeled trees for optimal succinctness, and beyond", FOCS '05 Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science, Pages 184-196
 <参照文献2>Wing-Kai Hon, Tsung-Han Ku, Rahul Shah, Sharma V. Thankachan, and Jerey Scott Vitter, “Faster compressed dictionary matching”, SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval, Pages 191-200
The first XBW is described in Reference Document 1, and the second XBW is described in Reference Document 2.
<Reference 1> Paolo Ferragina, Fabrizio Luccio, Giovanni Manzini and S. Muthukrishnan, "Structuring labeled trees for optimal succinctness, and beyond", FOCS '05 Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science, Pages 184-196
<Reference 2> Wing-Kai Hon, Tsung-Han Ku, Rahul Shah, Sharma V. Thankachan, and Jerey Scott Vitter, “Faster compressed dictionary matching”, SPIRE'10 Proceedings of the 17th international conference on String processing and information retrieval , Pages 191-200
 また、本実施形態では、接頭辞ごとにスコア(すなわち、接頭辞スコア)を定義する。接頭辞スコアは、その接頭辞で始まるキーに対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される。このスコアを式で表すと式1のようになる。式1における右辺のScoreは、文字列スコアを表わし、式1における左辺のScoreは、接頭辞スコアを表わす。 Also, in this embodiment, a score (that is, a prefix score) is defined for each prefix. The prefix score is defined by the largest character string score among character string scores associated with keys starting with the prefix. When this score is expressed by an expression, it becomes as shown in Expression 1. The score on the right side in Expression 1 represents a character string score, and the score on the left side in Expression 1 represents a prefix score.
 Score(p)=max{Score(接頭辞pで始まるキー)} ・・・(式1) Score (p) = max {Score (key starting with prefix p)} (Formula 1)
 本実施形態では、キーの集合が木構造で表わされているため、ある接頭辞で始まるキーは、その接頭辞に対応するノードの配下に存在する。したがって、接頭辞スコアは、そのノード配下に存在するキーの中で最も大きい文字列スコアになる。 In this embodiment, since a set of keys is represented by a tree structure, a key starting with a certain prefix exists under a node corresponding to the prefix. Therefore, the prefix score is the largest character string score among the keys existing under the node.
 本実施形態では、XBW構造に第一のRMQ構造を追加し、その第一のRMQ構造を利用して各ノードに対応する接頭辞スコアを特定できるようにする。具体的には、RMQで用いられる配列に、各接頭辞の接頭辞スコアを格納する。以下、接頭辞スコアが格納される配列を、接頭辞スコア列Rと記す。接頭辞は末尾を基準にソートされているため、同じ文字列で終わる接頭辞は、ひとつながりの範囲で特定される。したがって、第一のRMQ構造を用いることで、接頭辞スコア列Rの任意の範囲における最大値を特定できる。 In the present embodiment, a first RMQ structure is added to the XBW structure, and a prefix score corresponding to each node can be specified using the first RMQ structure. Specifically, the prefix score of each prefix is stored in an array used in RMQ. Hereinafter, the array in which the prefix score is stored is referred to as a prefix score column R p . Since the prefixes are sorted based on the end, prefixes that end with the same string are specified in a single range. Therefore, by using the first RMQ structure, the maximum value in any range of the prefix score string R p can be specified.
 さらに、本実施形態では、第二のRMQ構造を利用して各キーの文字列スコアを特定できるようにする。具体的には、RMQで用いられる配列に、各キーの文字列スコアを格納する。以下、文字列スコアが格納される配列を、文字列スコア列Rと記す。各キーは、先頭からソートされているため、ある接頭辞で始まるキーは、ひとつながりの範囲で特定される。したがって、第二のRMQ構造を用いることで、文字列スコア列Rの任意の範囲における最大値を特定できる。 Furthermore, in this embodiment, the character string score of each key can be specified using the second RMQ structure. Specifically, the character string score of each key is stored in an array used in RMQ. Hereinafter, an array in which character string scores are stored is referred to as a character string score string R k . Since each key is sorted from the top, keys starting with a certain prefix are specified in a range of connections. Therefore, the maximum value in an arbitrary range of the character string score string R k can be specified by using the second RMQ structure.
 図5は、検索情報記憶部が記憶するデータ構造の例を示す説明図である。本実施形態におけるXBW構造は、トライ木のノードごとに3つの要素の組を有する集合Sで表わされる。Slastは、バイナリのフラグであり、ノードの親ノードにとってそのノードが最後の子供である場合に1になり、それ以外は0になる。Sαは、ノードが表わす文字である。Sπは、ノードの親ノードに対応する接頭辞であり、ルートから親ノードまでの文字を順番に結合した文字列である。なお、Sπには、ノード自身の文字は含まれない。3つの要素の組は、各要素に含まれる接頭辞の末尾の文字から先頭の文字までを比較して辞書式順序でソートされる。図5に示す例では、ソートされた各組(Sπ,Sα,Slast)に、先頭から順に行番号が付与される。なお、図5において、$はキーの先頭を示し、#はキーの末尾を示す。 FIG. 5 is an explanatory diagram illustrating an example of a data structure stored in the search information storage unit. The XBW structure in the present embodiment is represented by a set S having a set of three elements for each node of the trie tree. S last is a binary flag, and is 1 when the node is the last child of the node's parent node, and 0 otherwise. S α is a character represented by the node. S π is a prefix corresponding to the parent node of the node, and is a character string obtained by sequentially combining characters from the root to the parent node. Note that does not include the characters of the node itself. The set of three elements is sorted in lexicographic order by comparing from the last character of the prefix included in each element to the first character. In the example shown in FIG. 5, row numbers are assigned to the sorted groups (S π , S α , S last ) in order from the top. In FIG. 5, $ indicates the beginning of the key, and # indicates the end of the key.
 また、図5に示すように、接頭辞ごとに接頭辞スコアRが定義される。接頭辞スコアRは、上述するように、各キーに対応づけられた文字列スコアから算出されるため、明示的に接頭辞スコアは保持されなくてもよい。図5に例示する接頭辞IDは、辞書に含まれる全ての接頭辞を、末尾からソートした順に付与される。したがって、接頭辞IDの順番は、Slastに1が設定されている接頭辞の順番と一致する。 Also, as shown in FIG. 5, a prefix score Rp is defined for each prefix. Since the prefix score Rp is calculated from the character string score associated with each key as described above, the prefix score does not need to be explicitly retained. The prefix IDs illustrated in FIG. 5 are assigned in the order in which all prefixes included in the dictionary are sorted from the end. Therefore, the order of prefix IDs matches the order of prefixes for which 1 is set in S last .
 図5に例示する構造によって、クエリPで終わる接頭辞の範囲を特定できる。例えば、クエリ“ab”で終わる接頭辞に対応する行は、行番号7~9に対応する行(すなわち、“$ab”,“$cab”に相当する行)であることが分かる。また、“$ab”および“$cab”の接頭辞スコアRは、それぞれ接頭辞ID=4,5に対応する9および4であることも分かる。 With the structure illustrated in FIG. 5, a range of prefixes ending with the query P can be specified. For example, it can be seen that the line corresponding to the prefix ending with the query “ab” corresponds to the line numbers 7 to 9 (that is, lines corresponding to “$ ab” and “$ cab”). It can also be seen that the prefix scores R p of “$ ab” and “$ cab” are 9 and 4 corresponding to the prefix IDs = 4 and 5, respectively.
 接頭辞の範囲を特定できれば、その範囲でスコアが最大となる接頭辞のIDを第一のRMQ構造で取得することができる。さらに、第一のRMQ構造を再帰的に利用することで、スコアの大きさが2位以下の接頭辞IDを取得することができる。 If the prefix range can be specified, the ID of the prefix having the maximum score in the range can be acquired with the first RMQ structure. Furthermore, by using the first RMQ structure recursively, it is possible to acquire a prefix ID having a score of second or lower.
 以上のことから、XBW構造を用いることで、クエリPで終わる接頭辞pの中から接頭辞スコアが上位となる接頭辞pを任意の数だけ選択することが可能になる。 From the above, by using the XBW structure, it is possible to select any number of prefixes p having a higher prefix score from the prefixes p ending with the query P.
 接頭辞集合特定部20は、検索情報記憶部50に記憶された接頭辞の集合から、入力された文字列を含む接頭辞の集合を特定する。具体的には、接頭辞集合特定部20は、入力された文字列で終わる接頭辞の集合を特定する。例えば、検索情報記憶部50が図5に例示する接頭辞の集合を記憶している場合、文字列として“ab”が入力されると、接頭辞集合特定部20は、行番号7~9の範囲に存在する接頭辞(すなわち、“$ab”,“$cab”)を接頭辞の集合として特定する。 The prefix set specifying unit 20 specifies a set of prefixes including the input character string from the set of prefixes stored in the search information storage unit 50. Specifically, the prefix set specifying unit 20 specifies a set of prefixes ending with the input character string. For example, when the search information storage unit 50 stores a set of prefixes illustrated in FIG. 5, when “ab” is input as a character string, the prefix set specifying unit 20 sets the line numbers 7 to 9. A prefix existing in the range (ie, “$ ab”, “$ cab”) is specified as a set of prefixes.
 接頭辞特定部31は、接頭辞集合特定部20によって特定された接頭辞の集合の中から、接頭辞スコアが上位の接頭辞を特定する。接頭辞特定部31は、接頭辞スコアが最も大きい接頭辞または接頭辞スコアの上位n件に該当する接頭辞を特定してもよい(nは任意の自然数)。 The prefix specifying unit 31 specifies a prefix having a higher prefix score from the set of prefixes specified by the prefix set specifying unit 20. The prefix identification unit 31 may identify a prefix having the largest prefix score or a prefix corresponding to the top n prefix scores (n is an arbitrary natural number).
 文字列特定部32は、特定された接頭辞で始まるキーのうち、文字列スコアが上位のキーを特定する。文字列特定部32は、文字列スコアが最も大きいキーまたは文字列スコアの上位m件に該当するキーを検索してもよい(mは任意の自然数)。 The character string specifying unit 32 specifies a key having a higher character string score among keys starting with the specified prefix. The character string specifying unit 32 may search for a key having the largest character string score or a key corresponding to the top m characters of the character string score (m is an arbitrary natural number).
 例えば、図5において、接頭辞特定部31が接頭辞を“$ab”と特定したとする。この場合、特定された接頭辞“$ab”で始まるキーは、“aba”と“abcc”である。“aba”の文字列スコアは3であり、“abcc”の文字列スコアは9である。この場合、文字列特定部32は、キーとして“abcc”を選択すればよい。 For example, in FIG. 5, it is assumed that the prefix identifying unit 31 identifies the prefix as “$ ab”. In this case, the keys starting with the identified prefix “$ ab” are “aba” and “abcc”. The character string score of “abba” is 3, and the character string score of “abcc” is 9. In this case, the character string specifying unit 32 may select “abcc” as a key.
 検索管理部30は、接頭辞特定部31が検索する接頭辞の範囲を特定する。また、検索管理部30は、文字列特定部32が検索するキーの範囲を特定し、文字列特定部32によって特定されたキーを、検索対象のキーと特定する。 The search management unit 30 specifies a prefix range to be searched by the prefix specifying unit 31. In addition, the search management unit 30 specifies a key range to be searched by the character string specifying unit 32, and specifies the key specified by the character string specifying unit 32 as a search target key.
 具体的には、検索管理部30は、まず、接頭辞集合特定部20が特定した接頭辞の範囲を、接頭辞特定部31が検索する接頭辞の範囲と特定する。そして、検索管理部30は、その特定された範囲内の接頭辞で始まるキーを文字列特定部32が検索するキーの範囲と特定する。そして、検索管理部30は、文字列特定部32によって特定されたキーを、検索対象のキーと特定する。 Specifically, the search management unit 30 first specifies the prefix range specified by the prefix set specifying unit 20 as the prefix range searched by the prefix specifying unit 31. Then, the search management unit 30 specifies a key starting with a prefix within the specified range as a key range to be searched by the character string specifying unit 32. Then, the search management unit 30 specifies the key specified by the character string specifying unit 32 as the search target key.
 その後、検索管理部30は、文字列特定部32によって特定されたキーの接頭辞で始まるキーの中から、すでに特定されたキーを除いた範囲を特定する。さらに、検索管理部30は、接頭辞集合特定部20により特定された接頭辞の集合から、接頭辞特定部31により特定された接頭辞を除いた範囲を特定する。 After that, the search management unit 30 specifies a range excluding the already specified key from the keys starting with the key prefix specified by the character string specifying unit 32. Further, the search management unit 30 specifies a range obtained by removing the prefix specified by the prefix specifying unit 31 from the set of prefixes specified by the prefix set specifying unit 20.
 そして、検索管理部30は、接頭辞特定部31および文字列特定部32に、各処理を実行させる。具体的には、接頭辞特定部31は、検索管理部30により特定された接頭辞の範囲から、接頭辞スコアが最大の接頭辞を特定する。さらに、文字列特定部32は、検索管理部30により特定されたキーの範囲から、文字列スコアが最大のキーを特定する。 Then, the search management unit 30 causes the prefix specifying unit 31 and the character string specifying unit 32 to execute each process. Specifically, the prefix specifying unit 31 specifies the prefix having the maximum prefix score from the prefix range specified by the search management unit 30. Furthermore, the character string specifying unit 32 specifies the key having the maximum character string score from the key range specified by the search management unit 30.
 検索管理部30は、接頭辞の範囲から特定された接頭辞の接頭辞スコアと、キーの範囲の中から特定されたキーの文字列スコアとを比較する。比較した結果、最も大きいスコアが文字列スコアだった場合、そのキーと同じ接頭辞で始まるキーの中で、そのキーの次に文字列スコアが大きいキーを検索する。具体的には、検索管理部30は、そのキーを特定した際に用いたキーの範囲から、そのキーを除外して二分し、その2つの範囲を特定する。文字列特定部32は、その2つの範囲の中から文字列スコアが最大となるキーを特定する。 The search management unit 30 compares the prefix score specified from the prefix range with the character string score of the key specified from the key range. As a result of the comparison, if the highest score is a character string score, a key starting with the same prefix as that key is searched for a key having the next highest character string score. Specifically, the search management unit 30 excludes the key from the key range used when the key is specified, and bisects the key range to specify the two ranges. The character string specifying unit 32 specifies a key having the maximum character string score from the two ranges.
 最も大きいスコアが接頭辞スコアだった場合、その接頭辞の次に接頭辞スコアが大きい接頭辞を検索する。具体的には、検索管理部30は、その接頭辞を特定した際に用いた接頭辞の範囲から、その接頭辞を除外して二分し、その2つの範囲を特定する。接頭辞特定部31は、その範囲の中から接頭辞スコアが最大となる接頭辞を特定する。 If the highest score is a prefix score, the prefix with the highest prefix score is searched after that prefix. Specifically, the search management unit 30 excludes the prefix from the prefix range used when specifying the prefix and bisects the two ranges to specify the two ranges. The prefix specifying unit 31 specifies a prefix having the maximum prefix score from the range.
 出力部40は、検索管理部30により特定されたキーを検索結果として出力する。 The output unit 40 outputs the key specified by the search management unit 30 as a search result.
 接頭辞集合特定部20と、検索管理部30と、接頭辞特定部31と、文字列特定部32とは、プログラム(文字列検索プログラム)に従って動作するコンピュータのCPUによって実現される。例えば、プログラムは、文字列検索装置の記憶部(図示せず)に記憶され、CPUは、そのプログラムを読み込み、プログラムに従って、接頭辞集合特定部20、検索管理部30、接頭辞特定部31および文字列特定部32として動作してもよい。 The prefix set specifying unit 20, the search managing unit 30, the prefix specifying unit 31, and the character string specifying unit 32 are realized by a CPU of a computer that operates according to a program (character string search program). For example, the program is stored in a storage unit (not shown) of a character string search device, and the CPU reads the program, and according to the program, a prefix set specifying unit 20, a search management unit 30, a prefix specifying unit 31 and The character string specifying unit 32 may be operated.
 また、接頭辞集合特定部20と、検索管理部30と、接頭辞特定部31と、文字列特定部32とは、それぞれが専用のハードウェアで実現されていてもよい。 Further, each of the prefix set specifying unit 20, the search managing unit 30, the prefix specifying unit 31, and the character string specifying unit 32 may be realized by dedicated hardware.
 次に、本実施形態の文字列検索装置の動作を説明する。図6は、本実施形態の文字列検索装置の動作例を示すフローチャートである。ここでは、k個のキーを候補として選択するものとする。また、検索管理部30が、接頭辞特定部31が特定した接頭辞および接頭辞スコアのペアと、文字列特定部32が特定したキーおよび文字列スコアのペアとを保持する優先度付きキュー(図示せず)を含んでいるものとする。この優先度付きキューは、候補とする情報を保持するためのキューである。以下の説明では、優先度付きキューのことを、単にキューと記す。 Next, the operation of the character string search device of this embodiment will be described. FIG. 6 is a flowchart showing an operation example of the character string search apparatus according to the present embodiment. Here, k keys are selected as candidates. In addition, the search management unit 30 holds a priority queue (prefix and prefix score pair specified by the prefix specifying unit 31 and a key and character string score pair specified by the character string specifying unit 32). (Not shown). This priority queue is a queue for holding candidate information. In the following description, a priority queue is simply referred to as a queue.
 入力部10は、検索する文字列を入力する(ステップS11)。接頭辞集合特定部20は、検索情報記憶部50を参照して、入力された文字列を含む接頭辞の集合を特定する(ステップS12)。 The input unit 10 inputs a character string to be searched (step S11). The prefix set specifying unit 20 refers to the search information storage unit 50 and specifies a set of prefixes including the input character string (step S12).
 接頭辞特定部31は、接頭辞集合特定部20が特定した接頭辞の集合の中から、接頭辞スコアが最も大きい接頭辞を特定し、特定した接頭辞と接頭辞スコアのペアをキューに保持する(ステップS13)。 The prefix identification unit 31 identifies a prefix having the largest prefix score from the prefix set identified by the prefix set identification unit 20, and holds the identified prefix / prefix score pair in the queue. (Step S13).
 文字列特定部32は、特定された接頭辞で始まるキーの中から、文字列スコアが最も大きいキーを特定し、特定したキーと文字列スコアのペアをキューに保持する(ステップS14)。 The character string specifying unit 32 specifies the key having the highest character string score from the keys starting with the specified prefix, and holds the specified key / character string score pair in the queue (step S14).
 次に、検索管理部30は、キューに保持された接頭辞スコアまたは文字列スコアのうち、最大のスコアの接頭辞またはキーを特定する(ステップS15)。そして、検索管理部30は、最大のスコアが、接頭辞スコアか文字列スコアかを判断する(ステップS16)。 Next, the search management unit 30 specifies the prefix or key of the maximum score among the prefix scores or character string scores held in the queue (step S15). Then, the search management unit 30 determines whether the maximum score is a prefix score or a character string score (step S16).
 最大のスコアが文字列スコアだった場合(ステップS16における「文字列スコア」)、検索管理部30は、その文字列スコアのキーを、出力対象として特定し、キューからは除外する(ステップS17)。そして、文字列特定部32は、除外されたキーを特定する際に用いられたキーの範囲の中で、除外したキーの次に大きい文字列スコアのキーを特定し、特定したキーと文字列スコアのペアをキューに保持する(ステップS18)。 When the maximum score is the character string score (“character string score” in step S16), the search management unit 30 specifies the key of the character string score as an output target and excludes it from the queue (step S17). . Then, the character string specifying unit 32 specifies the key having the next highest character string score after the excluded key in the key range used when specifying the excluded key, and the specified key and character string The score pair is held in the queue (step S18).
 一方、最大のスコアが接頭辞スコアだった場合(ステップS16における「接頭辞スコア」)、検索管理部30は、その接頭辞スコアの接頭辞を、キューから除外する(ステップS19)。そして、接頭辞特定部31は、除外された接頭辞を特定する際に用いられた接頭辞の範囲の中で、除外した接頭辞の次に大きい接頭辞スコアの接頭辞を特定し、特定した接頭辞と接頭辞スコアのペアをキューに保持する(ステップS20)。 On the other hand, when the maximum score is a prefix score (“prefix score” in step S16), the search management unit 30 excludes the prefix of the prefix score from the queue (step S19). Then, the prefix identification unit 31 identifies and identifies a prefix having the next largest prefix score after the excluded prefix in the range of prefixes used to identify the excluded prefix. A pair of prefix and prefix score is held in the queue (step S20).
 さらに、文字列特定部32は、ステップS20で特定された接頭辞で始まるキーの中から、文字列スコアが最も大きいキーを特定し、特定したキーと文字列スコアのペアをキューに保持する(ステップS21)。 Further, the character string specifying unit 32 specifies the key having the largest character string score from the keys starting with the prefix specified in step S20, and holds the specified key and character string score pair in the queue ( Step S21).
 キューが空になった、または、キューの中で最大のスコアが、それまでに見つかっているk番目に大きい文字列スコアを下回った場合(ステップS22におけるYes)、検索管理部30は、それまでに見つかったキーを上位のキーとして出力する(ステップS23)。一方、キーが空でなく、かつ、キューの中で最大のスコアが、それまでに見つかっているk番目に大きい文字列スコアを下回っていない場合(ステップS22におけるNo)、ステップS15以降の処理が繰り返される。 If the queue is empty or the maximum score in the queue is lower than the kth largest string score found so far (Yes in step S22), the search management unit 30 will The key found in (1) is output as the upper key (step S23). On the other hand, when the key is not empty and the maximum score in the queue is not lower than the k-th largest character string score that has been found so far (No in step S22), the processing after step S15 is performed. Repeated.
 このように、接頭辞特定部31によって特定された接頭辞と接頭辞スコアのペア、および、文字列特定部32によって特定されたキーと文字列スコアのペアを、同じ優先度付きキューに入れておくことで、接頭辞スコアまたは文字列スコアのうち、最も大きいスコアのペアを取り出すことができる。 In this way, the prefix / prefix score pair specified by the prefix specifying unit 31 and the key / string score pair specified by the character string specifying unit 32 are put in the same priority queue. By setting it, it is possible to take out the pair with the largest score from the prefix score or the character string score.
 以下、図6に例示する動作を具体例を用いて説明する。図7は、文字列スコアの大きいキーを選択する処理の例を示す説明図である。図7に示す例では、文字列“gres”が入力され、この文字列に部分一致するキーを3つ(k=3)検索する方法を示している。図7の左側の枠内に例示するリストが、XBW構造を模式的に示したリストであり、数字が接頭辞スコアを表わし、文字が接頭辞を表わす。また、図7の右側の枠内に例示するリストが、トライ木を模式的に示したリストであり、数字が文字列スコアを表わし、文字がキーを表わす。 Hereinafter, the operation illustrated in FIG. 6 will be described using a specific example. FIG. 7 is an explanatory diagram illustrating an example of processing for selecting a key having a large character string score. In the example shown in FIG. 7, a character string “gres” is input, and a method of searching for three keys (k = 3) partially matching this character string is shown. The list illustrated in the left frame of FIG. 7 is a list schematically showing the XBW structure, where a number represents a prefix score and a letter represents a prefix. In addition, the list illustrated in the right frame of FIG. 7 is a list schematically showing a trie tree, where a number represents a character string score and a character represents a key.
 接頭辞集合特定部20は、文字列“gres”に部分一致するキーの範囲をXBW構造で表わされた接頭辞の集合の中から、“aggres”,“congres”,“progres”を候補として特定する。接頭辞が特定されれば、その接頭辞で始まるキーを特定することができる。 The prefix set specifying unit 20 uses “aggres”, “congres”, and “progress” as candidates from the set of prefixes represented by the XBW structure for the key range that partially matches the character string “gres”. Identify. If a prefix is specified, keys that begin with that prefix can be specified.
 接頭辞特定部31は、決定された接頭辞の集合の中から、入力された文字列“gres”で終わる接頭辞の中で接頭辞スコアが最も大きい接頭辞を選択する。図7では、選択された接頭辞を接頭辞スコアが高い順に並べた状態を示している。図7に示す例では、“congres”の接頭辞スコアが45で最も大きい。そこで、接頭辞特定部31は、“congres”を接頭辞として特定する。 The prefix specifying unit 31 selects a prefix having the highest prefix score from prefixes ending with the input character string “gres” from the determined set of prefixes. FIG. 7 shows a state in which the selected prefixes are arranged in descending order of prefix score. In the example shown in FIG. 7, the prefix score of “congres” is 45, which is the largest. Therefore, the prefix specifying unit 31 specifies “congres” as a prefix.
 文字列特定部32は、選択された接頭辞で始まるキーの中から文字列スコアが最も大きいキーを選択する。図7に示す例では、“congres”が接頭辞になるキーは、“congress”,“congressional”,“congressmen”の3つ存在する。このうち、文字列スコアが最も大きいキーは、“congress”である。そこで、文字列特定部32は、1つ目のキーとして、“congress”を特定し、検索管理部30は、特定された“congress”を検索対象のキーと特定する。 The character string specifying unit 32 selects a key having the largest character string score from keys starting with the selected prefix. In the example illustrated in FIG. 7, there are three keys having “congres” as a prefix, “congress”, “congressional”, and “congressmen”. Among these, the key having the largest character string score is “congress”. Therefore, the character string specifying unit 32 specifies “congress” as the first key, and the search management unit 30 specifies the specified “congress” as a search target key.
 この段階では、まだキーは1つしか特定されていないため、キーを特定する処理が繰り返される。 At this stage, since only one key has been specified, the process of specifying the key is repeated.
 検索管理部30は、上述するように、候補とする情報を保持するための優先度付きキュー(図示せず)を備えておき、このキューにこれまで見つかった接頭辞およびキーをそのスコアとともに保持しておく。 As described above, the search management unit 30 includes a priority queue (not shown) for holding candidate information, and holds the prefix and key found so far in the queue together with the score. Keep it.
 検索管理部30は、キューを参照し、キューに保持されている接頭辞及びキーの中から、最大のスコアのものを選択する。選択されたものがキーであった場合、文字列特定部32は、そのキーを検索したときと同じキーの範囲のなかで、次に文字列スコアが大きいキーを検索する。選択されたものが接頭辞であった場合、接頭辞特定部31は、その接頭辞を検索したときと同じ接頭辞の範囲のなかで、その接頭辞の次に接頭辞スコアが大きい接頭辞を検索する。 The search management unit 30 refers to the queue and selects the one with the highest score from the prefixes and keys held in the queue. When the selected item is a key, the character string specifying unit 32 searches for the key having the next highest character string score within the same key range as that when the key is searched. If the selected one is a prefix, the prefix specifying unit 31 selects a prefix having the next highest prefix score after the prefix within the same prefix range as that when the prefix is searched. Search for.
 この例の場合、接頭辞スコア45の接頭辞“congres”と、文字列スコア45のキー“congress”がキューに保持されている。このときスコアは同値なので、キーと接頭辞のどちらを先に検索しても良い。キーを検索する場合、検索管理部30は、まずキー“congress”をポップしてキューから除外する。そして、文字列特定部32は、キー“congress”を得たときと同じ接頭辞“congres”で始まるキーの中で、キー“congress”の次に大きい文字列スコアを持つキーを検索する。具体的には、検索管理部30は、キー“congress”を得たときに探索したキーの範囲について、今度は“congress”を除外して二分し、文字列特定部32は、その2つの範囲の中で文字列スコアが最大となるキーを検索する。このとき、キー“congress”を除外して二分した2つの範囲のうち、“congress”よりも辞書順で前の範囲にはキーが存在しない。そのため、辞書順が後の範囲で最大の文字列スコアとなるキーだけを求めればよい。そのキーは文字列スコア13のキー“congressmen”である。よって、検索管理部30は、このキーを新たにキューに保持する。 In this example, the prefix “congres” of the prefix score 45 and the key “congress” of the character string score 45 are held in the queue. At this time, since the scores are the same value, either the key or the prefix may be searched first. When searching for a key, the search management unit 30 first pops the key “congress” and removes it from the queue. Then, the character string specifying unit 32 searches for a key having the next highest character string score after the key “congress” among keys starting with the same prefix “congres” as when the key “congress” was obtained. Specifically, the search management unit 30 bisects the key range searched for when the key “congress” is obtained, this time excluding “congress”, and the character string specifying unit 32 determines the two ranges. Search for the key with the highest string score. At this time, of the two ranges divided into two parts by excluding the key “congress”, there is no key in the range preceding the “congress” in dictionary order. Therefore, it is only necessary to obtain a key having a maximum character string score in the later range of the dictionary order. The key is the key “congressmen” of the character string score 13. Therefore, the search management unit 30 newly holds this key in the queue.
 接頭辞を検索する場合、検索管理部30は、まず接頭辞“congres”をポップしてキューから除外する。そして、接頭辞特定部31は、接頭辞“congres”の次に大きい接頭辞スコアを持つ接頭辞を検索する。具体的には、検索管理部30は、接頭辞“congres”を得た時に探索した接頭辞の範囲について、今度は“congres”を除外して二分し、接頭辞特定部31は、その2つの範囲の中で接頭辞スコアが最大となる接頭辞を検索する。このとき、接頭辞“congres”を除外して二分した2つの範囲で接頭辞スコアが大きい接頭辞は、それぞれ、接頭辞スコア12の接頭辞“aggres”および接頭辞スコア21の接頭辞“progres”である。よって、検索管理部30は、この2つの接頭辞を新たにキューに保持する。 When searching for a prefix, the search management unit 30 first pops the prefix “congres” and removes it from the queue. Then, the prefix specifying unit 31 searches for a prefix having a prefix score that is higher than the prefix “congres”. Specifically, the search management unit 30 bisects the range of the prefix searched when the prefix “congres” is obtained, excluding “congres”, and the prefix identification unit 31 Find the prefix with the highest prefix score in the range. At this time, prefixes having a large prefix score in the two ranges divided into two parts by excluding the prefix “congres” are prefix “aggres” of prefix score 12 and prefix “progres” of prefix score 21, respectively. It is. Therefore, the search management unit 30 newly holds these two prefixes in the queue.
 さらに続けて、文字列特定部32は、接頭辞“aggres”および接頭辞“progres”について、それぞれの接頭辞で始まる文字列スコアが最大のキーを取得する。これにより、文字列スコア12のキー“aggresive”、および文字列スコア21のキー“progress”が得られる。これにより、さきほど取得した2つの接頭辞のうち、接頭辞“aggres”の接頭辞スコアが12であり、接頭辞“progres”の接頭辞スコアが21であったことを確認できる。 Further, the character string specifying unit 32 acquires the key having the maximum character string score starting with each prefix for the prefix “aggres” and the prefix “progres”. Thereby, the key “aggressive” of the character string score 12 and the key “progress” of the character string score 21 are obtained. Thereby, it can be confirmed that the prefix score of the prefix “aggres” is 12 and the prefix score of the prefix “progres” is 21 of the two prefixes acquired earlier.
 本実施形態では、RMQ構造だけを保持しており、接頭辞スコア自体は保持していない。RMQ構造だけでは、接頭辞スコアが最大となる接頭辞がどれであるかを見つけることはできるが、その具体的な接頭辞スコアは計算できない。よって、その範囲で接頭辞スコアが最大となる接頭辞を取得したあと、その接頭辞スコアが具体的にどんな値であるか判断するためには、その接頭辞で始まるキーのうち最大となる文字列スコアを取得する必要がある。 In this embodiment, only the RMQ structure is retained, and the prefix score itself is not retained. The RMQ structure alone can find which prefix has the largest prefix score, but cannot calculate its specific prefix score. Therefore, after obtaining the prefix with the largest prefix score in the range, in order to determine the specific value of the prefix score, the largest character among the keys starting with that prefix Need to get column score.
 以上の処理により、キューには5つのスコアが保持されている。接頭辞スコア21の接頭辞“progres”、接頭辞スコア12の接頭辞“aggres”、文字列スコア21のキー“progress”、文字列スコア13のキー“congressmen”、文字列スコア12のキー“aggresive”である。 As a result of the above processing, five scores are held in the queue. Prefix “progres” of prefix score 21, prefix “aggres” of prefix score 12, key “progress” of character string score 21, key “congressmen” of character string score 13, key “aggressive” of character string score 12 ".
 このうち、スコアが最大であるのは、接頭辞スコア21の接頭辞“progres”もしくは文字列スコア21のキー“progress”なので、このどちらかの接頭辞もしくはキーについて探索すればよい。 Of these, the score is highest because the prefix “progress” of the prefix score 21 or the key “progress” of the character string score 21, and it is sufficient to search for either of these prefixes or keys.
 この処理を繰り返す。もし、新たに見つかった接頭辞の接頭辞スコアが、それまでに見つかっているk番目に大きい文字列スコアを下回っている場合は、検索管理部30は、その接頭辞をキューに登録しない。その接頭辞の次に接頭辞スコアが大きい接頭辞は、それよりもさらにスコアが小さいからである。同様に、新たに見つかったキーのスコアが、それまでに見つかっているk番目に大きい文字列スコアを下回っている場合、検索管理部30は、そのキーをキューに登録しない。これにより、接頭辞スコアが小さい接頭辞や文字列スコアが小さいキーについては探索せずに済ませることができ、上位k個のキーを効率的に収集できる。 Repeat this process. If the prefix score of the newly found prefix is lower than the k-th largest character string score found so far, the search management unit 30 does not register the prefix in the queue. This is because a prefix having the highest prefix score after that prefix has a smaller score. Similarly, when the score of a newly found key is lower than the kth largest character string score found so far, the search management unit 30 does not register the key in the queue. Thereby, it is possible to avoid searching for a prefix having a small prefix score or a key having a small character string score, and the top k keys can be efficiently collected.
 キューが空になるか、キューの中で最大となるスコアが、それまでに見つかっているk番目に大きい文字列スコアを下回ったら探索終了となる。 The search ends when the queue is empty or the maximum score in the queue falls below the kth largest string score found so far.
 以上のように、本実施形態によれば、接頭辞集合特定部20が、接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定し、接頭辞特定部31が、入力された文字列で終わる接頭辞の集合の中から、接頭辞スコアが最大の接頭辞を特定する。そして、文字列特定部32が、特定された接頭辞で始まるキーの中から、文字列スコアが最大のキーを特定する。 As described above, according to the present embodiment, the prefix set specifying unit 20 specifies a set of prefixes ending with the input character string from the set of prefixes, and the prefix specifying unit 31 is input. The prefix with the highest prefix score is identified from the set of prefixes ending with the specified string. And the character string specific | specification part 32 specifies the key with the largest character string score from the keys which begin with the specified prefix.
 具体的には、本実施形態では、接頭辞とキーについてのインデックスを作成しているため、全ての部分文字列についてインデックスを作成するよりも辞書サイズを削減できる。また、本実施形態では、接頭辞特定部31が、接頭辞スコアが大きい接頭辞を特定し、文字列特定部32が、その接頭辞の中から文字列スコアの大きいキーを探索するため、スコアが大きい方から探せば、効率的に上位k個のキーを検索できる。したがって、データ量を削減しつつ高速に文字列の部分一致検索を行うことができる。 Specifically, in this embodiment, since the index for the prefix and the key is created, the dictionary size can be reduced as compared with the case where the index is created for all the partial character strings. In this embodiment, the prefix specifying unit 31 specifies a prefix having a large prefix score, and the character string specifying unit 32 searches for a key having a large character string score from the prefix. If the key is searched from the largest, the top k keys can be efficiently searched. Therefore, it is possible to perform a partial match search for character strings at high speed while reducing the amount of data.
 例えば、日本語辞書や英語辞書、クエリログやURLは、共通する接頭辞を有することが多い。本実施形態の文字列検索装置では、共通する接頭辞をまとめることが可能なデータ構造としてトライ木を利用するため、データサイズを削減できる。なお、本実施形態では、トライ木のデータ構造を利用してキーを表わす場合を例示したが、データ構造は、パトリシア木であってもよい。パトリシア木を用いることで、トライ木の木構造よりもデータサイズを削減できる。 For example, Japanese and English dictionaries, query logs and URLs often have a common prefix. In the character string search apparatus according to the present embodiment, the trie tree is used as a data structure capable of collecting common prefixes, so that the data size can be reduced. In the present embodiment, the case where the key is represented using the data structure of the trie tree is illustrated, but the data structure may be a Patricia tree. By using the Patricia tree, the data size can be reduced more than the tree structure of the trie tree.
 また、本実施形態の文字列検索装置は、検索範囲を管理する検索管理部30を備えている。具体的には、検索管理部30は、文字列特定部32によって特定されたキーの接頭辞で始まるキーの中から、すでに特定されたキーを除く範囲を特定し、接頭辞集合特定部20により特定された接頭辞の集合から、接頭辞特定部31により特定された接頭辞を除く範囲を特定する。そして、接頭辞特定部31が、検索管理部30により特定された接頭辞の範囲から、接頭辞スコアが最大の接頭辞を特定し、文字列特定部32が、検索管理部30により特定されたキーの範囲から、文字列スコアが最大のキーを特定する。このようにすることで、辞書向けのデータ構造として利用されるXBWをTop-k検索に拡張できるため、上位k個の候補を部分一致検索する際、省スペースかつ高速に処理を実現できる。 In addition, the character string search device of the present embodiment includes a search management unit 30 that manages the search range. Specifically, the search management unit 30 specifies a range excluding the already specified key from the keys starting with the key prefix specified by the character string specifying unit 32, and the prefix set specifying unit 20 A range excluding the prefix specified by the prefix specifying unit 31 is specified from the specified prefix set. Then, the prefix specifying unit 31 specifies the prefix having the largest prefix score from the prefix range specified by the search managing unit 30, and the character string specifying unit 32 is specified by the search managing unit 30. The key with the highest string score is identified from the key range. In this way, XBW used as a data structure for a dictionary can be expanded to a Top-k search, and therefore, space saving and high-speed processing can be realized when a partial match search is performed for the top k candidates.
実施形態2.
 次に、本発明による文字列検索装置の第2の実施形態を説明する。本実施形態の文字列検索装置の構成は、第1の実施形態と同様である。ただし、第2の実施形態の文字列検索装置は、第1の実施形態の文字列装置よりも、保持するデータ量をより削減できるようにするものである。
Embodiment 2. FIG.
Next, a second embodiment of the character string search device according to the present invention will be described. The configuration of the character string search device of this embodiment is the same as that of the first embodiment. However, the character string search device according to the second embodiment can reduce the amount of data to be retained more than the character string device according to the first embodiment.
 第1の実施形態で説明したトライ木Tからは、2つのデータ構造が生成される。1つは、接頭辞に関するデータ構造であり、もう1つは、キーに関するデータ構造である。 Two data structures are generated from the trie tree T described in the first embodiment. One is a data structure related to a prefix, and the other is a data structure related to a key.
 接頭辞に関するデータ構造は、トライ木TのXBW表現であるxbwと、付随する第一のRMQ構造を含む。第1の実施形態で説明したように、xbw上では、接頭辞が末尾からソートされた順序に並んでいる。 The data structure related to the prefix includes xbw which is the XBW representation of the trie tree T and the accompanying first RMQ structure. As described in the first embodiment, on xbw, the prefixes are arranged in the order sorted from the end.
 また、第1の実施形態で示した接頭辞スコア列Rに対し、第一のRMQ構造が生成される。このとき、検索情報記憶部50は、接頭辞スコア列Rを、明示的に保持しなくてもよく、接頭辞スコア列Rから計算された第一のRMQ構造のみ保持するようにしてもよい。 Further, a first RMQ structure is generated for the prefix score string R p shown in the first embodiment. In this case, the search information storage unit 50, a prefix score column R p, may not be explicitly held, be held only the first RMQ structure calculated from the prefix score column R p Good.
 キーに関するデータ構造は、トライ木Tから生成されるパトリシア木Tと、第二のRMQ構造と、文字列スコア列Rを含む。パトリシア木Tの木構造は、DFUDSで表わされる。さらに、パトリシア木Tにおいて、木構造の葉ノードのみを識別するため、ノード数と同数のビット列が用意される。 The data structure relating to the key includes a Patricia tree T c generated from the trie tree T, a second RMQ structure, and a character string score string R k . Tree structure of a Patricia tree T c is represented by DFUDS. Further, in the Patricia tree Tc , in order to identify only the leaf nodes of the tree structure, the same number of bit strings as the number of nodes are prepared.
 一般的なパトリシア木は、各ノードに対応する文字列を保持する。一方、本実施形態の検索情報記憶部50は、各ノードに対応する文字列を除去し、ノード間の親子関係を表わす木構造のみ記憶する。このような木構造のみ記憶する理由については、後述される。 A general Patricia tree holds a character string corresponding to each node. On the other hand, the search information storage unit 50 of this embodiment removes the character string corresponding to each node and stores only the tree structure representing the parent-child relationship between the nodes. The reason for storing only such a tree structure will be described later.
 以下の説明では、図5に例示するように、各キーが、先頭の文字から辞書式順序にソートされ、各キーには、その順番にキーIDが振られているものとする。また、各接頭辞も、末尾から辞書式順序にソートされ、各接頭辞には、その順番に接頭辞IDが振られているものとする。また、接頭辞IDの範囲を[s,e]と記し、接頭辞を表わす集合S上の範囲を[s,e]と記す。 In the following description, as illustrated in FIG. 5, it is assumed that each key is sorted in lexicographic order from the first character, and each key is assigned a key ID in that order. Each prefix is also sorted in lexicographic order from the end, and each prefix is assigned a prefix ID in that order. Further, the range of the prefix ID is written as [s p , e p ], and the range on the set S representing the prefix is written as [s s , e s ].
 接頭辞集合特定部20は、入力された文字列で終わる接頭辞IDの範囲[s,e]を特定する。具体的には、接頭辞集合特定部20は、xbwを用いて、接頭辞の末尾が入力された文字列になっている範囲[s,e]を特定する。ただし、この範囲[s,e]は、集合S上の範囲であるため、接頭辞IDの範囲[s,e]に変換する必要がある。そこで、接頭辞集合特定部20は、Slast上で[s,e]に含まれる最初の1と最後の1が何番目の1であるか特定することで、[s,e]を特定する。Slast上で1になる要素が、接頭辞IDと同じ順序で一対一に対応しているからである。 The prefix set specifying unit 20 specifies a range [s p , e p ] of prefix IDs ending with the input character string. Specifically, the prefix set identifying unit 20 uses the XBW, range that is a character string end is entered prefix [s s, e s] to identify. However, since this range [s s , e s ] is a range on the set S, it is necessary to convert it to a prefix ID range [s p , e p ]. Therefore, the prefix set identification unit 20 identifies [s p , e p by identifying what number the first 1 and the last 1 included in [s s , e s ] are on S last. ] Is specified. This is because the elements that become 1 on S last correspond one-to-one in the same order as the prefix ID.
 接頭辞特定部31は、特定された接頭辞IDの範囲[s,e]から、接頭辞スコアが最大の接頭辞を特定する。具体的には、接頭辞特定部31は、第一のRMQ構造を利用して、[s,e]の範囲内で接頭辞スコアが最大になる接頭辞の位置を特定する。なお、ここで特定された接頭辞の位置をiと記す。 The prefix specifying unit 31 specifies the prefix having the maximum prefix score from the range [s p , e p ] of the specified prefix ID. Specifically, the prefix specifying unit 31 uses the first RMQ structure to specify the position of the prefix having the maximum prefix score within the range of [s p , e p ]. Incidentally, denoted here identified position of the prefix and i p.
 検索管理部30は、特定された接頭辞の位置iから、この接頭辞で始まるキーの範囲を特定する。以下、特定された接頭辞で始まるキーの範囲を[s,e]と記す。具体的には、検索管理部30は、まず、S上での対応する位置iとして、接頭辞の位置iに対応する接頭辞を有する最後のノードを特定する。 Search management unit 30 from the position i p of the identified prefix identifies a range of keys that start with the prefix. Hereinafter, a range of keys starting with the specified prefix is denoted as [s k , e k ]. Specifically, the retrieval management section 30 first as the corresponding position i s on S, identifies the last node having the prefix which corresponds to the position i p prefix.
 次に、検索管理部30は、xbwにおいて、この接頭辞を表わす文字列を復元する。具体的には、検索管理部30は、xbwにおけるi行目が表わすノードから親を辿っていったときの文字を結合することで、文字列を復元する。ノードから親に向かって移動する回数は、接頭辞の長さと等しい。 Next, the search management unit 30 restores a character string representing this prefix in xbw. Specifically, the retrieval management unit 30, by combining the characters when went following the parent from a node representing i s th row in XBW, restores the character string. The number of moves from node to parent is equal to the prefix length.
 このとき、検索管理部30は、辿ったS上の各位置iについて、iより手前で最も近いSlast[i]=1になる位置iとの差分d=i-iを計算する。そして、検索管理部30は、親に向かって辿った順序と逆の順に、計算した値を配列dへ格納する。ただし、上記のiが存在しない場合、検索管理部30は、i=0とした値を配列dへ格納する。 In this case, the retrieval management unit 30, for each location i s on traced S, the difference between i s closest S last [i f] than in front = becomes 1 position i f d = i s -i f Calculate Then, the search management unit 30 stores the calculated values in the array d in the reverse order of the order traced toward the parent. However, if the above i f are not present, the search management unit 30 stores the value obtained by the i f = 0 to the array d.
 次に、検索管理部30は、パトリシア木Tにおいて、配列dに保存されている順序に従い、親ノードから子ノードへ対象とする位置を移動させる。ただし、配列dの対応する値が1である場合、検索管理部30は、その値を無視して、次の値について処理を行う。 Next, the search management unit 30 moves the target position from the parent node to the child node according to the order stored in the array d in the Patricia tree Tc . However, when the corresponding value of the array d is 1, the search management unit 30 ignores the value and performs processing for the next value.
 xbwとTは、同じトライ木Tから生成されているため、ノードの子が1つである場合を除き、各ノードが持つ子の数とその順序は一致する。そのため、配列dに従ってT上の対象とする位置を移動させると、その位置は、接頭辞に対応するT上のノードuに到達する。 Since xbw and Tc are generated from the same trie tree T, the number of children of each node and the order thereof are the same except when there is one child of the node. Therefore, when the target position on T c is moved according to the array d, the position reaches the node u c on T c corresponding to the prefix.
 検索管理部30は、次に、DFUDSを用いて、到達したノードuの子孫に対応するキーの範囲[s,e]を特定する。[s,e]に含まれるキーは、全てuの子であることから、[s,e]は、特定された接頭辞で始まるキーの範囲を示していると言える。 Search management unit 30 then uses the DFUDS, range of keys corresponding to the descendants of node u c has been reached [s k, e k] specifying the. Since the keys included in [s k , e k ] are all children of u c , it can be said that [s k , e k ] indicates a range of keys starting with the specified prefix.
 文字列特定部32は、特定されたキーの範囲[s,e]から、文字列スコアが最大のキーID(以下、iと記す。)を特定する。具体的には、文字列特定部32は、第二のRMQ構造を利用して、[s,e]の範囲内で接頭辞スコアが最大になるキーの位置iを特定する。 The character string specifying unit 32 specifies the key ID (hereinafter referred to as i k ) having the maximum character string score from the specified key range [s k , e k ]. Specifically, the character string specifying unit 32 uses the second RMQ structure to specify the key position i k having the maximum prefix score within the range of [s k , e k ].
 文字列特定部32は、特定したキーIDの位置iから、キーの文字列を特定する。iは、パトリシア木T上のi番目の葉ノードuに対応する。そこで、文字列特定部32は、uからパトリシア木Tの親ノードへ辿り、親に向かって辿った順序と逆の順に、子ノードの番号を配列dへ格納する。文字列特定部32は、この配列dに従って、xbwをルートから順に辿ることで、uに対応するxbw上のノードの位置を特定できる。なお、子ノードが1つの場合、配列dを参照せず無条件に葉ノードの方向へ移動させればよい。パトリシア木Tの葉ノードに対応するトライ木のノードの子孫に枝分かれは存在しない。そのため、文字列特定部32は、一本鎖を辿ることで、キーを正確に復元できる。 Character string specifying unit 32, from the position i k of the specified key ID, and identifies the string of keys. i k corresponds to the i k th leaf node u i on the Patricia tree T c . Therefore, the character string specifying unit 32 traces from u i to the parent node of the Patricia tree T c and stores the numbers of the child nodes in the array d in the reverse order of the order traced toward the parent. The character string specifying unit 32 can specify the position of the node on xbw corresponding to u i by tracing xbw sequentially from the root according to this array d. When there is one child node, it is only necessary to move it in the direction of the leaf node unconditionally without referring to the array d. There is no branching in the descendants of the trie node corresponding to the leaf node of the Patricia tree Tc . Therefore, the character string specifying unit 32 can accurately restore the key by following a single chain.
 以上のように、キーの情報はxbwから得られる。そのため、パトリシア木の各ノードが保持している文字列を除外して、ノード間の親子関係だけ残しておけばよい。 As described above, key information can be obtained from xbw. Therefore, it is sufficient to exclude the character string held by each node of the Patricia tree and leave only the parent-child relationship between the nodes.
 例えば、図5において、“$ab”には、ルートから1番目の子供を選択し、さらに、選択したノードで1番目の子供を選択すると辿りつける。そこで、検索管理部30は、選択したノードを特定するための配列dにd=1,1という情報を格納してもよい。 For example, in FIG. 5, “$ ab” can be reached by selecting the first child from the route and further selecting the first child at the selected node. Therefore, the search management unit 30 may store information of d = 1, 1 in the array d for specifying the selected node.
 以下、図5に示す例を使って、本実施形態の文字列検索装置の具体的な動作を説明する。ここでは、検索クエリP=“ab”とし、k=2とする。Pで終わるS上の範囲は、[s,e]=[7,9]である。これに対応するR上の範囲は、[s,e]=[4,5]である。この範囲で最大の接頭辞スコアの位置は、i=4である。これは、S上における位置i=8に対応する。このiに対応する接頭辞は、“$ab”であり、どちらも1番目の子であるため、配列d=1,1が得られる。 Hereinafter, the specific operation of the character string search apparatus of this embodiment will be described using the example shown in FIG. Here, the search query P = “ab” and k = 2. The range on S that ends with P is [s s , e s ] = [7, 9]. The corresponding range on R p is [s p , e p ] = [4, 5]. The position of the largest prefix score in this range is i p = 4. This corresponds to a position i S = 8 on S. Since the prefix corresponding to i S is “$ ab” and both are the first children, the array d = 1, 1 is obtained.
 T上で、ルートノードを出発し、1番目の子ノードに移動し、もう一度1番目の子ノードに移動すると、“$ab”に対応するノードに到着する。このノード配下で最大の文字列スコアは9であり、このキーは、キーID=1で識別されるキーである。そこで、キーIDと文字列スコアのペア〈1,9〉が得られる。これが、図5における辞書で、最大の文字列スコアのキーである。 On Tc , the route node is departed, moved to the first child node, and once again moved to the first child node, the node corresponding to “$ ab” is reached. The maximum character string score under this node is 9, and this key is a key identified by key ID = 1. Therefore, the key ID and character string score pair <1, 9> is obtained. This is the key of the maximum character string score in the dictionary in FIG.
 上位2個目のキーは、同じ接頭辞の2番目のキーか、他の接頭辞の1番目のキーである。同じ接頭辞の2番目のキーは、キーID=0で識別されるキーであり、その文字列スコア=3である(以下、〈0,3〉と記す。)。一方、他の接頭辞の1番目のキーを求めるには、最大の接頭辞スコアとしてさきほど特定された接頭辞i=4を除外した接頭辞の範囲を特定する。この接頭辞i=4を除外して接頭辞の範囲を2分することで、範囲[s,e]=[5,5]が特定される。そこで、範囲[s,e]=[5,5]で最大の接頭辞スコアの接頭辞を特定し、その中で最大の文字列スコアのキーを特定する処理が行われる。図5に示す例では、接頭辞“$cab”で始まるキーの中で最大文字列スコアのキーを特定することに相当する。この結果、新たに、キーIDと文字列スコアのペア〈2,4〉が特定される。 The top second key is the second key with the same prefix or the first key with another prefix. The second key of the same prefix is a key identified by key ID = 0, and its character string score = 3 (hereinafter referred to as <0, 3>). On the other hand, in order to obtain the first key of another prefix, the prefix range excluding the prefix i p = 4 specified earlier as the maximum prefix score is specified. The range [s p , e p ] = [5, 5] is specified by excluding the prefix i p = 4 and dividing the prefix range into two. Therefore, the range [s p, e p] = [5,5] in identifying the prefix largest prefix score, processing for specifying the key for maximum word score takes place. In the example shown in FIG. 5, this corresponds to specifying the key with the maximum character string score among the keys starting with the prefix “$ cab”. As a result, a key ID / character string score pair <2, 4> is newly specified.
 ここまでに、3つの候補のペアが特定されるが、スコアが小さい〈0,3〉は除外される。最終的に残ったペアは、〈1,9〉と〈2,4〉である。この2つのペアが特定された後、各ペアからキーを復元する処理が行われる。Tにおけるパスdは、それぞれ、1,2と、2,1である。このキーに対応するもとの辞書におけるキーは、xbwをルートから辿ることで一意に求まり、“$abcc#”と“$cab#”になる。 So far, three candidate pairs have been identified, but <0,3> having a small score is excluded. The finally remaining pairs are <1, 9> and <2, 4>. After these two pairs are specified, a process for restoring the key from each pair is performed. The paths d at Tc are 1, 2, and 2, 1, respectively. The key in the original dictionary corresponding to this key is uniquely obtained by tracing xbw from the root, and becomes “$ abcc #” and “$ cab #”.
 次に、本実施形態で説明したデータ構造を用いた場合のデータサイズについて説明する。トライ木Tとスコア配列Rが与えられるとすると、ノード数t>キー数lが成り立つ。一般に、ノード数tは、キー数lの10倍程度である。 Next, the data size when the data structure described in this embodiment is used will be described. If a trie tree T and a score array R k are given, the number of nodes t> the number of keys l holds. In general, the number of nodes t is about 10 times the number of keys l.
 本実施形態で説明したデータ構造を用いた場合、データサイズは、以下に示す式2で表わされる。 When the data structure described in this embodiment is used, the data size is expressed by the following equation 2.
 |XBW|+|第一のRMQ構造(接頭辞)|
+|T(パトリシア木)|+|第二のRMQ構造(キー)|+|R(スコア)|
                                ・・・(式2)
| XBW | + | First RMQ structure (prefix) |
+ | T c (Patricia tree) | + | second RMQ structure (key) | + | R k (score) |
... (Formula 2)
 式2において、|XBW|は、トライ木Tをxbwで表わした場合のデータサイズを示し、|R(スコア)|は、文字列スコアの配列のサイズを表わす。 In Expression 2, | XBW | represents the data size when the trie tree T is represented by xbw, and | R k (score) | represents the size of the array of character string scores.
 また、T(パトリシア木)は、トライ木Tから生成され、その木構造は、DFUDSで表現される。本実施形態では、木構造の葉ノードだけを識別するため、ノード数と同数のビット列が用意され、このビット列の各ビットを用いて葉ノードか否かが識別される。そして、本実施形態のパトリシア木は、文字列が除去され、木構造のみで表わされる。上述するように、文字列の情報は、xbwから得られるためである。 T c (Patricia tree) is generated from the trie tree T, and the tree structure is expressed in DFUDS. In the present embodiment, in order to identify only the leaf nodes of the tree structure, the same number of bit strings as the number of nodes are prepared, and each bit of this bit string is used to identify whether or not it is a leaf node. The Patricia tree of this embodiment is represented only by a tree structure from which character strings are removed. This is because character string information is obtained from xbw as described above.
 パトリシア木のノード数は、最大2l-1であるが、DFUDSでノード数の2倍、葉ノードを識別するビット列にノード数と同数のビットが必要になる。そのため、|T(パトリシア木)|は、6l+o(l)ビットで表現される。 The maximum number of nodes in the Patricia tree is 2l-1, but twice the number of nodes in DFUDS, and the bit string for identifying leaf nodes requires the same number of bits as the number of nodes. Therefore, | T c (Patricia tree) | is represented by 6l + o (l) bits.
 さらに、キーの文字列スコア配列R(スコア)に対して、第二のRMQ構造(キー)が生成される。|第二のRMQ構造(キー)|は、2l+o(l)ビットで表現される。 Further, a second RMQ structure (key) is generated for the key character string score array R k (score). The | second RMQ structure (key) | is represented by 2l + o (l) bits.
 |XBW|と|R(スコア)|を、辞書およびスコアの実現に必要な最低限のデータとすると、本実施形態で説明したデータ構造を実現する場合、オーバヘッドは、最大で2t+6l+o(t)となり、一般的な方法に比べ、データ量が削減されている。 When | XBW | and | R k (score) | are the minimum data necessary for realizing the dictionary and the score, when realizing the data structure described in this embodiment, the overhead is 2t + 6l + o (t) at the maximum. Thus, the amount of data is reduced compared to the general method.
 次に、本実施形態で説明したデータ構造を用いた場合の計算量について説明する。計算量は、O(k(log(k)+|P|+h))で算出される。ここで、|P|はクエリの長さを示し、hは、辞書に登録されているキーの平均長さを表わす。このように、本実施形態で説明したデータ構造を用いた場合、データサイズには依存せずに検索処理を実行できる。 Next, the amount of calculation when the data structure described in this embodiment is used will be described. The calculation amount is calculated by O (k (log (k) + | P | + h)). Here, | P | indicates the length of the query, and h indicates the average length of the keys registered in the dictionary. Thus, when the data structure described in the present embodiment is used, the search process can be executed without depending on the data size.
 次に、本発明の概要を説明する。図8は、本発明による文字列検索装置の概要を示すブロック図である。本発明による文字列検索装置は、優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列(例えば、キー)の集合から、入力された文字列を含む検索候補文字列を検索する文字列検索装置であって、各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合(例えば、XBWのデータ構造による接頭辞の集合)から、入力された文字列で終わる接頭辞の集合を決定する接頭辞集合特定部81(例えば、接頭辞集合特定部20)と、入力された文字列で終わる接頭辞の集合の中から、接頭辞ごとにその接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコア(例えば、式1により定義される接頭辞スコア)が最大の接頭辞を特定する接頭辞特定部82(例えば、接頭辞特定部31)と、特定された接頭辞で始まる検索候補文字列の中から、文字列スコアが最大の検索候補文字列を特定する文字列特定部83(例えば、文字列特定部32)とを備えている。 Next, the outline of the present invention will be described. FIG. 8 is a block diagram showing an outline of a character string search apparatus according to the present invention. A character string search device according to the present invention includes a search candidate character string including an input character string from a set of search candidate character strings (for example, keys) associated with a character string score indicating a degree to be preferentially searched. Is a character string search device for searching for a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string (for example, a set of prefixes based on an XBW data structure) From the prefix set specifying unit 81 (for example, the prefix set specifying unit 20) that determines a set of prefixes ending with the input character string and the prefix set ending with the input character string For each word, the prefix score defined by the largest character string score among the character string scores associated with the search candidate character string starting with the prefix (for example, the prefix score defined by Equation 1) is the largest. prefix A character string specification that specifies a search candidate character string having the maximum character string score from among a prefix specification unit 82 (for example, prefix specification unit 31) that specifies the character string and a search candidate character string that starts with the specified prefix Part 83 (for example, character string specifying part 32).
 このように、接頭辞特定部82が、接頭辞スコアが大きい接頭辞を特定し、文字列特定部83が、その接頭辞の中から文字列スコアの大きい検索候補文字列を探索するため、スコアが大きい方から探せば、効率的に上位k個の検索候補文字列を検索できる。 Thus, the prefix specifying unit 82 specifies a prefix having a large prefix score, and the character string specifying unit 83 searches for a search candidate character string having a large character string score from the prefix. If the search is started from the larger one, the top k search candidate character strings can be efficiently searched.
 さらに、文字列検索装置は、検索範囲を管理する検索管理部(例えば、検索管理部30)を備えていてもよい。検索管理部は、文字列特定部83によって特定された検索候補文字列の接頭辞で始まる検索候補文字列の中から、すでに特定された検索候補文字列を除く検索候補文字列の範囲を特定し、接頭辞集合特定部81により特定された接頭辞の集合から、接頭辞特定部82により特定された接頭辞を除いた接頭辞の範囲を特定してもよい。また、接頭辞特定部82は、検索管理部により特定された接頭辞の範囲から、接頭辞スコアが最大の接頭辞を特定し、文字列特定部83は、検索管理部により特定された検索候補文字列の範囲から、文字列スコアが最大の検索候補文字列を特定してもよい。 Furthermore, the character string search device may include a search management unit (for example, the search management unit 30) that manages the search range. The search management unit specifies the range of the search candidate character string excluding the already specified search candidate character string from the search candidate character strings starting with the prefix of the search candidate character string specified by the character string specifying unit 83. The prefix range excluding the prefix specified by the prefix specifying unit 82 from the set of prefixes specified by the prefix set specifying unit 81 may be specified. Further, the prefix specifying unit 82 specifies the prefix having the maximum prefix score from the prefix range specified by the search management unit, and the character string specifying unit 83 is the search candidate specified by the search management unit. A search candidate character string having the maximum character string score may be specified from the character string range.
 このとき、検索管理部は、接頭辞特定部82が特定した接頭辞と接頭辞スコアのペア、および、文字列特定部83が特定した検索対象文字列と文字列スコアのペアを保持するキュー(例えば、優先度付きキュー)を含んでいてもよい。そして、検索管理部は、キューに保持されたペアの中から、接頭辞スコアまたは文字列スコアのうち、最大のスコアの接頭辞または検索対象文字列を特定し、最大のスコアが文字列スコアだった場合、その文字列スコアの検索対象文字列をキューから除外して出力対象と特定し、最大のスコアが接頭辞スコアだった場合、当該接頭辞スコアの接頭辞をキューから除外してもよい。さらに、接頭辞特定部82は、最大のスコアが接頭辞スコアだった場合、キューから除外された接頭辞の次に大きい接頭辞スコアの接頭辞を特定し、文字列特定部83は、最大のスコアが文字列スコアだった場合、キューから除外された検索対象文字列を特定する際に用いられた接頭辞と同じ接頭辞で始まる検索対象文字列の中で、その除外された検索対象文字列の次に大きい文字列スコアを特定し、最大のスコアが接頭辞スコアだった場合、接頭辞特定部82によって特定された接頭辞で始まる検索対象文字列の中から、文字列スコアが最も大きい検索対象文字列を特定してもよい。 At this time, the search management unit holds a pair of prefix and prefix score specified by the prefix specifying unit 82 and a pair of search target character string and character string score specified by the character string specifying unit 83 ( For example, a priority queue may be included. Then, the search management unit identifies the prefix or the search target string with the maximum score from the prefix score or the string score from the pairs held in the queue, and the maximum score is the string score. If the maximum score is a prefix score, the search target character string of the character string score is excluded from the queue and specified as the output target, and the prefix of the prefix score may be excluded from the queue . Furthermore, if the maximum score is a prefix score, the prefix specifying unit 82 specifies a prefix with the next largest prefix score after the prefix excluded from the queue, and the string specifying unit 83 sets the maximum score. If the score is a string score, the excluded search target string in the search target string that starts with the same prefix used to identify the search target string excluded from the queue If the next highest character string score is specified and the largest score is the prefix score, the search with the highest character string score from the search target character strings starting with the prefix specified by the prefix specifying unit 82 The target character string may be specified.
 このように、接頭辞と接頭辞スコアのペア、および、検索対象文字列と文字列スコアのペアの両方のペアを1つのキューに保持することで、キュー内に保持される接頭辞スコアまたは文字列スコアを基に、最も大きいスコアが接頭辞スコアか文字列スコアかを判断できる。この最も大きいスコアに基づいて接頭辞特定部82および文字列特定部83が上述する処理を繰り返すことで、文字列スコアが上位の検索対象文字列を効率的に特定できる。 In this way, the prefix score or character that is retained in the queue by retaining both the prefix-prefix score pair and both the search target string-string score pair pair in one queue. Based on the column score, it can be determined whether the highest score is a prefix score or a string score. Based on the highest score, the prefix specifying unit 82 and the character string specifying unit 83 repeat the above-described processing, whereby the search target character string having the higher character string score can be efficiently specified.
 また、文字列検索装置は、トライ木のデータ構造で表わされた検索候補文字列の集合から生成され、XBWのデータ構造を有する接頭辞の集合(例えば、xbw)と、トライ木のデータ構造から生成されるパトリシア木であって、そのパトリシア木の各ノードに対応する文字列が除外されノード間の親子関係を表わす木構造のみ有するパトリシア木(例えば、パトリシア木T)とを記憶する検索情報記憶部(例えば、検索情報記憶部50)を備えていてもよい。そして、接頭辞特定部82が、XBWのデータ構造を有する接頭辞の集合から、接頭辞スコアが最大の接頭辞の位置を特定し、検索管理部は、特定された接頭辞の位置から、パトリシア木において対応するノードの位置(例えば、u)を特定してもよい。このような構成により、検索に用いる記憶するデータ量を削減できる。 The character string search device generates a set of prefixes (for example, xbw) having a XBW data structure generated from a set of search candidate character strings represented by a trie data structure, and a trie tree data structure. A search for storing a Patricia tree generated from a Patricia tree having only a tree structure that represents a parent-child relationship between the nodes excluding character strings corresponding to each node of the Patricia tree (for example, Patricia tree T c ) An information storage unit (for example, search information storage unit 50) may be provided. Then, the prefix identification unit 82 identifies the position of the prefix having the largest prefix score from the set of prefixes having the XBW data structure, and the search management unit identifies the Patricia from the position of the identified prefix. The position of the corresponding node in the tree (eg, u c ) may be specified. With such a configuration, it is possible to reduce the amount of data to be stored used for search.
 このとき、文字列特定部83は、検索管理部により特定されたノードの位置配下に存在する検索候補文字列の中から文字列スコアが最大の検索候補文字列の位置(例えば、u)を特定し、XBWのデータ構造を有する接頭辞の集合から、特定した位置に対応する検索候補文字列を特定してもよい。 At this time, the character string specifying unit 83 determines the position (for example, u i ) of the search candidate character string having the maximum character string score from the search candidate character strings existing under the position of the node specified by the search management unit. The search candidate character string corresponding to the specified position may be specified from the set of prefixes having the XBW data structure.
 また、接頭辞特定部82は、第一のRMQ構造で表わされた接頭辞と接頭辞スコアとの関係をもとに、その第一のRMQ構造を用いて、特定された接頭辞の集合を範囲検索することにより、最大の接頭辞スコアの接頭辞を特定してもよい。 In addition, the prefix specifying unit 82 uses the first RMQ structure based on the relationship between the prefix represented by the first RMQ structure and the prefix score, and sets the prefixes specified. The prefix of the largest prefix score may be specified by performing a range search.
 また、文字列特定部83は、第二のRMQ構造で表わされた検索候補文字列と文字列スコアとの関係をもとに、その第二のRMQ構造を用いて、特定された接頭辞で始まる検索候補文字列を範囲検索することにより、最大の文字列スコアの検索候補文字列を特定してもよい。 In addition, the character string specifying unit 83 uses the second RMQ structure based on the relationship between the search candidate character string represented by the second RMQ structure and the character string score, and identifies the prefix specified. The search candidate character string having the maximum character string score may be specified by performing a range search on the search candidate character string starting with “.”
 以上、実施形態及び実施例を参照して本願発明を説明したが、本願発明は上記実施形態および実施例に限定されるものではない。本願発明の構成や詳細には、本願発明のスコープ内で当業者が理解し得る様々な変更をすることができる。 As mentioned above, although this invention was demonstrated with reference to embodiment and an Example, this invention is not limited to the said embodiment and Example. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
 この出願は、2013年8月21日に出願された日本特許出願2013-171291を基礎とする優先権を主張し、その開示の全てをここに取り込む。 This application claims priority based on Japanese Patent Application No. 2013-171291 filed on August 21, 2013, the entire disclosure of which is incorporated herein.
 本発明は、入力された文字列に部分一致するキーを検索する文字列検索装置に好適に適用される。本発明による文字列検索装置は、例えば、検索サービスを提供する際に利用可能である。 The present invention is preferably applied to a character string search device that searches for a key that partially matches an input character string. The character string search device according to the present invention can be used, for example, when providing a search service.
 10 入力部
 20 接頭辞集合特定部
 30 検索管理部
 31 接頭辞特定部
 32 文字列特定部
 40 出力部
50 検索情報記憶部
DESCRIPTION OF SYMBOLS 10 Input part 20 Prefix set specific part 30 Search management part 31 Prefix specific part 32 Character string specific part 40 Output part 50 Search information storage part

Claims (11)

  1.  優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列の集合から、入力された文字列を含む検索候補文字列を検索する文字列検索装置であって、
     各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定する接頭辞集合特定部と、
     入力された文字列で終わる接頭辞の集合の中から、前記接頭辞ごとに当該接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコアが最大の接頭辞を特定する接頭辞特定部と、
     特定された接頭辞で始まる検索候補文字列の中から、前記文字列スコアが最大の検索候補文字列を特定する文字列特定部とを備えた
     ことを特徴とする文字列検索装置。
    A character string search device for searching a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched,
    A prefix set specifying unit that specifies a set of prefixes ending with the input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string;
    A prefix defined by the largest string score among the string scores associated with the search candidate string starting with the prefix from the set of prefixes ending with the input string A prefix identification part that identifies the prefix with the highest score;
    A character string search device comprising: a character string specifying unit that specifies a search candidate character string having the maximum character string score from search candidate character strings starting with the specified prefix.
  2.  検索範囲を管理する検索管理部を備え、
     前記検索管理部は、文字列特定部によって特定された検索候補文字列の接頭辞で始まる検索候補文字列の中から、すでに特定された検索候補文字列を除く検索候補文字列の範囲を特定し、接頭辞集合特定部により特定された接頭辞の集合から、接頭辞特定部により特定された接頭辞を除いた接頭辞の範囲を特定し、
     接頭辞特定部は、前記検索管理部により特定された接頭辞の範囲から、接頭辞スコアが最大の接頭辞を特定し、
     文字列特定部は、前記検索管理部により特定された検索候補文字列の範囲から、文字列スコアが最大の検索候補文字列を特定する
     請求項1記載の文字列検索装置。
    It has a search management unit that manages the search range,
    The search management unit specifies a range of search candidate character strings excluding the already specified search candidate character strings from search candidate character strings starting with a prefix of the search candidate character string specified by the character string specifying unit. Identify a range of prefixes from the set of prefixes specified by the prefix set specification part, excluding the prefixes specified by the prefix specification part,
    The prefix identification unit identifies a prefix having the largest prefix score from the range of prefixes identified by the search management unit,
    The character string search device according to claim 1, wherein the character string specifying unit specifies a search candidate character string having a maximum character string score from a range of search candidate character strings specified by the search management unit.
  3.  検索管理部は、接頭辞特定部が特定した接頭辞と接頭辞スコアのペア、および、文字列特定部が特定した検索対象文字列と文字列スコアのペアを保持するキューを含み、
     前記検索管理部は、前記キューに保持されたペアの中から、接頭辞スコアまたは文字列スコアのうち、最大のスコアの接頭辞または検索対象文字列を特定し、最大のスコアが文字列スコアだった場合、当該文字列スコアの検索対象文字列をキューから除外して出力対象と特定し、最大のスコアが接頭辞スコアだった場合、当該接頭辞スコアの接頭辞をキューから除外し、
     接頭辞特定部は、最大のスコアが接頭辞スコアだった場合、キューから除外された接頭辞の次に大きい接頭辞スコアの接頭辞を特定し、
     文字列特定部は、最大のスコアが文字列スコアだった場合、キューから除外された検索対象文字列を特定する際に用いられた接頭辞と同じ接頭辞で始まる検索対象文字列の中で、当該除外された検索対象文字列の次に大きい文字列スコアを特定し、最大のスコアが接頭辞スコアだった場合、前記接頭辞特定部によって特定された接頭辞で始まる検索対象文字列の中から、文字列スコアが最も大きい検索対象文字列を特定する
     請求項2記載の文字列検索装置。
    The search management unit includes a queue that holds a prefix / prefix score pair identified by the prefix identification unit, and a search target string / string score pair identified by the string identification unit,
    The search management unit identifies a prefix or a search target character string having a maximum score among prefix scores or character string scores from the pairs held in the queue, and the maximum score is a character string score. If the maximum score is a prefix score, exclude the prefix of the prefix score from the queue,
    The prefix identification part identifies the prefix with the next highest prefix score after the prefix excluded from the queue if the largest score is the prefix score,
    When the maximum score is a string score, the string specifying unit, in the search target string starting with the same prefix as the prefix used to specify the search target string excluded from the queue, When the next largest string score is specified after the excluded search target character string and the maximum score is a prefix score, the search target character string starting with the prefix specified by the prefix specifying unit is selected. The character string search device according to claim 2, wherein a search target character string having the largest character string score is specified.
  4.  トライ木のデータ構造で表わされた検索候補文字列の集合から生成され、XBWのデータ構造を有する接頭辞の集合と、前記トライ木のデータ構造から生成されるパトリシア木であって、当該パトリシア木の各ノードに対応する文字列が除外されノード間の親子関係を表わす木構造のみ有するパトリシア木とを記憶する検索情報記憶部を備え、
     接頭辞特定部は、前記XBWのデータ構造を有する接頭辞の集合から、接頭辞スコアが最大の接頭辞の位置を特定し、
     検索管理部は、特定された前記接頭辞の位置から、前記パトリシア木において対応するノードの位置を特定する
     請求項1から請求項3のうちのいずれか1項に記載の文字列検索装置。
    A Patricia tree generated from a set of search candidate character strings represented by a trie tree data structure and having an XBW data structure, and a Patricia tree generated from the trie tree data structure, A search information storage unit for storing a Patricia tree having only a tree structure in which a character string corresponding to each node of the tree is excluded and a parent-child relationship between the nodes is included;
    The prefix specifying unit specifies the position of the prefix having the maximum prefix score from the set of prefixes having the XBW data structure,
    The character string search device according to any one of claims 1 to 3, wherein the search management unit specifies a position of a corresponding node in the Patricia tree from the specified position of the prefix.
  5.  文字列特定部は、検索管理部により特定されたノードの位置配下に存在する検索候補文字列の中から文字列スコアが最大の検索候補文字列の位置を特定し、XBWのデータ構造を有する接頭辞の集合から、前記特定した位置に対応する検索候補文字列を特定する
     請求項4記載の文字列検索装置。
    The character string specifying unit specifies the position of the search candidate character string having the maximum character string score from the search candidate character strings existing under the position of the node specified by the search management unit, and has a prefix having an XBW data structure. The character string search device according to claim 4, wherein a search candidate character string corresponding to the specified position is specified from a set of words.
  6.  接頭辞特定部は、第一のRMQ構造で表わされた接頭辞と接頭辞スコアとの関係をもとに、当該第一のRMQ構造を用いて、特定された接頭辞の集合を範囲検索することにより、最大の接頭辞スコアの接頭辞を特定する
     請求項1から請求項5のうちのいずれか1項に記載の文字列検索装置。
    The prefix specifying unit performs a range search on the specified set of prefixes using the first RMQ structure based on the relationship between the prefix represented by the first RMQ structure and the prefix score. The character string search device according to any one of claims 1 to 5, wherein a prefix having a maximum prefix score is specified by
  7.  文字列特定部は、第二のRMQ構造で表わされた検索候補文字列と文字列スコアとの関係をもとに、当該第二のRMQ構造を用いて、特定された接頭辞で始まる検索候補文字列を範囲検索することにより、最大の文字列スコアの検索候補文字列を特定する
     請求項1から請求項6のうちのいずれか1項に記載の文字列検索装置。
    The character string specifying unit uses the second RMQ structure to start a specified prefix based on the relationship between the search candidate character string represented by the second RMQ structure and the character string score. The character string search device according to any one of claims 1 to 6, wherein a search candidate character string having a maximum character string score is specified by performing a range search on the candidate character string.
  8.  優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列の集合から、入力された文字列を含む検索候補文字列を検索する文字列検索方法であって、
     各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定する接頭辞集合特定ステップと、
     入力された文字列で終わる接頭辞の集合の中から、前記接頭辞ごとに当該接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコアが最大の接頭辞を特定する接頭辞特定ステップと、
     特定された接頭辞で始まる検索候補文字列の中から、前記文字列スコアが最大の検索候補文字列を特定する文字列特定ステップとを含む
     ことを特徴とする文字列検索方法。
    A character string search method for searching a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched,
    A prefix set specifying step for specifying a set of prefixes ending with the input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string;
    A prefix defined by the largest string score among the string scores associated with the search candidate string starting with the prefix from the set of prefixes ending with the input string A prefix identification step to identify the prefix with the highest score;
    A character string search method comprising: a character string specifying step of specifying a search candidate character string having the maximum character string score from search candidate character strings starting with the specified prefix.
  9.  検索範囲を管理する検索管理ステップを含み、
     前記検索管理ステップで、文字列特定ステップで特定された検索候補文字列の接頭辞で始まる検索候補文字列の中から、すでに特定された検索候補文字列を除く検索候補文字列の範囲を特定し、接頭辞集合特定ステップで特定された接頭辞の集合から、接頭辞特定ステップで特定された接頭辞を除いた接頭辞の範囲を特定し、
     接頭辞特定ステップで、前記検索管理ステップで特定された接頭辞の範囲から、接頭辞スコアが最大の接頭辞を特定し、
     文字列特定ステップで、前記検索管理ステップで特定された検索候補文字列の範囲から、文字列スコアが最大の検索候補文字列を特定する
     請求項8記載の文字列検索方法。
    Including a search management step for managing the search scope;
    In the search management step, the search candidate character string range excluding the already specified search candidate character string is specified from the search candidate character strings starting with the prefix of the search candidate character string specified in the character string specifying step. Identify the range of prefixes from the set of prefixes identified in the prefix set identification step, excluding the prefix identified in the prefix identification step,
    In the prefix identification step, the prefix with the largest prefix score is identified from the prefix range identified in the search management step,
    The character string search method according to claim 8, wherein in the character string specifying step, a search candidate character string having a maximum character string score is specified from a range of search candidate character strings specified in the search management step.
  10.  優先的に検索すべき度合いを示す文字列スコアが対応づけられた検索候補文字列の集合から、入力された文字列を含む検索候補文字列を検索するコンピュータに適用される文字列検索プログラムであって、
     前記コンピュータに、
     各検索候補文字列の先頭文字から抽出される連続する1文字以上の文字列である接頭辞の集合から、入力された文字列で終わる接頭辞の集合を特定する接頭辞集合特定処理、
     入力された文字列で終わる接頭辞の集合の中から、前記接頭辞ごとに当該接頭辞で始まる検索候補文字列に対応づけられた文字列スコアのうち最も大きい文字列スコアで定義される接頭辞スコアが最大の接頭辞を特定する接頭辞特定処理、および、
     特定された接頭辞で始まる検索候補文字列の中から、前記文字列スコアが最大の検索候補文字列を特定する文字列特定処理
     を実行させるための文字列検索プログラム。
    A character string search program applied to a computer that searches a search candidate character string including an input character string from a set of search candidate character strings associated with a character string score indicating a degree to be preferentially searched. And
    In the computer,
    A prefix set specifying process for specifying a set of prefixes ending with the input character string from a set of prefixes that are one or more consecutive character strings extracted from the first character of each search candidate character string;
    A prefix defined by the largest string score among the string scores associated with the search candidate string starting with the prefix from the set of prefixes ending with the input string A prefix identification process that identifies the prefix with the highest score, and
    A character string search program for executing a character string specifying process for specifying a search candidate character string having the maximum character string score from search candidate character strings that start with the specified prefix.
  11.  コンピュータに、
     検索範囲を管理する検索管理処理を実行させ、
     前記検索管理処理で、文字列特定処理で特定された検索候補文字列の接頭辞で始まる検索候補文字列の中から、すでに特定された検索候補文字列を除く検索候補文字列の範囲を特定させ、接頭辞集合特定処理で特定された接頭辞の集合から、接頭辞特定処理で特定された接頭辞を除いた接頭辞の範囲を特定させ、
     接頭辞特定処理で、前記検索管理処理で特定された接頭辞の範囲から、接頭辞スコアが最大の接頭辞を特定させ、
     文字列特定処理で、前記検索管理処理で特定された検索候補文字列の範囲から、文字列スコアが最大の検索候補文字列を特定させる
     請求項10記載の文字列検索プログラム。
    On the computer,
    Execute search management processing to manage the search range,
    In the search management process, a range of search candidate character strings excluding an already specified search candidate character string is specified from search candidate character strings that start with a prefix of the search candidate character string specified in the character string specifying process. , From the set of prefixes specified by the prefix set specifying process, the range of the prefix excluding the prefix specified by the prefix specifying process is specified,
    In the prefix identification process, the prefix with the largest prefix score is identified from the prefix range identified in the search management process.
    The character string search program according to claim 10, wherein the character string specifying process specifies a search candidate character string having a maximum character string score from a range of search candidate character strings specified by the search management process.
PCT/JP2014/003817 2013-08-21 2014-07-18 Text character string search device, text character string search method, and text character string search program WO2015025467A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP14838200.5A EP3037986A4 (en) 2013-08-21 2014-07-18 Text character string search device, text character string search method, and text character string search program
JP2015532688A JP6072922B2 (en) 2013-08-21 2014-07-18 Character string search device, character string search method, and character string search program
CN201480046496.4A CN105474214A (en) 2013-08-21 2014-07-18 Text character string search device, text character string search method, and text character string search program
US14/909,793 US20160196303A1 (en) 2013-08-21 2014-07-18 String search device, string search method, and string search program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013-171291 2013-08-21
JP2013171291 2013-08-21

Publications (1)

Publication Number Publication Date
WO2015025467A1 true WO2015025467A1 (en) 2015-02-26

Family

ID=52483264

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/003817 WO2015025467A1 (en) 2013-08-21 2014-07-18 Text character string search device, text character string search method, and text character string search program

Country Status (5)

Country Link
US (1) US20160196303A1 (en)
EP (1) EP3037986A4 (en)
JP (1) JP6072922B2 (en)
CN (1) CN105474214A (en)
WO (1) WO2015025467A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9892789B1 (en) 2017-01-16 2018-02-13 International Business Machines Corporation Content addressable memory with match hit quality indication
JP2020098583A (en) * 2017-03-15 2020-06-25 センシェア アーゲー Efficient use of trie data structure in databases

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222238B (en) * 2019-04-30 2022-02-25 上海交通大学 Query method and system for bidirectional mapping of character string and identifier
US11860884B2 (en) * 2021-03-30 2024-01-02 Snap Inc. Search query modification database
CN114065733B (en) * 2021-10-18 2024-07-26 浙江香侬慧语科技有限责任公司 Dependency syntax analysis method, device and medium based on machine reading understanding

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008090606A1 (en) * 2007-01-24 2008-07-31 Fujitsu Limited Information search program, recording medium containing the program, information search device, and information search method
WO2011104754A1 (en) * 2010-02-24 2011-09-01 三菱電機株式会社 Search device and search program

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7941310B2 (en) * 2003-09-09 2011-05-10 International Business Machines Corporation System and method for determining affixes of words
US8156156B2 (en) * 2006-04-06 2012-04-10 Universita Di Pisa Method of structuring and compressing labeled trees of arbitrary degree and shape
CN102084363B (en) * 2008-07-03 2014-11-12 加利福尼亚大学董事会 A method for efficiently supporting interactive, fuzzy search on structured data
CN101916263B (en) * 2010-07-27 2012-10-31 武汉大学 Fuzzy keyword query method and system based on weighing edit distance
US8930391B2 (en) * 2010-12-29 2015-01-06 Microsoft Corporation Progressive spatial searching using augmented structures

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008090606A1 (en) * 2007-01-24 2008-07-31 Fujitsu Limited Information search program, recording medium containing the program, information search device, and information search method
WO2011104754A1 (en) * 2010-02-24 2011-09-01 三菱電機株式会社 Search device and search program

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
BO-JUNE (PAUL) HSU ET AL.: "Space- efficient data structures for Top-k completion", WWW '13 PROCEEDINGS OF THE 22ND INTERNATIONAL CONFERENCE ON WORLD WIDE WEB, 13 May 2013 (2013-05-13), pages 583 - 593, XP058019880, Retrieved from the Internet <URL:http://dl.acm.org/citation.cfm?id=2488440> [retrieved on 20141007] *
BO-JUNE (PAUL) HSU; GIUSEPPE OTTAVIANO: "Space-Efficient Data Structures for Top-k Completion", WWW' 13 PROCEEDINGS OF THE 22ND INTERNATIONAL CONFERENCE ON WORLD WIDE WEB, May 2013 (2013-05-01), pages 583 - 594, XP058019880, DOI: doi:10.1145/2488388.2488440
PAOLO FERRAGINA ET AL.: "Structuring labeled trees for optimal succinctness, and beyond", FOCS '05 PROCEEDINGS OF THE 46TH ANNUAL IEEE SYMPOSIUM ON FOUNDATIONS OF COMPUTER SCIENCE, 2005, pages 184 - 193, XP010851712, Retrieved from the Internet <URL:http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1530713> [retrieved on 20141007] *
PAOLO FERRAGINA; FABRIZIO LUCCIO; GIOVANNI MANZINI; S. MUTHUKRISHNAN: "Structuring labeled trees for optimal succinctness, and beyond", FOCS '05 PROCEEDINGS OF THE 46TH ANNUAL IEEE SYMPOSIUM ON FOUNDATIONS OF COMPUTER SCIENCE, pages 184 - 196
See also references of EP3037986A4
WING-KAI HON; RAHUL SHAH; SHARMA V. THANKACHAN: "Towards an Optimal Space-and-Query-Time Index for Top-k Document Retrieval", CPM' 12 PROCEEDINGS OF THE 23RD ANNUAL CONFERENCE ON COMBINATORIAL PATTERN MATCHING, pages 173 - 184
WING-KAI HON; TSUNG-HAN KU; RAHUL SHAH; SHARMA V. THANKACHAN; JEREY SCOTT VITTER: "Faster compressed dictionary matching", SPIRE'10 PROCEEDINGS OF THE 17TH INTERNATIONAL CONFERENCE ON STRING PROCESSING AND INFORMATION RETRIEVAL, pages 191 - 200

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9892789B1 (en) 2017-01-16 2018-02-13 International Business Machines Corporation Content addressable memory with match hit quality indication
US10347337B2 (en) 2017-01-16 2019-07-09 International Business Machines Corporation Content addressable memory with match hit quality indication
US10614884B2 (en) 2017-01-16 2020-04-07 International Business Machines Corporation Content addressable memory with match hit quality indication
JP2020098583A (en) * 2017-03-15 2020-06-25 センシェア アーゲー Efficient use of trie data structure in databases
US11275740B2 (en) 2017-03-15 2022-03-15 Censhare Gmbh Efficient use of trie data structure in databases
US11347741B2 (en) 2017-03-15 2022-05-31 Censhare Gmbh Efficient use of TRIE data structure in databases
JP7198192B2 (en) 2017-03-15 2022-12-28 センシェア ゲーエムベーハー Efficient Use of Trie Data Structures in Databases
US11899667B2 (en) 2017-03-15 2024-02-13 Censhare Gmbh Efficient use of trie data structure in databases

Also Published As

Publication number Publication date
CN105474214A (en) 2016-04-06
JP6072922B2 (en) 2017-02-01
EP3037986A4 (en) 2017-01-04
US20160196303A1 (en) 2016-07-07
JPWO2015025467A1 (en) 2017-03-02
EP3037986A1 (en) 2016-06-29

Similar Documents

Publication Publication Date Title
US7912818B2 (en) Web graph compression through scalable pattern mining
JP6072922B2 (en) Character string search device, character string search method, and character string search program
CN111159990B (en) Method and system for identifying general special words based on pattern expansion
CN109902142B (en) Character string fuzzy matching and query method based on edit distance
CN105843882A (en) Information matching method and apparatus
US20050120017A1 (en) Efficient retrieval of variable-length character string data
CN105426412A (en) Multi-mode string matching method and device
CN104268176A (en) Recommendation method and system based on search keyword
CN104021202B (en) The entry processing unit and method of a kind of knowledge sharing platform
JP5980520B2 (en) Method and apparatus for efficiently processing a query
CN105608201A (en) Text matching method supporting multi-keyword expression
US9529835B2 (en) Online compression for limited sequence length radix tree
CN113065419B (en) Pattern matching algorithm and system based on flow high-frequency content
KR101089722B1 (en) Method and apparatus for prefix tree based indexing, and recording medium thereof
CN111814009B (en) Mode matching method based on search engine retrieval information
CN101576877A (en) Fast word segmentation realization method
JP5190192B2 (en) SEARCH DEVICE, SEARCH METHOD, AND PROGRAM
JP2000194713A (en) Method and device for retrieving character string, and storage medium stored with character string retrieval program
JP2011138365A (en) Term extraction device, method, and data structure of term dictionary
JP5628365B2 (en) Search device
CN113641783B (en) Content block retrieval method, device, equipment and medium based on key sentences
CN118193543B (en) Method for searching node tree based on EDA, electronic equipment and storage medium
KR101080898B1 (en) Method and apparatus for indexing character string
JP5237400B2 (en) Search device
CN111061884A (en) Method for constructing K12 education knowledge graph based on DeepDive technology

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201480046496.4

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14838200

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2015532688

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 14909793

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2014838200

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE