WO2012049883A1 - データ構造、インデックス作成装置、データ検索装置、インデックス作成方法、データ検索方法およびコンピュータ読み取り可能な記録媒体 - Google Patents
データ構造、インデックス作成装置、データ検索装置、インデックス作成方法、データ検索方法およびコンピュータ読み取り可能な記録媒体 Download PDFInfo
- Publication number
- WO2012049883A1 WO2012049883A1 PCT/JP2011/063792 JP2011063792W WO2012049883A1 WO 2012049883 A1 WO2012049883 A1 WO 2012049883A1 JP 2011063792 W JP2011063792 W JP 2011063792W WO 2012049883 A1 WO2012049883 A1 WO 2012049883A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- node
- list
- ancestor
- data
- transposed
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/319—Inverted lists
Definitions
- the present invention relates to a data structure, an index creation device, a data retrieval device, an index creation method, a data retrieval method, and a computer-readable recording medium, and in particular, a data structure using an inverted index, an index creation device, a data retrieval device, and an index creation.
- the present invention relates to a method, a data search method, and a computer-readable recording medium.
- Patent Document 1 discloses the following configuration. That is, the computerized document search system includes an index storage means for storing an index word, a document frequency and a document identifier of a registered document including the index word, and an in-document frequency and an appearance position in each registered document of the index word, and registration A document dividing unit that divides a document into index words that are chains of n (n is an integer of 1 or more) characters; a search word dividing unit that divides a document into index words that are one or more n character chains covering the search terms; When a search term is divided into two or more index terms, search condition analysis means for generating a search condition tree synthesized by a position operator that specifies the distance between the appearance positions of a plurality of index terms, And search condition evaluation means for executing a search result synthesis process and obtaining a search result.
- search condition analysis means for generating a search condition tree synthesized by a position operator that specifies the distance between the appearance positions of a plurality of index terms
- search condition evaluation means
- Patent Document 2 discloses the following method. That is, when compressing a document identification number into a byte string using the variable byte method, w bits in the byte string are used to express the number of occurrences of the index word in the document, and x bits are attribute information of posting. Used to express The number of occurrences that cannot be expressed in w bits is written after a special value indicating that it is a numerical value that cannot be expressed in w bits.
- x and w are integers given as parameters. Further, a means for reading the compressed posting even from a position in the middle of the transposed list is provided, and a binary search on the transposed list is possible.
- Non-Patent Document 1 describes techniques for searching electronic documents using inverted indexes.
- inverted indexes are Zobel, Justin and Moffat, Alistair “Inverted Files for Text Search Engines”, ACM Computing Surveys (New York: Association for Computing Machinery), pp.8-9 pp. .19-23 Vol.38 No.2 Article 6, July 2006 (Non-Patent Document 1)
- Patent Document 3 an example of a data compression technique in a tree structure is disclosed in Japanese translations of PCT publication No. 2003-501749 (Patent Document 3). That is, the memory is implemented as a directory structure consisting of a tree-shaped hierarchy with nodes at a number of different hierarchy levels. In this directory structure, a pointer is first added to a node whose table contains a given first number of elements and whose width is compressed. In order to maximize the performance of the functional tree structure, the addition of pointers pointing to individual width-compressed nodes is allowed until the number of pointers in the node corresponds to a predetermined threshold value less than the first number. It is. A width-compressed node is converted into a cluster of nodes formed by a parent node and individual child nodes as soon as the number of pointers accepted by the width-compressed node exceeds the threshold value.
- a data set (hereinafter also referred to as taxonomy) of a directed acyclic graph (DAG) composed of a plurality of tags and a plurality of semantic classes may be used. It is done.
- DAG directed acyclic graph
- FIG. 31 is a diagram specifically showing an example of the taxonomy and search target data in the index creation device and data search device according to the first embodiment of the present invention.
- the taxonomy is a directed acyclic graph composed of a plurality of tags and a plurality of semantic classes.
- a tag is a leaf node in the graph and is information having at least a label.
- a label is a character string that represents a tag. In FIG. 31, the tag is represented by a rectangle, and the label of each tag is represented as a character string surrounded by ““ ””.
- the semantic class is information expressing the concept of collecting tags, and has a label or identifier that is a character string.
- each semantic class is represented by an ellipse, and the label of each semantic class is described within the ellipse.
- a tag or semantic class having the label “X” may be expressed as “X node”.
- Semantic inclusion relationships exist between semantic classes and tags in the taxonomy.
- this relationship is indicated by an arrow.
- entity node is a high-level concept of “electrical appliance manufacturer node”
- electric appliance manufacturer node is a high-level concept of “Yamamoto electric node”.
- this inclusion relationship includes synonymous relationships.
- a company “Yamamoto Electric” is sometimes abbreviated as “yamaden”.
- the “Yamamoto Electric Node” can also be regarded as a concept higher than the “yamaden node”.
- the search target data is data associated with the tag in the taxonomy, and has a unique identifier that can be distinguished from other search target data.
- each article has an identifier represented by an integer, and has a word appearing in each article as a tag.
- FIG. 31 shows seven search target data and an identifier corresponding to each search target data.
- the search target data with the identifier “001” includes data “Yamamoto Electric” and has a relationship with “Yamamoto Electric Node” in the taxonomy.
- the data search device When information indicating a certain semantic class or tag is input, the data search device according to the first exemplary embodiment of the present invention indicates an identifier indicating a set of data that can be reached from a node in the taxonomy specified by the information Returns a list of.
- the data search device when the tag “yamaden” is input to the data search device, the data search device returns two identifiers 003 and 004 as search targets.
- the data search device returns four identifiers 004, 005, 006, 007 as search targets.
- FIG. 32 is a diagram conceptually illustrating an example of a taxonomy and search target data in the index creation device and the data search device according to the first embodiment of the present invention.
- FIG. 32 shows an example of a taxonomy, where an ellipse represents a semantic class and a rectangle represents a tag.
- D (X) connected from each tag represents a set of identifiers of search target data that can be reached from the node X.
- d (X) corresponding to the semantic class X becomes a Union of d (y) (y (Y) of each tag in the set Y of tags reachable from the semantic class X.
- a method A using a transposed index of only a tag there are two types, that is, a method B using a transposed index corresponding to all nodes.
- Method A in order to refer to the transposed list corresponding to the upper semantic class, the processing time for calculating the set of reachable tags and the transposed list corresponding to the set of tags obtained from the calculation result And a processing time for calculating a union (union) of the data set. For this reason, when the method A is adopted, when the size of the taxonomy and the data set is increased, these two processing times are increased.
- the processing can be completed only by reading the permutation list corresponding to the search, but the amount of data that must be stored increases overwhelmingly.
- the inquiry speed may be reduced.
- Patent Documents 1 to 3 and Non-Patent Document 1 do not disclose a configuration for solving the above-described problems in electronic document search using a taxonomy.
- the present invention has been made to solve the above-described problems, and its object is to provide a data structure, an index creation device, and data capable of reducing the data capacity for search processing and speeding up the search processing.
- a search device, an index creation method, a data search method, and a computer-readable recording medium are provided.
- a data structure is a data structure for extracting a set of search target data reachable from each node in the taxonomy in a taxonomy having a tag related to the search target data.
- the taxonomy includes ancestor reference data indicating an ancestor node that is an upper node of each node, and a transposed list of each node.
- the ancestor node is included in the ancestor reference data.
- the registered node transpose list is a list of integer values indicating positions in the transpose list corresponding to the registered ancestor node, and the list of integer values in the transpose list of each node is a plurality of blocks. The difference between the integer value in the block and the integer value immediately before the integer value There and a converted inverted list data into a bit string of variable length integer codes.
- an index creation device is a transposition used to extract a set of search target data reachable from each node in the taxonomy in a taxonomy having a tag related to the search target data.
- An index creation device for creating a list, wherein an integer value list in a transposed list of each node is divided into a plurality of blocks, and a difference value between an integer value in the block and an integer value immediately before the integer value Is converted to a bit string of a variable-length integer code, and for each node in the taxonomy, one ancestor node that is an upper node of the node is selected, and ancestor reference data indicating the selected ancestor node is generated.
- An ancestor node search unit for generating an ancestor node list indicating one or a plurality of ancestor nodes of the tag in Mi and an identifier of the search target data, and the highest node among the nodes in the ancestor node list Is added as an element of the corresponding transposition list, and for the nodes other than the highest-level node, as an element of the corresponding transposition list, instead of the identifier, corresponds to a node immediately above the node.
- an ancestor number conversion unit for adding an integer value indicating the position in the transposed list.
- a data search device is a taxonomy having a tag related to search target data, and extracts a set of search target data that can be reached from a specified node in the taxonomy.
- Including a transposed list of each node in the taxonomy, and the transposed list of the highest node among the nodes is a list of integer values indicating identifiers of the search target data
- the transposed list of nodes other than the highest node is a list of integer values indicating positions in the transposed list corresponding to the node immediately above the node, instead of the identifier, and in the transposed list of each node.
- the list of integer values is divided into multiple blocks, and the integer value in the block and the integer
- An ancestor number transposition list storage unit for storing transpose list data in which a difference value from an integer value immediately before is converted into a bit string of a variable-length integer code, and the permutation list Processing for extracting the integer value of the transposed list corresponding to the upper node of the designated node corresponding to the position indicated by the integer value of the transposed list corresponding to the designated node based on the data for the top node
- An identifier conversion unit for creating a list of identifiers of the search target data corresponding to the designated node by repeating until the identifier of the corresponding transposition list is extracted.
- an index creation method is a transposition used to extract a set of search target data reachable from each node in the taxonomy in a taxonomy having a tag related to the search target data.
- An index creation method for creating a list wherein the list of integer values in the transposed list of each node is divided into a plurality of blocks, and the difference value between the integer value in the block and the integer value immediately before the integer value is variable Selecting one ancestor node that is an upper node of the node for each node in the taxonomy, and generating ancestor reference data indicating the selected ancestor node; Based on the ancestor reference data, one of the tags in the taxonomy Or generating an ancestor node list indicating a plurality of ancestor nodes, and receiving the identifier of the search target data, and for each of the top nodes in the ancestor node list, the identifier is transposed correspondingly.
- a data search method is a taxonomy having a tag related to search target data, wherein data for extracting a set of search target data reachable from a specified node specified in the taxonomy
- a search method includes a transposed list of each node in the taxonomy, and the transposed list of the highest node among the nodes is a list of integer values indicating identifiers of the search target data, and the highest
- the transposed list of nodes other than the node is a list of integer values indicating positions in the transposed list corresponding to the node immediately above the node instead of the identifier, and further, the integer values in the transposed list of each node.
- a computer-readable recording medium is a taxonomy having a tag related to search target data, and extracts a set of search target data that can be reached from each node in the taxonomy.
- ancestor reference data indicating an ancestor node
- generating an ancestor node list indicating one or more ancestor nodes of the tag in the taxonomy based on the ancestor reference data
- the identifier is added as an element of the corresponding transposition list for the highest node among the nodes in the ancestor node list, and the nodes of the corresponding transposition list are added for nodes other than the top node.
- a computer-readable recording medium provides a data search program for extracting a set of search target data that can be reached from a specified node in the taxonomy in a taxonomy having a tag related to the search target data.
- a recorded computer-readable recording medium wherein the data search program includes a transposed list of each node in the taxonomy in the computer, and the transposed list of the highest node among the nodes is the search It is a list of integer values indicating the identifier of the target data, and the transposed list of nodes other than the top node is an integer value indicating the position in the transposed list corresponding to the node immediately above the node, instead of the identifier And the transposed squirrel of each node above A step of obtaining transposed list data in which a list of integer values in is divided into a plurality of blocks, and a difference value between the integer value in the block and the integer value immediately before the integer value is converted into a bit string of a variable-length integer code And receiving the information indicating the designated node, and based on the transposed list data, the position corresponding to the upper node of the designated node corresponding to the position indicated by the integer value of the transposed list corresponding to the designated node.
- FIG. 1A shows an example of a data structure using an inverted index.
- FIG. 1B shows another example of a data structure using an inverted index.
- It is a schematic block diagram of the information processing apparatus which concerns on the 1st Embodiment of this invention.
- It is a block diagram which shows the control structure which the information processing apparatus which concerns on the 1st Embodiment of this invention provides.
- FIG. 1 It is a figure which shows an example of the ancestor number transposition list memorize
- FIG. 1 It is a block diagram which shows the control structure which the information processing apparatus which concerns on the 2nd Embodiment of this invention provides.
- FIG. 6 is a diagram illustrating an example of information stored in a child node storage unit 21.
- FIG. It is a flowchart which shows the process sequence of the recursive function list. It is a flowchart which shows the process sequence of function listJoin. It is a figure which shows an example of a taxonomy and search object data concretely. It is a figure which shows an example of a taxonomy and search object data notionally.
- the data search apparatus provides a function of inputting X indicating a node in a taxonomy and returning d (X).
- a method for realizing such a function it is conceivable to use an inverted index.
- FIG. 1 (a) shows an example of a data structure using an inverted index.
- FIG. 1B shows another example of a data structure using an inverted index.
- Fig. 1 (a) shows the transposed index created for the tag in the taxonomy.
- each row represents a transposed list of each tag.
- the tag L is related to the search target data Nos. 1, 456, and 12000
- the tag M is related to the search target data Nos. 23 and 16333.
- the data search device When the tag is given as input, the data search device using this inverted index returns the data of the row corresponding to the tag among the rows in the inverted index. Also, when a semantic class is given as an input, the data search apparatus examines a set of reachable tags in the taxonomy, extracts d (X) corresponding to each tag, and extracts the d (X) of the extracted d (X). Calculate and return Union.
- FIG. 1 (b) shows transposed indexes created for all nodes in the taxonomy.
- each row represents a transposed list of each node.
- semantic class B is related to search target data No. 1, No. 23, No. 456, No. 12000 and No. 16333.
- the data search device using this transposed index takes out the data of the row corresponding to the node X out of each row in the transposed index.
- the transposed index shown in FIG. 1 is a list of integer values corresponding to each key.
- FIG. 2 is a diagram showing an example of a transposed list compression method in the index creation device and data search device according to the first embodiment of the present invention.
- FIG. 2 shows an example of a method of compressing a list of 10 integer values “1, 2, 5, 7, 8, 11, 15, 16, 18, 19”.
- the first numerical value is left as it is, and the subsequent numerical value is replaced with the difference from the previous numerical value.
- variable-length integer code is an integer sequence encoding method in which a short integer is expressed with a short bit length and a long integer is expressed with a long bit length.
- An example of a variable length integer code is described in Non-Patent Document 1.
- FIG. 3 is a diagram illustrating an example of a variable-length integer code in the index creation device and the data search device according to the first embodiment of the present invention.
- FIG. 3 shows an example of a ⁇ code as an example of a variable length integer code.
- the integer 1 is represented by 0 and the integer 2 is represented by 100.
- the reason why the compression method as shown in FIG. 2 is effective is as follows. That is, first, the integer value of the transposed list is reduced by expressing the integer value as a difference. The data length is shortened by using the variable length integer code. As a result, the transposed list can be compressed.
- An information processing apparatus including an index creation apparatus and a data search apparatus typically has a basic structure of a computer having a general-purpose architecture, and executes a preinstalled program By doing so, various functions as described later are provided.
- a program is stored in a recording medium such as a flexible disk and a CD-ROM (Compact Disk Read Only Memory) or distributed via a network or the like.
- the program according to the first embodiment of the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself according to the first embodiment of the present invention does not include a module included in another program as described above, and the process is executed in cooperation with the other program. Is done. In other words, the program according to the first embodiment of the present invention may be a form incorporated in such another program.
- program execution may be implemented as a dedicated hardware circuit.
- FIG. 4 is a schematic configuration diagram of the information processing apparatus according to the first embodiment of the present invention.
- an information processing apparatus 201 includes a CPU (Central Processing Unit) 101 that is an arithmetic processing unit, a main memory 102, a hard disk 103, an input interface 104, a display controller 105, and a data reader / writer. 106 and a communication interface 107. These units are connected to each other via a bus 121 so that data communication is possible.
- the CPU 101 performs various operations by developing programs (codes) stored in the hard disk 103 in the main memory 102 and executing them in a predetermined order.
- the main memory 102 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory), and stores data indicating various arithmetic processing results in addition to the program read from the hard disk 103. To do.
- the hard disk 103 is a non-volatile magnetic storage device, and stores various setting values in addition to programs executed by the CPU 101.
- the program installed in the hard disk 103 is distributed in a state of being stored in the recording medium 111 as will be described later.
- a semiconductor storage device such as a flash memory may be employed.
- the input interface 104 mediates data transmission between the CPU 101 and an input unit such as a keyboard 108, a mouse 109, and a touch panel (not shown). That is, the input interface 104 receives an external input such as an operation command given by the user operating the input unit.
- an input unit such as a keyboard 108, a mouse 109, and a touch panel (not shown). That is, the input interface 104 receives an external input such as an operation command given by the user operating the input unit.
- the display controller 105 is connected to a display 110 that is a typical example of a display unit, and controls display on the display 110. That is, the display controller 105 displays the result of image processing by the CPU 101 to the user.
- the display 110 is, for example, an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube).
- the data reader / writer 106 mediates data transmission between the CPU 101 and the recording medium 111. That is, the recording medium 111 is distributed in a state where a program executed by the information processing apparatus 201 is stored, and the data reader / writer 106 reads the program from the recording medium 111. Further, the data reader / writer 106 writes the processing result in the information processing apparatus 201 into the recording medium 111 in response to the internal command of the CPU 101.
- the recording medium 111 may be, for example, a general-purpose semiconductor storage device such as CF (Compact Flash) and SD (Secure Digital), a magnetic storage medium such as a flexible disk, or a CD-ROM (Compact Disk Read Only). Memory).
- the communication interface 107 mediates data transmission between the CPU 101 and other personal computers and server devices.
- the communication interface 107 typically has an Ethernet (registered trademark) or USB (Universal Serial Bus) communication function.
- Ethernet registered trademark
- USB Universal Serial Bus
- the information processing apparatus 201 may be connected to another output device such as a printer as necessary.
- FIG. 5 is a block diagram showing a control structure provided by the information processing apparatus according to the first embodiment of the present invention.
- Each block in the CPU 101 shown in FIG. 5 is provided by developing a program (code) stored in the hard disk 103 in the main memory 102 and causing the CPU 101 to execute it.
- a program code
- part or all of the control structure of the information processing apparatus 201 shown in FIG. 5 may be realized by dedicated hardware and / or a wiring circuit.
- information processing apparatus 201 has, as its control structure, ancestor node determination unit 11, ancestor node search unit 13, identifier assignment unit 14, ancestor number conversion unit 15, and ancestor frequency conversion unit 17. And an identifier conversion unit 18.
- the information processing apparatus 201 includes an ancestor node storage unit 12, an ancestor number transposition list storage unit 16, and a frequency list storage unit 19. These storage units correspond to the main memory 102 or the hard disk 103 shown in FIG.
- the CPU 101 is generally composed of many units in addition to the components shown in FIG. 5, but other units are not shown in order to simplify the description.
- the ancestor node determination unit 11 receives the taxonomy and the frequency information indicating the frequency distribution in the search target data set of each node in the taxonomy, and determines an ancestor node to be registered in the ancestor node storage unit 12 for each node. To do.
- the ancestor node storage unit 12 stores one ancestor node of each node determined by the ancestor node determination unit 11.
- the ancestor node search unit 13 receives a certain tag, refers to the ancestor node storage unit 12, and extracts all ancestor nodes that are higher-order nodes of the tag.
- the identifier assigning unit 14 receives search target data including one or more tags and the frequency of each tag, assigns an identifier to the search target data, and outputs the data to the ancestor number conversion unit 15.
- the ancestor number transposed list storage unit 16 holds a transposed list for each label indicating a node in the taxonomy.
- the ancestor number conversion unit 15 receives the search target data to which the identifier is assigned from the identifier addition unit 14 and extracts a set of ancestor nodes using the ancestor node search unit 13 for each tag in the search target data. Then, the ancestor number conversion unit 15 organizes a set of ancestor nodes corresponding to each tag as a trie structure, and outputs the identifier, the trie structure, and the frequency for each tag to the ancestor frequency conversion unit 17.
- the ancestor number conversion unit 15 adds an identifier to the transposition list corresponding to the highest node among the set of ancestor nodes in the ancestor number transposition list storage unit 16, and remembers the added position. The position for the ancestor node that was remembered immediately before is added to the transposition list for the node that is the next higher node. Then, the ancestor number conversion unit 15 further repeats such processing for the transposed list of the lower nodes.
- the ancestor frequency conversion unit 17 receives the identifier, the trie structure in which the ancestor nodes of each tag are gathered, and the frequency for each tag from the ancestor number conversion unit 15, and calculates the frequency corresponding to each ancestor node. After adding the frequency to the transposition list corresponding to the highest node among the set of ancestor nodes in the list storage unit 19, remember the added frequency, and in the transposition list for the node next to the node The frequency is compared with the frequency memorized immediately before. If they are equal, “0” is added, and if not, the frequency in the permutation list is added as it is. Then, the ancestor frequency conversion unit 17 repeats such processing for the transposed list of the lower nodes.
- the identifier conversion unit 18 receives a label indicating the node in the taxonomy from the outside, takes out a set of ancestor nodes of the node using the ancestor node search unit 13, and then transposes the list corresponding to the node indicated by the label Is extracted from the ancestor number transposition list storage unit 16. Then, the identifier conversion unit 18 refers to the position in the ancestor number transposition list storage unit 16 indicated by each integer value in the transposition list and the position in the frequency list storage unit 19 to set the pair of the identifier and the corresponding frequency. Create and rearrange the created identifiers in order of frequency and output.
- FIG. 6 is a diagram showing an example of ancestor reference data stored in the ancestor node storage unit in the information processing apparatus according to the first embodiment of the present invention.
- the ancestor node storage unit 12 holds information indicating the ancestor node of each node in the taxonomy.
- FIG. 6 shows an example of ancestor reference data corresponding to the taxonomy shown in FIG.
- each row corresponds to each node, and the ancestor node of each node is indicated.
- “ ⁇ ” is used as a symbol indicating empty. For example, there is no ancestor node for semantic class A, and the ancestor node for semantic class B is A.
- the ancestor reference data may indicate “ ⁇ ” even when an ancestor node actually exists in the taxonomy. For example, in the taxonomy shown in FIG. 2, the node A exists as a parent node for the node C, but the ancestor reference data shown in FIG. 6 indicates “ ⁇ ” having no ancestor node.
- the ancestor node determination unit 11 determines which of the ancestor nodes is selected for each node and whether the ancestor node is “none”. This process will be described later.
- FIG. 7 is a diagram showing an example of the ancestor number transposition list stored in the ancestor number transposition list storage unit in the information processing apparatus according to the first embodiment of the present invention.
- ancestor number transposition list storage unit 16 essentially stores information indicating a column of identifiers of search target data corresponding to each node.
- the ancestor number transposed list storage unit 16 does not hold the identifier columns corresponding to all the nodes as the transposed list as they are, but the ancestor reference data in the ancestor reference data are registered as ancestor nodes. Holds the position in the corresponding transpose list.
- transposition list d (B) corresponding to the node B five integer values “1, 2, 3, 6, 7” are described. These mean the positions in the transposed list corresponding to the ancestor node A of the node B.
- 1 is an identifier 1 which is the first value in the list corresponding to the node A
- 2 is an identifier 23 which is the second value in the list corresponding to A
- 3 Means the identifier identifier 456 which is the third value in the list corresponding to the node A.
- FIG. 7 is a diagram showing the information held in the ancestor number transposition list storage unit 16 so that it can be easily understood.
- the ancestor number transposition list storage unit 16 holds the data shown in FIG. 7 as it is. Not to do.
- the ancestor number transposed list storage unit 16 holds, for each transposed list, the data compressed using the compression method shown in FIG. 2 and the length of the transposed list.
- FIG. 8 is a diagram showing an example of a frequency list stored in the frequency list storage unit in the information processing apparatus according to the first embodiment of the present invention.
- the frequency list storage unit 19 stores the frequency corresponding to the search target data number for each node in the transposed list stored in the ancestor number transposed list storage unit 16.
- the frequency list corresponding to node C has a value of “2, 3, 5”.
- the semantic class C has a frequency 2 in the search target data 2451, a frequency 3 in the search target data 3443, and a frequency 5 in the search target data 18921.
- the value “0” exists, but as described above, this does not represent the frequency “0”, but means that it is equal to the frequency of the ancestor node shown in FIG. To do.
- the frequency value at the top of the frequency list of semantic class B is “0”
- the frequency in the search target data No. 1 of semantic class B is the value “4” at the top of the frequency list of ancestor node A. It is.
- the operation of the information processing apparatus according to the first embodiment of the present invention will be described with reference to the drawings.
- the index creation method and the data search method according to the first embodiment of the present invention are performed by operating the information processing apparatus. Therefore, the description of the index creation method and the data search method according to the first embodiment of the present invention is replaced with the following description of the operation of the information processing apparatus 201. In the following description, FIG. 5 will be referred to as appropriate.
- the processing of the information processing apparatus according to the first embodiment of the present invention is largely composed of three processes: a taxonomy registration process, a data registration process, and a data search process.
- FIG. 9 is a flowchart showing an operation procedure when the information processing apparatus according to the first embodiment of the present invention performs a taxonomy registration process.
- the information processing apparatus 201 starts the taxonomy registration process.
- the data distribution is a predicted value of the number of data related to each node in the taxonomy, and in the following description, the predicted value of the number of data corresponding to the node X is freq (X).
- the ancestor node determination unit 11 first performs the following processing in order for each node.
- a list of ancestor nodes is collected by tracing the taxonomy for the node X (RT01).
- the ancestor node collection process can be realized by a depth-first search or a breadth-first search.
- the ancestor node determination unit 11 selects an ancestor node candidate to be registered in the ancestor node storage unit 12 from the list of ancestor nodes (RT02). In this process, the ancestor node determination unit 11 performs the following operation on each ancestor node Y of the node X.
- csomp (Y, X) entopy (All, freq (X)) ⁇ entopy (freq (Y), freq (X))
- the entity (All, freq (X)) is the data length when the node X is expressed as a normal transposed list
- the entity (freq (Y), freq (X)) is the node X of the ancestor node Y. This means the data length when expressed with reference to the transposed list.
- comp (Y, X) represents a data length that can be compressed when node Y is selected as the ancestor node of X.
- the ancestor node determination unit 11 selects ancestor node candidates by excluding ancestor nodes whose comp (Y, X) is smaller than the threshold ⁇ .
- the ancestor node determination unit 11 scores the list of ancestor nodes and determines one ancestor node having the maximum score ( RT03).
- AncScore is defined as a specific example of scoring.
- a score AnScore corresponding to the ancestor node Y of the node X is calculated by the following equation.
- AncScore (Y, X) ⁇ ⁇ dis (X, Y) + ⁇ ⁇ 1 / entopy (freq (Y), freq (X))
- ⁇ and ⁇ are constants, which mean the degree of emphasizing the search speed and the degree of emphasizing the data amount, respectively.
- dis (X, Y) means the number of edges to be traced to reach the node X from the node Y. In other words, the higher dis (X, Y), the higher the ancestor node can be selected, and the search cost is reduced.
- the ancestor node determination unit 11 extracts the node Y having the highest score AnScore (Y, X) from each ancestor node candidate, and inserts the pair of the node X and the ancestor node Y into the ancestor node storage unit 12.
- the ancestor node determination unit 11 stores the nodes X and ⁇ in the ancestor node storage unit 12 assuming that no ancestor node exists (RT04). .
- the ancestor node is selected from the search speed and the amount of data, but if the user's inquiry frequency for each node is known, it is replaced with another criterion such as selecting an ancestor node with a high inquiry frequency. Alternatively, another standard may be added.
- FIG. 10 is a flowchart showing an operation procedure when the information processing apparatus according to the first embodiment of the present invention performs a data registration process.
- the information processing apparatus 201 starts a data registration process when one or more tags and data having the frequency of the tags are input to the identifier assigning unit 14.
- the identifier assigning unit 14 has a counter inside, adds 1 to the counter each time data is input, and assigns the value as an identifier to the data (RD01). Then, the identifier assigning unit 14 outputs the identifier, one or more tags, and the frequency of each tag to the ancestor number conversion unit 15.
- the ancestor number conversion unit 15 performs the following operation for each tag received from the identifier assigning unit 14. That is, the ancestor number conversion unit 15 first passes the tag X to the ancestor node search unit 13 and extracts a set of ancestor nodes of the tag X from the ancestor node search unit 13 (RD02).
- FIG. 11 is a flowchart showing an operation procedure when the ancestor node search unit performs an ancestor node set calculation process.
- ancestor node search unit 13 first assigns tag X to temporary variable node, and initializes output list ancList in an empty state (RD021).
- the ancestor node search unit 13 refers to the ancestor node storage unit 12 to retrieve the ancestor node of the node, and anc.
- the ancestor node search unit 13 uses node.
- anc ⁇ (true in RD022)
- node is added to ancList and ancList is output (RD023).
- the ancestor node search unit 13 can take out the designated tag and all ancestor nodes above the tag.
- the ancestor node search unit 13 sets 3 as ⁇ L, B, A ⁇ .
- One node is returned to the ancestor number conversion unit 15.
- the ancestor number conversion unit 15 examines all the ancestor node lists of the respective tags, and constructs a trie tree in which common ancestor nodes are collected, that is, the above-described trie structure (RD03).
- FIG. 12 is a diagram illustrating an example of a trie tree constructed by the ancestor number conversion unit.
- this trie tree generates a list ⁇ L, B, A ⁇ , ⁇ N, B, A ⁇ , ⁇ O, C ⁇ of ancestor nodes corresponding to three tags L, N, and O, respectively. It is a tri-tree.
- the element of this trie tree is a label of each ancestor node, and is a form in which common ancestors in the three ancestor node lists are collected.
- FIG. 13 is a flowchart showing the processing procedure of the recursive function insertTrie.
- InsertTrie first checks the contents of the given list (RD031), and if the list is empty (true in RD031), does nothing and ends the process (RD036).
- RD031 if there is no child element having a label (true in RD033), a cnode having the label as an element is created, cnode is added as a child element of the element tnode (RD034), and cnode and list are used as arguments to RD031 to The same process as shown in RD034 is performed (RD035).
- FIG. 14 is a flowchart showing an operation procedure when the ancestor number conversion unit performs a trie tree construction process.
- the ancestor number conversion unit 15 first creates a root element root of a trie tree (RD041).
- the ancestor number conversion unit 15 calls the recursive function insertTrie for the root element root and the ancestor node list list of each tag (RD042).
- each ancestor node list is added as a descendant of the root element root, and a trie tree as shown in FIG. 12 can be constructed.
- the ancestor number conversion unit 15 performs the update process of the transposed list in the ancestor number transposed list storage unit 16 based on the trie tree and the identifier (RD04).
- FIG. 15 is a flowchart showing the processing procedure of the recursive function insert.
- the function insert has two arguments, a node meaning an element in the trie tree and an integer value number.
- the number is added to the end of the transposed list, and the length of the transposed list at the time of addition is added. That is, an additional position of number is taken out and set as pos (RDI01).
- the function process insert (child, pos) for adding pos is called for each child node, that is, child in the node trie tree.
- the information pos of the addition position of the current element specified by the node is added to the transposed list of each child element (RDI02).
- the ancestor number conversion unit 15 calls insert (rchild, identifier) for each child element rchild of the root in the trie tree, thereby transposing the list in the ancestor number transposition list storage unit 16. Is updated (RD04).
- the ancestor number conversion unit 15 outputs the trie tree, the identifier, and the frequency of each tag to the ancestor frequency conversion unit 17 in parallel with the transposed list update process (RD04).
- the ancestor frequency conversion unit 17 calculates the frequency of the ancestor node based on the trie tree received from the ancestor number conversion unit 15 and the frequency of each tag (RD05).
- FIG. 16 is a diagram illustrating a calculation result of the frequency in the trie tree performed by the ancestor frequency conversion unit.
- the ancestor frequency conversion unit 17 calculates the frequency of the ancestor node by calculating the sum of the frequencies of the child nodes of each node while tracing in the parent direction from the leaf node of the trie tree.
- the ancestor frequency conversion unit 17 uses the frequency 5 of the tag B, the tag A frequency 5 and tag C frequency 1 are calculated.
- the ancestor frequency conversion unit 17 updates the frequency list (RD06).
- FIG. 17 is a flowchart showing the processing procedure of the recursive function insertFreq.
- the recursive function insertFreq has two arguments, a node that means a node in the trie tree and an integer value freq that indicates the frequency.
- the node frequency freq and the child node frequency child are added as a frequency by calling insertFreq (child, 0).
- the frequency of the child node is added as it is by calling insertFreq (child, child.freq) (RDIF02).
- the ancestor frequency conversion unit 17 updates the frequency list by calling insert (rchild, rchchild.freq) for each child node rchid of root in the trie tree (RD06).
- FIG. 18 is a flowchart showing an operation procedure when the information processing apparatus according to the first embodiment of the present invention performs a data search process.
- the information processing apparatus 201 starts a data search process.
- the identifier conversion unit 18 uses the ancestor node search unit 13 to obtain a list of ancestor nodes corresponding to the designated node indicated by the label (S01).
- the identifier converting unit 18 refers to the ancestor number transposed list storage unit 16 and reads the transposed list corresponding to the designated node as an integer string (S02).
- the identifier conversion unit 18 reads the frequency list corresponding to the designated node as an integer string with reference to the frequency list storage unit 19 (S03).
- the identifier conversion unit 18 creates a list of pairs of identifiers and frequencies based on the ancestor node list, the transposed list, and the frequency list. More specifically, the identifier conversion unit 18 performs a process using the function trans on the pair of values in the transposed list of length n and the frequency list (S04).
- FIG. 19 is a flowchart showing the processing procedure of the function trans.
- the processing procedure of the function trans (inv [i], hit [i], ancList) as arguments is shown.
- inv [i] is substituted for temporary variable pos
- hit [i] is substituted for temporary variable freq (S041).
- the ancestor nodes in ancList are referred to in order from the lower nodes, and the following processing is performed on each ancestor node anc.
- the function selectFreq (pos, anc) performs a process of extracting the pos-th value from the frequency list corresponding to the node anc in the frequency list storage unit 19.
- the frequency list is stored after being divided into L blocks, so that the frequency can be extracted only by performing L decoding processes at most.
- the extracted value is substituted into the variable freq.
- the function select performs a process of extracting the pos-th value from the transposed list of the node anc stored in the ancestor number transposed list storage unit 16.
- the transposed list is divided into L blocks as described above and differentially compressed, in order to extract the pos-th value from the transposed list, at most L decoding and Addition is sufficient.
- pos is output as an identifier, that is, a data number, and freq is output as a frequency (S043).
- the identifier conversion unit 18 rearranges the identifiers in the order of frequency and outputs them as search results (S05).
- the identifier conversion unit 18 is configured to output the identifiers sorted in the order of frequency.
- the information processing apparatus 201 corresponds to each identifier.
- a configuration may be employed in which some evaluation value is provided outside and the identifiers are rearranged based on the frequency and the evaluation value.
- the identifier column in the transposed list corresponding to the node in the taxonomy, is not stored, but in the transposed list of the ancestor node that is an upper node of the node.
- the position This realizes an index with a small amount of data and a high-speed search.
- the ancestor number transposition list storage unit 16 stores the transposed list shown in FIG. Held in.
- the nodes A and C whose ancestor nodes are ⁇ in the ancestor node list shown in FIG. 6 have identifiers corresponding to these nodes as they are.
- the transposed list other than is smaller in the difference between the elements, and can be compressed at a high rate.
- the identifier conversion unit 18 extracts lines corresponding to d (B) from the transposition list shown in FIG. 7 and the frequency list shown in FIG.
- the identifier conversion unit 18 extracts a row corresponding to the node A from the transposed list shown in FIG. 7 as the transposed list corresponding to the ancestor node of the node B.
- the identifier conversion unit 18 collates each data in the row corresponding to d (B) with the transposed list of the ancestor node A, and includes an identifier column (1, 23, 456, 12000, 16333) and (4 , 1, 3, 2, 5) can be extracted.
- the identifier conversion unit 18 can rearrange the extracted identifier sequence in order of frequency as (16333, 1, 456, 12000, 23), and return the data in order from the data strongly associated with the designated node. .
- the ancestor node selection method can be changed.
- the information stored in the ancestor number transposition list storage unit 16 is the transposition list shown in FIG.
- FIG. 20 is a diagram showing an example of the ancestor number transposition list stored in the ancestor number transposition list storage unit in the information processing apparatus according to the first embodiment of the present invention.
- the integer sequence in this transposed list has a smaller difference between the values in the integer sequence corresponding to node C than the transposed list shown in FIG. 7, and compresses the difference. As a result, the data amount of the transposed list can be further reduced.
- the data structure according to the first embodiment of the present invention is a data structure for extracting a set of search target data that can be reached from each node in the taxonomy in the taxonomy having tags related to the search target data. is there.
- the data structure includes ancestor reference data and transposed list data, that is, an ancestor number transposed list.
- This ancestor reference data indicates an ancestor node that is an upper node of each node in the taxonomy.
- the transposition list data includes a transposition list of each node, and among the nodes, the transposition list of the node in which the ancestor node is registered in the ancestor reference data is in the transposition list corresponding to the registered ancestor node.
- the index creation device creates a transposed list used to extract a set of search target data that can be reached from each node in the taxonomy in the taxonomy having tags related to the search target data.
- the list of integer values in the transposed list of each node is divided into a plurality of blocks, and the difference value between the integer value in the block and the integer value immediately before the integer value is converted into a bit string of variable-length integer code.
- the ancestor node determination unit 11 selects one ancestor node that is an upper node of the node for each node in the taxonomy, and generates ancestor reference data indicating the selected ancestor node.
- the ancestor node search unit 13 generates an ancestor node list indicating one or more ancestor nodes of the tag in the taxonomy based on the ancestor reference data. Then, the ancestor number conversion unit 15 receives the identifier of the search target data and adds the identifier as an element of the corresponding transpose list for the highest node among the nodes in the ancestor node list. For a node, an integer value indicating the position in the transposed list corresponding to the node immediately above the node is added as an element of the corresponding transposed list instead of the identifier.
- the data search device extracts a set of search target data that can be reached from a designated node specified in the taxonomy in a taxonomy having a tag related to the search target data.
- the ancestor number transposed list storage unit 16 stores transposed list data.
- This transposed list data includes a transposed list of each node in the taxonomy, and the transposed list of the highest node among the nodes is a list of integer values indicating the identifiers of the search target data.
- the node transposition list is an integer value list indicating a position in the transposition list corresponding to the node immediately above the node, instead of the identifier.
- the list of integer values in the transpose list of each node is divided into a plurality of blocks, and the difference value between the integer value in the block and the integer value immediately before the integer value is converted into a bit string of variable-length integer code. It has been converted.
- the identifier conversion unit 18 receives the information indicating the designated node, and based on the transposed list data, transposes corresponding to the upper node of the designated node corresponding to the position indicated by the integer value of the transposed list corresponding to the designated node. By repeating the process of extracting the integer value of the list until the identifier of the transposed list corresponding to the highest node is extracted, a list of identifiers of search target data corresponding to the designated node is created.
- wrist with small data length can be created by expressing the said difference with a variable-length integer code.
- the transposed index may hold a sequence of identifiers as shown in FIG.
- the identifier string shown in FIG. 7 is configured with a smaller value than the identifier string shown in FIG. 1B, a process of taking a difference between these values and compressing it with a variable-length integer code is added.
- the transposed index can be greatly compressed.
- the transposed index corresponding to each node in the taxonomy can be expressed in a compact manner, and the inquiry to the upper node in the taxonomy can be performed at high speed.
- the data structure for the search processing is reduced and searched by the minimum configuration including the ancestor reference data and the transposed list data among the components in the data structure according to the first embodiment of the present invention. It is possible to achieve the object of the present invention to increase the processing speed.
- the minimum configuration including the ancestor node determination unit 11, the ancestor node search unit 13, and the ancestor number conversion unit 15 is used for search processing. It is possible to achieve the object of the present invention to reduce the data volume and to speed up the search process.
- the data volume for search processing is reduced by the minimum configuration including the ancestor number transposed list storage unit 16 and the identifier conversion unit 18.
- Patent Document 1 discloses a configuration for improving the efficiency of search processing by converting a search word into a tree structure.
- search is performed. This is a configuration in which the entire index is compressed by converting the index used in the above to a tree structure, which is completely different from the configuration described in Patent Document 1.
- the ancestor node determination unit 11 determines an ancestor node for each node based on the data distribution corresponding to the ancestor node. That is, the ancestor node determination unit 11 receives the frequency distribution indicating how often each node in the taxonomy has or is likely to have in the predetermined data set, and each ancestor of the node for each node in the taxonomy. Based on the frequency corresponding to the node, the data length of the corresponding transposed list when each ancestor node is selected is calculated, and among the ancestor nodes, the ancestor node having a smaller data length and the higher ancestor node in the taxonomy are given priority. To select.
- the frequency list storage unit 19 stores a frequency list indicating the frequency corresponding to each search target data of each node in the taxonomy. Then, the identifier conversion unit 18 further sorts the identifiers based on the frequency list in the list of one or more search target data identifiers corresponding to the designated node.
- the data structure according to the first exemplary embodiment of the present invention compares the frequency list of the node with the frequency list of the ancestor node of the node when expressing the frequency list of a certain node in the taxonomy, It has frequency data given a short code when it has the same frequency.
- This configuration makes it possible to create a frequency list with a small data length.
- the present embodiment relates to an information processing apparatus that improves the efficiency of processing when a plurality of nodes are designated as compared to the information processing apparatus according to the first embodiment.
- the contents other than those described below are the same as those of the information processing apparatus according to the first embodiment.
- FIG. 21 is a block diagram showing a control structure provided by the information processing apparatus according to the second embodiment of the present invention.
- the information processing apparatus 202 includes a multiple identifier conversion unit 28 instead of the identifier conversion unit 18 as compared with the information processing apparatus according to the first embodiment of the present invention.
- the multi-identifier conversion unit 28 receives two or more designated nodes in the taxonomy and extracts the ancestor node list using the ancestor node search unit 13 for each designated node.
- the multi-identifier conversion unit 28 creates a trie tree in which common ancestor nodes are collected for each designated node. Then, the multiple identifier conversion unit 28 creates a list of identifiers indicating search target data sets related to all the designated nodes while performing depth-first search of the created trie tree.
- FIG. 22 is a flowchart showing an operation procedure when the information processing apparatus according to the second embodiment of the present invention performs a data search process.
- the information processing apparatus 201 starts a data search process.
- the multi-identifier conversion unit 28 uses the ancestor node search unit 13 to obtain a list of ancestor nodes for a plurality of input designated nodes (J01). This process is the same as the process S01 in the first embodiment of the present invention.
- the multi-identifier conversion unit 18 constructs a trie tree in which common ancestor nodes are collected from a list of ancestor nodes of each designated node (J02). This process is the same as the process RD03 in the first embodiment of the present invention.
- the multiple identifier conversion unit 18 outputs the obtained result as a search result by repeating the JOIN process while performing depth-first search in the constructed trie tree (J03).
- processing J03 will be described in detail.
- the two processes node-in and node-out shown in FIGS. 23 and 24 are performed while performing depth-first search in the trie tree.
- the process node-in is a process performed when entering the node
- the process node-out is a process performed when leaving the node. It is.
- FIG. 23 is a flowchart showing the procedure of processing node-in.
- a buffer list corresponding to a target node which is a newly entered node is created (J031).
- the buffer list is a list for temporarily holding an integer string.
- the transposed list corresponding to the target node is read from the ancestor number transposed list storage unit 16, and the integer sequence of the read transposed list is targeted. It adds to the buffer list of the node (J033).
- the search process is continued for each child node in the trie tree, and the process ends when all the children have been traced. (J034).
- FIG. 24 is a flowchart showing a procedure of processing node-out.
- a process of converting the integer string in the buffer list is performed (J036). This processing is performed by executing the above-described function select (pos, node) for the target node node and each value pos in the buffer list.
- the integer string in the buffer list of the parent node is compared with the integer string in the buffer list of the target node, and the JOIN process, that is, the process of leaving only the integer value included in both (J039).
- FIG. 25 is a diagram showing a specific example of a processing procedure for depth-first search in the information processing apparatus according to the second embodiment of the present invention.
- FIG. 25 shows a trie tree created from these input nodes, and P0 to P13 indicate the search order for this trie tree.
- node A is searched and a buffer list of node A is created (P1).
- node B is searched and a buffer list of node B is created (P2).
- the node L is searched and the transposition list of the node L is read (P3).
- the transposition list of node L is added to the buffer list of node B (P4).
- the node N is searched and the transposition list of the node N is read (P5).
- the buffer list of the node B is converted into a value in the transposition list of the node B and added to the buffer list of the node A (P7).
- the buffer list of node A is converted into a value in the transposition list of node A and added to the buffer list of root (P8).
- node C is searched and a buffer list of node C is created (P9).
- the node O is searched and the transposition list of the node O is read (P10).
- the buffer list of node C is converted into a value in the transposition list of node C, and JOIN processing is performed with the buffer list of root (P12).
- a column of identifiers of search target data related to all of the nodes L, N, and O can be created and output.
- the data search device when a plurality of nodes in a taxonomy are specified, a set of search target data that can be reached from any of the specified nodes is obtained. Take out.
- the multiple identifier conversion unit 28 acquires a transposed list corresponding to each designated node from the ancestor number transposed list storage unit 16, and designates the designated node as a designated node.
- the taxonomy is the most common among the upper nodes common to the set of designated nodes.
- an integer value in a transposed list corresponding to a common ancestor node that is a lower upper node is extracted, a common integer value is extracted in a set of designated nodes.
- the multiple identifier conversion unit 28 creates a list of identifiers of search target data corresponding to the plurality of designated nodes, using the extracted integer values.
- each of the transposed lists corresponding to a plurality of input designated nodes is converted into an identifier column. It is necessary to perform the JOIN process after referring to all the transposed lists of the ancestor nodes referred to by the transposed list of the designated node.
- each designated node is related to 100 search target data
- the position of the transposed list in all ancestor nodes is examined for a total of 300 integers. Need to be converted to an identifier.
- the number of the designated node is the number in the transposed list corresponding to the lowest ancestor node.
- the JOIN processing of the node L and the node N is performed for the position in the transposed list corresponding to the node B without converting each transposed list into an identifier. For this reason, the process of referring to the transposed list corresponding to the node higher than the node B has only to be performed on the result of the JOIN process, and therefore the process can be performed efficiently.
- ⁇ L, B, A ⁇ is obtained as an ancestor node list corresponding to the node L by the process of step J01 in the data search process shown in FIG.
- ⁇ N, B, A ⁇ is obtained as an ancestor node list corresponding to the node N.
- an ancestor trie as shown in FIG. 26 is constructed (J02).
- FIG. 26 is a diagram showing a specific example of a processing procedure for depth-first search in the information processing apparatus according to the second embodiment of the present invention.
- step J03 in the data search process shown in FIG. 22 the process of step J03 in the data search process shown in FIG. 22 is performed, the JOIN process is performed according to the following processing procedure.
- a root buffer list is created (Q0).
- node A is searched and a buffer list of node A is created (Q1).
- node B is searched and a buffer list of node B is created (Q2).
- the node L is searched, and the transposition list ⁇ 1, 3, 4 ⁇ of the node L is read (Q3).
- the transposition list of node L is added to the buffer list of node B (Q4).
- the node N is searched, and the transposition list ⁇ 3 ⁇ of the node N is read (Q5).
- the buffer list of node B is converted to the value ⁇ 3 ⁇ in the transposition list of node B and added to the buffer list of node A (Q7).
- the buffer list of the node A is converted into the value in the transposition list of the node A, that is, the identifier ⁇ 456 ⁇ , and added to the root buffer list (Q8).
- steps Q7 and Q9 the permutation lists corresponding to the node B and the node A are referred to, respectively. This data reference is performed for only one value. Specifically, a process of converting 3 to 3 is performed in step Q7, and a process of converting 3 to 456 is performed in step Q9.
- the permutation list of the upper node is referred to for each of the values in the transposition list ⁇ 1, 3, 4 ⁇ corresponding to the node L and the transposition list ⁇ 3 ⁇ corresponding to the node N. It takes time to process.
- the present embodiment relates to an information processing apparatus that obtains a set of nodes related to search results as compared to the information processing apparatus according to the first embodiment.
- the contents other than those described below are the same as those of the information processing apparatus according to the first embodiment.
- FIG. 27 is a block diagram showing a control structure provided by the information processing apparatus according to the third embodiment of the present invention.
- the information processing device 203 further includes a child node storage unit 21 and a top-down search unit 22 as compared with the information processing device according to the first embodiment of the present invention.
- the child node storage unit 21 stores information indicating a set of child nodes for each node and one virtual root of each node, created by collecting the ancestor nodes described in the ancestor node storage unit 12. To do.
- FIG. 28 is a diagram illustrating an example of information stored in the child node storage unit 21.
- FIG. 28 shows an example of child node information when the ancestor reference data shown in FIG. 6 is obtained.
- a list of child nodes for node A is registered, and in node A, a child node called node B is registered.
- node A and node C are registered as child nodes corresponding to the virtual route root.
- the node A and the node C are registered as the highest node where no ancestor node exists. Assuming that these nodes virtually have a single root root at the upper level and the list of the highest nodes A and C in the taxonomy exists below the root root, the list of child nodes shown in FIG. Has been created.
- the top-down search unit 22 receives the list of identifiers indicating a subset of all the specified search target data and the integer min, and designates them while referring to the transposed list in the ancestor number transposed list storage unit 16. Returns a list of nodes that appear min times or more in the set of search target data.
- the top-down search unit 22 starts a list process when an identifier list indicating a set of search target data and an integer value min are input.
- the identifier conversion unit 18 performs a search process
- the obtained list of identifiers and a preset min value are called as inputs. These may be called by a user or an external program.
- the top-down search unit 22 traces the nodes in the taxonomy from the root node stored in the child node storage unit 21 to the top-down, and determines whether the frequency in the designated search target data set is equal to or higher than min. Investigate. At this time, the top-down search unit 22 executes a recursive function list.
- FIG. 29 is a flowchart showing the processing procedure of the recursive function list.
- the recursive function list includes three nodes: a node indicating a specific node among the nodes stored in the child node storage unit 21, an integer string input representing a search target data set, and a threshold value min. Called with a value as input.
- the transposed list corresponding to the node specified by the node is read from the ancestor number transposed list storage unit 16 and set as nList (L01).
- FIG. 30 is a flowchart showing the processing procedure of the function listJoin.
- listJoin is a function that examines two input integer arrays a and b in order, and returns a list of positions in integer array a where numerical values included in both a and b appear.
- variable ap indicating a position in the array a is initialized to 0
- a variable bp indicating a position in the array b is initialized to 0
- the variable out indicating the integer string to be initialized is initialized in an empty state (L021).
- the top-down search unit 22 examines the length of temp using the integer string obtained as a result of executing the function listJoin as temp (L03).
- the top-down search unit 22 determines that a lower node corresponding to a descendant of the node indicated by the node should also be searched.
- the top-down search unit 22 determines that a lower node corresponding to a descendant of the node indicated by the node should also be searched, the top-down search unit 22 refers to the child node storage unit 21 and acquires a list of child nodes of the node (L04).
- the top-down search unit 22 calls the same function list with each child node cnode, an integer string temp indicating the current search target data, and min as arguments (L05). As a result, a similar search is performed for the descendant nodes.
- the top-down search unit 22 outputs the label of the node specified by the node and the length of temp as the frequency (L06).
- the top-down search unit 22 ends the process.
- the top-down search unit 22 uses the root node root stored in the child node storage unit 21, the input identifier list dids, and the threshold value min as arguments to list (root, dids , Min).
- the top-down search unit 22 starts processing on the child nodes A and C of the root in the child node reference data shown in FIG.
- node A the transposed list shown in the first line of FIG. 7 is read, and processing by the function listJoin is performed.
- listJoin (d (A) ⁇ 2451, 3443 ⁇ ), ⁇ 4, 5 ⁇ Is obtained.
- the length of ⁇ 4, 5 ⁇ is 2, which is larger than min.
- the top-down search unit 22 further performs a search process on the child node B of the node A, and executes listJoin (d (B), ⁇ 4, 5 ⁇ ).
- the top-down search unit 22 does not perform search processing for nodes below the node B, but performs similar processing for another child node C in the root.
- the top-down search unit 22 executes listJoin (d (C), ⁇ 2451, 3443 ⁇ ) and obtains ⁇ 1,2 ⁇ as a result.
- the length of ⁇ 1, 2 ⁇ is 2, which is larger than min.
- the top-down search unit 22 executes listJoin (d (O), ⁇ 1, 2 ⁇ ) on the child node O of the node C to obtain ⁇ 1 ⁇ .
- the top-down search unit 22 outputs information that the frequency of the node A is 2, the frequency of the node O is 1, and the frequency of the node C is 2 in the specified set of search target data. To do.
- the child node storage unit 21 stores child node information indicating a set of child nodes that are lower nodes of each node in the taxonomy.
- the top-down search unit 22 receives a list of identifiers of search target data corresponding to the designated node, and extracts a node having a frequency equal to or higher than a predetermined value in the set of search target data indicated by the list from each node in the taxonomy. .
- the top-down search unit 22 detects the position of the identifier in the transposed list corresponding to the highest node, and based on the child node information, the transposed list corresponding to the detected position and the node one level lower than the highest node. Is compared with the integer value in, and at least a process of calculating the node frequency based on the matched integer value is performed. Furthermore, the top-down search unit 22 uses an integer value corresponding to the position of the identifier in the transposed list corresponding to the upper node and an integer value in the transposed list corresponding to the node one level lower than the upper node for the lower level node. And the process of calculating the frequency of the next lower node based on the matched integer value is performed zero or more times. Thereby, the top-down search part 22 calculates the frequency of each node in search object data.
- the process of searching for a high-frequency node in the search target data set is useful for query expansion and facet search.
- a set of nodes related to the search results can be obtained. Therefore, it is possible to perform query expansion using a set of these nodes and present the set of these nodes as facets.
- a transposed list corresponding to each node in the taxonomy is read from the ancestor number transposed list storage unit 16, an ancestor node is referred to for each transposed list, and an identifier column is obtained. It is necessary to perform the JOIN process after conversion to. Such a method is time consuming because of increased waste.
- the JOIN process can be performed at each node without converting it into an identifier column each time while checking the nodes in the taxonomy top-down. Therefore, the processing efficiency can be improved.
- a data structure for extracting a set of search target data reachable from each node in the taxonomy In the taxonomy, ancestor reference data indicating an ancestor node that is an upper node of each node; Including a transposed list of each node, and among the nodes, the transposed list of the node in which the ancestor node is registered in the ancestor reference data is a position in the transposed list corresponding to the registered ancestor node.
- the integer value list in the transposed list of each node is further divided into a plurality of blocks, and the difference value between the integer value in the block and the integer value immediately before the integer value is variable length.
- a data structure comprising transposed list data converted into a bit string of an integer code.
- an index creation device for creating a transposed list used to extract a set of search target data reachable from each node in the taxonomy,
- the list of integer values in the transposition list of each node is divided into a plurality of blocks, and a difference value between the integer value in the block and the integer value immediately before the integer value is converted into a bit string of a variable-length integer code,
- an ancestor node determination unit for selecting one ancestor node that is an upper node of the node and generating ancestor reference data indicating the selected ancestor node;
- An ancestor node search unit for generating an ancestor node list indicating one or more ancestor nodes of the tag in the taxonomy based on the ancestor reference data;
- the identifier of the search target data for the highest node among the nodes in the ancestor node list, the identifier
- the ancestor node determination unit receives a frequency distribution indicating how much each node in the taxonomy has or is likely to have in a predetermined data set, and for each node in the taxonomy, Based on the frequency corresponding to the ancestor node, the data length of the corresponding transpose list when each ancestor node is selected is calculated, and among the ancestor nodes, the ancestor node having the smaller data length and the taxonomy
- the index creation device according to attachment 2 wherein the higher-order ancestor node is selected with priority.
- a data search device for extracting a set of search target data reachable from a specified node specified in the taxonomy
- a transposition list of each node in the taxonomy, and the transposition list of the highest node among the nodes is a list of integer values indicating identifiers of the search target data, and transpositions of nodes other than the highest node
- the list is a list of integer values indicating positions in a transposed list corresponding to a node immediately above the node instead of the identifier, and the list of integer values in the transposed list of each node is a plurality of blocks.
- An ancestor number transposed list storage unit for storing transposed list data in which a difference value between an integer value in the block and an integer value immediately before the integer value is converted into a bit string of a variable-length integer code;
- the permutation list corresponding to the upper node of the designated node corresponding to the position indicated by the integer value of the permutation list corresponding to the designated node based on the data for the permuted list upon receiving information indicating the designated node
- An identifier conversion unit for creating a list of identifiers of the search target data corresponding to the designated node by repeating the process of extracting the integer value until the identifier of the transposed list corresponding to the highest node is extracted.
- a data search device comprising:
- the data search device further includes: A frequency list storage unit for storing a frequency list indicating the frequency corresponding to each search target data of each node in the taxonomy, The data search device according to appendix 4, wherein the identifier conversion unit further sorts the identifiers based on the frequency list in a list of identifiers of the one or more search target data corresponding to the designated node.
- the data search device takes out a set of search target data that can be reached from any of the specified nodes,
- the identifier conversion unit acquires the transposed list corresponding to each designated node from the ancestor number transposed list storage unit, and the transposed corresponding to the designated node
- the integer value in the transposed list corresponding to the common ancestor node that is the lowest upper node in the taxonomy is extracted, the integer value that is common in the set of the specified nodes is extracted, and the plurality of integer values are extracted using the extracted integer value.
- the data search device according to appendix 4, wherein a list of identifiers of the search target data corresponding to the designated node is created .
- the data search device further includes: A child node storage unit for storing child node information indicating a set of child nodes that are lower nodes of each node in the taxonomy; Receiving a list of identifiers of the search target data corresponding to the designated node, and extracting from the nodes in the taxonomy a node having a frequency equal to or higher than a predetermined value in the set of search target data indicated by the list With a top-down search unit, The top-down search unit detects the position of the identifier in the transposed list corresponding to the highest node, and based on the child node information, detects the position and the node one level lower than the highest node.
- an index creation method for creating a transposed list used to extract a set of search target data reachable from each node in the taxonomy
- the list of integer values in the transposition list of each node is divided into a plurality of blocks, and a difference value between the integer value in the block and the integer value immediately before the integer value is converted into a bit string of a variable-length integer code, Selecting one ancestor node that is an upper node of the node for each node in the taxonomy, and generating ancestor reference data indicating the selected ancestor node; Generating an ancestor node list indicating one or more ancestor nodes of a tag in the taxonomy based on the ancestor reference data; In response to the identifier of the search target data, for the highest node among the nodes in the ancestor node list, the identifier is added as an element of the corresponding transposed list, and for nodes
- each node in the taxonomy receives a frequency distribution indicating how much each node in the taxonomy has or is likely to have in a predetermined data set, and for each node in the taxonomy And calculating the data length of the corresponding transposed list when each ancestor node is selected based on the frequency corresponding to each ancestor node of the node, and among the ancestor nodes, the ancestor node having the smaller data length
- the index creation method according to appendix 8 wherein the higher-ranked ancestor node is preferentially selected in the taxonomy.
- a data search method for extracting a set of search target data that can be reached from a designated node specified in the taxonomy A transposition list of each node in the taxonomy, and the transposition list of the highest node among the nodes is a list of integer values indicating identifiers of the search target data, and transpositions of nodes other than the highest node
- the list is a list of integer values indicating positions in a transposed list corresponding to a node immediately above the node instead of the identifier, and the list of integer values in the transposed list of each node is a plurality of blocks.
- the permutation list corresponding to the upper node of the designated node corresponding to the position indicated by the integer value of the permutation list corresponding to the designated node based on the data for the permuted list upon receiving information indicating the designated node Generating a list of identifiers of the search target data corresponding to the designated node by repeating the process of extracting the integer value until the identifier of the transposed list corresponding to the highest node is extracted.
- Data search method is
- the data search method further includes: Obtaining a frequency list indicating the frequency corresponding to each search target data of each node in the taxonomy;
- the data search method according to claim 10 further comprising: sorting the identifiers based on the frequency list in a list of identifiers of the one or more search target data corresponding to the designated node.
- the data search method further includes: Obtaining child node information indicating a set of child nodes that are lower nodes of each node in the taxonomy; Receiving a list of identifiers of the search target data corresponding to the designated node, and extracting from the nodes in the taxonomy a node having a frequency equal to or higher than a predetermined value in the set of search target data indicated by the list; Including In the step of extracting the node, the position of the identifier in the transposed list corresponding to the highest node is detected, and the detected node and a node one level lower than the highest node based on the child node information At least a process of calculating the frequency of the node based on the matched integer value, and for the lower nodes of the node, the transpose corresponding to the upper node The integer value corresponding to the position of the identifier in the list is compared with the integer value in the transposed list corresponding to the node one level lower than the
- a taxonomy having a tag related to search target data a computer-readable recording medium recording an index creation program for creating a transposed list used for extracting a set of search target data reachable from each node in the taxonomy ,
- the list of integer values in the transposition list of each node is divided into a plurality of blocks, and a difference value between the integer value in the block and the integer value immediately before the integer value is converted into a bit string of a variable-length integer code,
- the index creation program is stored in a computer.
- each node in the taxonomy receives a frequency distribution indicating how much each node in the taxonomy has or is likely to have in a predetermined data set, and for each node in the taxonomy And calculating the data length of the corresponding transposed list when each ancestor node is selected based on the frequency corresponding to each ancestor node of the node, and among the ancestor nodes, the ancestor node having the smaller data length 15.
- Appendix 16 A taxonomy having a tag related to search target data, wherein the data search program is a computer-readable recording medium storing a data search program for extracting a set of search target data that can be reached from a designated node specified in the taxonomy.
- a transposition list of each node in the taxonomy, and the transposition list of the highest node among the nodes is a list of integer values indicating identifiers of the search target data, and transpositions of nodes other than the highest node
- the list is a list of integer values indicating positions in a transposed list corresponding to a node immediately above the node instead of the identifier, and the list of integer values in the transposed list of each node is a plurality of blocks.
- the permutation list corresponding to the upper node of the designated node corresponding to the position indicated by the integer value of the permutation list corresponding to the designated node based on the data for the permuted list upon receiving information indicating the designated node The process of extracting the integer value of is performed until the identifier of the transposed list corresponding to the highest node is extracted, thereby executing the step of creating a list of identifiers of the search target data corresponding to the designated node
- a computer-readable recording medium that is a program for the above.
- the data search program is further stored in a computer. Obtaining a frequency list indicating the frequency corresponding to each search target data of each node in the taxonomy; 18.
- the computer-readable program according to appendix 16 which is a program for executing a step of sorting the identifiers based on the frequency list in a list of one or more identifiers of the search target data corresponding to the designated node. Recording medium.
- the data search program is further stored in a computer.
- Is a program for executing In the step of extracting the node, the position of the identifier in the transposed list corresponding to the highest node is detected, and the detected node and a node one level lower than the highest node based on the child node information At least a process of calculating the frequency of the node based on the matched integer value, and for the lower nodes of the node, the transpose corresponding to the upper node
- the integer value corresponding to the position of the identifier in the list is compared with the integer value in the transposed
- the present invention it is possible to provide a search engine that can search a data set at high speed when managing a large taxonomy and data set. Therefore, the present invention has industrial applicability.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
本発明の第1の実施の形態に係るデータ検索装置は、タクソノミ中のノードを示すXを入力し、d(X)を返す機能を提供する。このような機能を実現する方法としては、転置インデックスを用いることが考えられる。
本発明の第1の実施の形態に係るインデックス作成装置およびデータ検索装置を備える情報処理装置は、典型的には、汎用的なアーキテクチャを有するコンピュータを基本構造としており、予めインストールされたプログラムを実行することで、後述するような各種機能を提供する。一般的に、このようなプログラムは、フレキシブルディスク(Flexible Disk)およびCD-ROM(Compact Disk Read Only Memory)などの記録媒体に格納されて、あるいはネットワークなどを介して流通する。
次に、情報処理装置201におけるインデックス作成機能およびデータ検索機能を提供するための制御構造について説明する。
次に、本発明の第1の実施の形態に係る情報処理装置の動作について図面を用いて説明する。本発明の第1の実施の形態では、情報処理装置を動作させることによって、本発明の第1の実施の形態に係るインデックス作成方法およびデータ検索方法が実施される。よって、本発明の第1の実施の形態に係るインデックス作成方法およびデータ検索方法の説明は、以下の情報処理装置201の動作説明に代える。なお、以下の説明においては、適宜図5を参照する。
図9は、本発明の第1の実施の形態に係る情報処理装置がタクソノミ登録処理を行なう際の動作手順を示すフローチャートである。
csomp(Y、X)=entopy(All、freq(X))-entopy(freq(Y),freq(X))
entopy(m,n)=n×log(m/n)+(m-n)×log(m/(m-n))
AncScore(Y,X)=α×dis(X,Y)+β×1/entopy(freq(Y),freq(X))
次に、データ登録処理を説明する。
次に、データ検索処理を説明する。
本実施の形態は、第1の実施の形態に係る情報処理装置と比べて複数のノードが指定された場合の処理の効率化を図る情報処理装置に関する。以下で説明する内容以外は第1の実施の形態に係る情報処理装置と同様である。
本実施の形態は、第1の実施の形態に係る情報処理装置と比べて検索結果に関連するノードの集合を得る情報処理装置に関する。以下で説明する内容以外は第1の実施の形態に係る情報処理装置と同様である。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すためのデータ構造であって、
前記タクソノミにおいて、前記各ノードの上位ノードである先祖ノードを示す先祖参照用データと、
前記各ノードの転置リストを含み、前記各ノードのうち、前記先祖参照用データにおいて前記先祖ノードが登録されているノードの転置リストは、登録された前記先祖ノードに対応する転置リスト中の位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データとを備える、データ構造。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すために用いる転置リストを作成するためのインデックス作成装置であって、
前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換されており、
前記タクソノミにおけるノードごとに、前記ノードの上位ノードである先祖ノードを1つ選択し、選択した前記先祖ノードを示す先祖参照用データを生成するための先祖ノード決定部と、
前記先祖参照用データに基づいて、前記タクソノミにおけるタグの1または複数の先祖ノードを示す先祖ノードリストを生成するための先祖ノード探索部と、
前記検索対象データの識別子を受けて、前記先祖ノードリストにおける各前記ノードのうち最上位のノードについては、前記識別子を対応の転置リストの要素として追加し、前記最上位ノード以外のノードについては、対応の転置リストの要素として、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値を追加するための先祖番号変換部とを備える、インデックス作成装置。
前記先祖ノード決定部は、前記タクソノミにおける各ノードが所定のデータ集合内でどの程度の頻度を持つか、あるいは持ちそうかを示す頻度分布を受けて、前記タクソノミにおけるノードごとに、前記ノードの各先祖ノードに対応する頻度に基づいて、前記各先祖ノードを選んだ場合における対応の転置リストのデータ長を計算し、前記各先祖ノードのうち、前記データ長が小さい前記先祖ノードおよび前記タクソノミにおいてより上位の前記先祖ノードを優先して選択する、付記2に記載のインデックス作成装置。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける指定された指定ノードから到達可能な検索対象データの集合を取り出すためのデータ検索装置であって、
前記タクソノミにおける各ノードの転置リストを含み、前記各ノードのうち、最上位のノードの転置リストは、前記検索対象データの識別子を示す整数値のリストであり、前記最上位ノード以外のノードの転置リストは、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データを記憶するための先祖番号転置リスト記憶部と、
前記指定ノードを示す情報を受けて、前記転置リスト用データに基づき、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を、前記最上位ノードに対応する前記転置リストの前記識別子を取り出すまで繰り返すことにより、前記指定ノードに対応する前記検索対象データの識別子のリストを作成するための識別子変換部とを備える、データ検索装置。
前記データ検索装置は、さらに、
前記タクソノミにおける各ノードの、各検索対象データに対応する頻度を示す頻度リストを記憶するための頻度リスト記憶部を備え、
前記識別子変換部は、さらに、前記指定ノードに対応する1または複数の前記検索対象データの識別子のリストにおいて、前記頻度リストに基づいて前記識別子をソートする、付記4に記載のデータ検索装置。
前記データ検索装置は、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードのいずれからも到達可能な検索対象データの集合を取り出し、
前記識別子変換部は、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードに対応する前記転置リストを前記先祖番号転置リスト記憶部から取得し、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を行なう際に、前記指定ノードの組で共通の前記上位ノードのうち、前記タクソノミにおいて最も下位の上位ノードである共通先祖ノードに対応する前記転置リストにおける整数値を取り出すとき、前記指定ノードの組において共通の前記整数値を取り出し、取り出した前記整数値を用いて、前記複数の指定ノードに対応する前記検索対象データの識別子のリストを作成する、付記4に記載のデータ検索装置。
前記データ検索装置は、さらに、
前記タクソノミにおける各ノードの下位ノードである子ノードの集合を示す子ノード情報を記憶するための子ノード記憶部と、
前記指定ノードに対応する前記検索対象データの識別子のリストを受けて、前記タクソノミにおける各ノードの中から、前記リストが示す前記検索対象データの集合において所定値以上の頻度を持つノードを取り出すためのトップダウン探索部とを備え、
前記トップダウン探索部は、前記最上位ノードに対応する前記転置リストにおける前記識別子の位置を検出し、前記子ノード情報に基づいて、検出した前記位置と前記最上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記ノードの頻度を算出する処理を少なくとも行い、さらに、前記ノードの下位のノードについて、上位ノードに対応する前記転置リストにおいて前記識別子の位置に対応する整数値と、前記上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記1つ下位のノードの頻度を算出する処理を0回以上行なうことにより、前記検索対象データにおける前記各ノードの頻度を算出する、付記4に記載のデータ検索装置。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すために用いる転置リストを作成するインデックス作成方法であって、
前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換されており、
前記タクソノミにおけるノードごとに、前記ノードの上位ノードである先祖ノードを1つ選択し、選択した前記先祖ノードを示す先祖参照用データを生成するステップと、
前記先祖参照用データに基づいて、前記タクソノミにおけるタグの1または複数の先祖ノードを示す先祖ノードリストを生成するステップと、
前記検索対象データの識別子を受けて、前記先祖ノードリストにおける各前記ノードのうち最上位のノードについては、前記識別子を対応の転置リストの要素として追加し、前記最上位ノード以外のノードについては、対応の転置リストの要素として、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値を追加するステップとを含む、インデックス作成方法。
前記先祖参照用データを生成するステップにおいては、前記タクソノミにおける各ノードが所定のデータ集合内でどの程度の頻度を持つか、あるいは持ちそうかを示す頻度分布を受けて、前記タクソノミにおけるノードごとに、前記ノードの各先祖ノードに対応する頻度に基づいて、前記各先祖ノードを選んだ場合における対応の転置リストのデータ長を計算し、前記各先祖ノードのうち、前記データ長が小さい前記先祖ノードおよび前記タクソノミにおいてより上位の前記先祖ノードを優先して選択する、付記8に記載のインデックス作成方法。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける指定された指定ノードから到達可能な検索対象データの集合を取り出すデータ検索方法であって、
前記タクソノミにおける各ノードの転置リストを含み、前記各ノードのうち、最上位のノードの転置リストは、前記検索対象データの識別子を示す整数値のリストであり、前記最上位ノード以外のノードの転置リストは、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データを取得するステップと、
前記指定ノードを示す情報を受けて、前記転置リスト用データに基づき、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を、前記最上位ノードに対応する前記転置リストの前記識別子を取り出すまで繰り返すことにより、前記指定ノードに対応する前記検索対象データの識別子のリストを作成するステップとを含む、データ検索方法。
前記データ検索方法は、さらに、
前記タクソノミにおける各ノードの、各検索対象データに対応する頻度を示す頻度リストを取得するステップと、
前記指定ノードに対応する1または複数の前記検索対象データの識別子のリストにおいて、前記頻度リストに基づいて前記識別子をソートするステップとを含む、付記10に記載のデータ検索方法。
前記データ検索方法では、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードのいずれからも到達可能な検索対象データの集合を取り出し、
前記識別子のリストを作成するステップにおいては、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードに対応する前記転置リストを取得し、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を行なう際に、前記指定ノードの組で共通の前記上位ノードのうち、前記タクソノミにおいて最も下位の上位ノードである共通先祖ノードに対応する前記転置リストにおける整数値を取り出すとき、前記指定ノードの組において共通の前記整数値を取り出し、取り出した前記整数値を用いて、前記複数の指定ノードに対応する前記検索対象データの識別子のリストを作成する、付記10に記載のデータ検索方法。
前記データ検索方法は、さらに、
前記タクソノミにおける各ノードの下位ノードである子ノードの集合を示す子ノード情報を取得するステップと、
前記指定ノードに対応する前記検索対象データの識別子のリストを受けて、前記タクソノミにおける各ノードの中から、前記リストが示す前記検索対象データの集合において所定値以上の頻度を持つノードを取り出すステップとを含み、
前記ノードを取り出すステップにおいては、前記最上位ノードに対応する前記転置リストにおける前記識別子の位置を検出し、前記子ノード情報に基づいて、検出した前記位置と前記最上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記ノードの頻度を算出する処理を少なくとも行い、さらに、前記ノードの下位のノードについて、上位ノードに対応する前記転置リストにおいて前記識別子の位置に対応する整数値と、前記上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記1つ下位のノードの頻度を算出する処理を0回以上行なうことにより、前記検索対象データにおける前記各ノードの頻度を算出する、付記10に記載のデータ検索方法。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すために用いる転置リストを作成するインデックス作成プログラムを記録した、コンピュータ読み取り可能な記録媒体であって、
前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換されており、
前記インデックス作成プログラムは、コンピュータに、
前記タクソノミにおけるノードごとに、前記ノードの上位ノードである先祖ノードを1つ選択し、選択した前記先祖ノードを示す先祖参照用データを生成するステップと、
前記先祖参照用データに基づいて、前記タクソノミにおけるタグの1または複数の先祖ノードを示す先祖ノードリストを生成するステップと、
前記検索対象データの識別子を受けて、前記先祖ノードリストにおける各前記ノードのうち最上位のノードについては、前記識別子を対応の転置リストの要素として追加し、前記最上位ノード以外のノードについては、対応の転置リストの要素として、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値を追加するステップとを実行させるためのプログラムである、コンピュータ読み取り可能な記録媒体。
前記先祖参照用データを生成するステップにおいては、前記タクソノミにおける各ノードが所定のデータ集合内でどの程度の頻度を持つか、あるいは持ちそうかを示す頻度分布を受けて、前記タクソノミにおけるノードごとに、前記ノードの各先祖ノードに対応する頻度に基づいて、前記各先祖ノードを選んだ場合における対応の転置リストのデータ長を計算し、前記各先祖ノードのうち、前記データ長が小さい前記先祖ノードおよび前記タクソノミにおいてより上位の前記先祖ノードを優先して選択する、付記14に記載のコンピュータ読み取り可能な記録媒体。
検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける指定された指定ノードから到達可能な検索対象データの集合を取り出すデータ検索プログラムを記録した、コンピュータ読み取り可能な記録媒体であって、前記データ検索プログラムは、コンピュータに、
前記タクソノミにおける各ノードの転置リストを含み、前記各ノードのうち、最上位のノードの転置リストは、前記検索対象データの識別子を示す整数値のリストであり、前記最上位ノード以外のノードの転置リストは、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データを取得するステップと、
前記指定ノードを示す情報を受けて、前記転置リスト用データに基づき、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を、前記最上位ノードに対応する前記転置リストの前記識別子を取り出すまで繰り返すことにより、前記指定ノードに対応する前記検索対象データの識別子のリストを作成するステップとを実行させるためのプログラムである、コンピュータ読み取り可能な記録媒体。
前記データ検索プログラムは、さらに、コンピュータに、
前記タクソノミにおける各ノードの、各検索対象データに対応する頻度を示す頻度リストを取得するステップと、
前記指定ノードに対応する1または複数の前記検索対象データの識別子のリストにおいて、前記頻度リストに基づいて前記識別子をソートするステップとを実行させるためのプログラムである、付記16に記載のコンピュータ読み取り可能な記録媒体。
前記データ検索プログラムでは、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードのいずれからも到達可能な検索対象データの集合を取り出し、
前記識別子のリストを作成するステップにおいては、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードに対応する前記転置リストを取得し、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を行なう際に、前記指定ノードの組で共通の前記上位ノードのうち、前記タクソノミにおいて最も下位の上位ノードである共通先祖ノードに対応する前記転置リストにおける整数値を取り出すとき、前記指定ノードの組において共通の前記整数値を取り出し、取り出した前記整数値を用いて、前記複数の指定ノードに対応する前記検索対象データの識別子のリストを作成する、付記16に記載のコンピュータ読み取り可能な記録媒体。
前記データ検索プログラムは、さらに、コンピュータに、
前記タクソノミにおける各ノードの下位ノードである子ノードの集合を示す子ノード情報を取得するステップと、
前記指定ノードに対応する前記検索対象データの識別子のリストを受けて、前記タクソノミにおける各ノードの中から、前記リストが示す前記検索対象データの集合において所定値以上の頻度を持つノードを取り出すステップとを実行させるためのプログラムであり、
前記ノードを取り出すステップにおいては、前記最上位ノードに対応する前記転置リストにおける前記識別子の位置を検出し、前記子ノード情報に基づいて、検出した前記位置と前記最上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記ノードの頻度を算出する処理を少なくとも行い、さらに、前記ノードの下位のノードについて、上位ノードに対応する前記転置リストにおいて前記識別子の位置に対応する整数値と、前記上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記1つ下位のノードの頻度を算出する処理を0回以上行なうことにより、前記検索対象データにおける前記各ノードの頻度を算出する、付記16に記載のコンピュータ読み取り可能な記録媒体。
12 先祖ノード記憶部
13 先祖ノード探索部
14 識別子付与部
15 先祖番号変換部
16 先祖番号転置リスト記憶部
17 先祖頻度変換部
18 識別子変換部
19 頻度リスト記憶部
21 子ノード記憶部
22 トップダウン探索部
28 複数識別子変換部
101 CPU
102 メインメモリ
103 ハードディスク
104 入力インターフェイス
105 表示コントローラ
106 データリーダ/ライタ
107 通信インターフェイス
108 キーボード
109 マウス
110 ディスプレイ
111 記録媒体
121 バス
201,202,203 情報処理装置
Claims (10)
- 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すためのデータ構造であって、
前記タクソノミにおいて、前記各ノードの上位ノードである先祖ノードを示す先祖参照用データと、
前記各ノードの転置リストを含み、前記各ノードのうち、前記先祖参照用データにおいて前記先祖ノードが登録されているノードの転置リストは、登録された前記先祖ノードに対応する転置リスト中の位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データとを備える、データ構造。 - 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すために用いる転置リストを作成するためのインデックス作成装置であって、
前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換されており、
前記タクソノミにおけるノードごとに、前記ノードの上位ノードである先祖ノードを1つ選択し、選択した前記先祖ノードを示す先祖参照用データを生成するための先祖ノード決定部と、
前記先祖参照用データに基づいて、前記タクソノミにおけるタグの1または複数の先祖ノードを示す先祖ノードリストを生成するための先祖ノード探索部と、
前記検索対象データの識別子を受けて、前記先祖ノードリストにおける各前記ノードのうち最上位のノードについては、前記識別子を対応の転置リストの要素として追加し、前記最上位ノード以外のノードについては、対応の転置リストの要素として、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値を追加するための先祖番号変換部とを備える、インデックス作成装置。 - 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける指定された指定ノードから到達可能な検索対象データの集合を取り出すためのデータ検索装置であって、
前記タクソノミにおける各ノードの転置リストを含み、前記各ノードのうち、最上位のノードの転置リストは、前記検索対象データの識別子を示す整数値のリストであり、前記最上位ノード以外のノードの転置リストは、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データを記憶するための先祖番号転置リスト記憶部と、
前記指定ノードを示す情報を受けて、前記転置リスト用データに基づき、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を、前記最上位ノードに対応する前記転置リストの前記識別子を取り出すまで繰り返すことにより、前記指定ノードに対応する前記検索対象データの識別子のリストを作成するための識別子変換部とを備える、データ検索装置。 - 前記データ検索装置は、さらに、
前記タクソノミにおける各ノードの、各検索対象データに対応する頻度を示す頻度リストを記憶するための頻度リスト記憶部を備え、
前記識別子変換部は、さらに、前記指定ノードに対応する1または複数の前記検索対象データの識別子のリストにおいて、前記頻度リストに基づいて前記識別子をソートする、
請求の範囲第3項に記載のデータ検索装置。 - 前記データ検索装置は、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードのいずれからも到達可能な検索対象データの集合を取り出し、
前記識別子変換部は、前記タクソノミにおける複数のノードが指定された場合には、各前記指定ノードに対応する前記転置リストを前記先祖番号転置リスト記憶部から取得し、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を行なう際に、前記指定ノードの組で共通の前記上位ノードのうち、前記タクソノミにおいて最も下位の上位ノードである共通先祖ノードに対応する前記転置リストにおける整数値を取り出すとき、前記指定ノードの組において共通の前記整数値を取り出し、取り出した前記整数値を用いて、前記複数の指定ノードに対応する前記検索対象データの識別子のリストを作成する、請求の範囲第3項に記載のデータ検索装置。 - 前記データ検索装置は、さらに、
前記タクソノミにおける各ノードの下位ノードである子ノードの集合を示す子ノード情報を記憶するための子ノード記憶部と、
前記指定ノードに対応する前記検索対象データの識別子のリストを受けて、前記タクソノミにおける各ノードの中から、前記リストが示す前記検索対象データの集合において所定値以上の頻度を持つノードを取り出すためのトップダウン探索部とを備え、
前記トップダウン探索部は、前記最上位ノードに対応する前記転置リストにおける前記識別子の位置を検出し、前記子ノード情報に基づいて、検出した前記位置と前記最上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記ノードの頻度を算出する処理を少なくとも行い、さらに、前記ノードの下位のノードについて、上位ノードに対応する前記転置リストにおいて前記識別子の位置に対応する整数値と、前記上位ノードの1つ下位のノードに対応する前記転置リストにおける整数値とを比較し、一致した整数値に基づいて前記1つ下位のノードの頻度を算出する処理を0回以上行なうことにより、前記検索対象データにおける前記各ノードの頻度を算出する、請求の範囲第3項に記載のデータ検索装置。 - 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すために用いる転置リストを作成するインデックス作成方法であって、
前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換されており、
前記タクソノミにおけるノードごとに、前記ノードの上位ノードである先祖ノードを1つ選択し、選択した前記先祖ノードを示す先祖参照用データを生成するステップと、
前記先祖参照用データに基づいて、前記タクソノミにおけるタグの1または複数の先祖ノードを示す先祖ノードリストを生成するステップと、
前記検索対象データの識別子を受けて、前記先祖ノードリストにおける各前記ノードのうち最上位のノードについては、前記識別子を対応の転置リストの要素として追加し、前記最上位ノード以外のノードについては、対応の転置リストの要素として、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値を追加するステップとを含む、インデックス作成方法。 - 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける指定された指定ノードから到達可能な検索対象データの集合を取り出すデータ検索方法であって、
前記タクソノミにおける各ノードの転置リストを含み、前記各ノードのうち、最上位のノードの転置リストは、前記検索対象データの識別子を示す整数値のリストであり、前記最上位ノード以外のノードの転置リストは、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データを取得するステップと、
前記指定ノードを示す情報を受けて、前記転置リスト用データに基づき、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を、前記最上位ノードに対応する前記転置リストの前記識別子を取り出すまで繰り返すことにより、前記指定ノードに対応する前記検索対象データの識別子のリストを作成するステップとを含む、データ検索方法。 - 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける各ノードから到達可能な検索対象データの集合を取り出すために用いる転置リストを作成するインデックス作成プログラムを記録した、コンピュータ読み取り可能な記録媒体であって、
前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換されており、
前記インデックス作成プログラムは、コンピュータに、
前記タクソノミにおけるノードごとに、前記ノードの上位ノードである先祖ノードを1つ選択し、選択した前記先祖ノードを示す先祖参照用データを生成するステップと、
前記先祖参照用データに基づいて、前記タクソノミにおけるタグの1または複数の先祖ノードを示す先祖ノードリストを生成するステップと、
前記検索対象データの識別子を受けて、前記先祖ノードリストにおける各前記ノードのうち最上位のノードについては、前記識別子を対応の転置リストの要素として追加し、前記最上位ノード以外のノードについては、対応の転置リストの要素として、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値を追加するステップとを実行させるためのプログラムである、コンピュータ読み取り可能な記録媒体。 - 検索対象データに関するタグを有するタクソノミにおいて、前記タクソノミにおける指定された指定ノードから到達可能な検索対象データの集合を取り出すデータ検索プログラムを記録した、コンピュータ読み取り可能な記録媒体であって、前記データ検索プログラムは、コンピュータに、
前記タクソノミにおける各ノードの転置リストを含み、前記各ノードのうち、最上位のノードの転置リストは、前記検索対象データの識別子を示す整数値のリストであり、前記最上位ノード以外のノードの転置リストは、前記識別子の代わりに、前記ノードの1つ上のノードに対応する転置リストにおける位置を示す整数値のリストであり、さらに、前記各ノードの転置リストにおける整数値のリストが複数のブロックに分割され、前記ブロックにおける整数値と前記整数値の直前の整数値との差分値が可変長整数コードのビット列に変換された転置リスト用データを取得するステップと、
前記指定ノードを示す情報を受けて、前記転置リスト用データに基づき、前記指定ノードに対応する前記転置リストの整数値が示す前記位置に対応する、前記指定ノードの上位ノードに対応する前記転置リストの整数値を取り出す処理を、前記最上位ノードに対応する前記転置リストの前記識別子を取り出すまで繰り返すことにより、前記指定ノードに対応する前記検索対象データの識別子のリストを作成するステップとを実行させるためのプログラムである、コンピュータ読み取り可能な記録媒体。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012538590A JP5812007B2 (ja) | 2010-10-15 | 2011-06-16 | インデックス作成装置、データ検索装置、インデックス作成方法、データ検索方法、インデックス作成プログラムおよびデータ検索プログラム |
US13/824,740 US9600565B2 (en) | 2010-10-15 | 2011-06-16 | Data structure, index creation device, data search device, index creation method, data search method, and computer-readable recording medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010-232896 | 2010-10-15 | ||
JP2010232896 | 2010-10-15 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012049883A1 true WO2012049883A1 (ja) | 2012-04-19 |
Family
ID=45938120
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/063792 WO2012049883A1 (ja) | 2010-10-15 | 2011-06-16 | データ構造、インデックス作成装置、データ検索装置、インデックス作成方法、データ検索方法およびコンピュータ読み取り可能な記録媒体 |
Country Status (3)
Country | Link |
---|---|
US (1) | US9600565B2 (ja) |
JP (1) | JP5812007B2 (ja) |
WO (1) | WO2012049883A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2021517306A (ja) * | 2017-03-15 | 2021-07-15 | センシェア ゲーエムベーハー | データベースにおけるトライデータ構造の有効使用 |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2517477A (en) * | 2013-08-22 | 2015-02-25 | Deep Secure Ltd | A method of transmitting data structures from one computer to another computer |
WO2016038858A1 (ja) * | 2014-09-09 | 2016-03-17 | 日本電気株式会社 | データ管理システム、データ管理装置、データ管理方法、および、プログラム |
US10599726B2 (en) * | 2017-04-19 | 2020-03-24 | A9.Com, Inc. | Methods and systems for real-time updating of encoded search indexes |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10301959A (ja) * | 1997-02-28 | 1998-11-13 | Fujitsu Ltd | データ圧縮/復元装置およびデータ圧縮/復元方法 |
JP2001216314A (ja) * | 2000-02-01 | 2001-08-10 | Nippon Telegr & Teleph Corp <Ntt> | 時空間に適応する情報検索方法及び装置及び時空間に適応する情報検索プログラムを格納した記憶媒体 |
JP2007102786A (ja) * | 2005-09-30 | 2007-04-19 | Internatl Business Mach Corp <Ibm> | 大規模フルテキスト・インデックスでタキソノミーにインデックスを付け、探索することをサポートする方法、装置およびシステム |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3132738B2 (ja) * | 1992-12-10 | 2001-02-05 | ゼロックス コーポレーション | テキスト検索方法 |
US6233575B1 (en) * | 1997-06-24 | 2001-05-15 | International Business Machines Corporation | Multilevel taxonomy based on features derived from training documents classification using fisher values as discrimination values |
JP4011595B2 (ja) | 1998-02-02 | 2007-11-21 | 株式会社リコー | 電子化文書検索システムおよび記録媒体 |
FI991261A (fi) | 1999-06-02 | 2000-12-03 | Nokia Networks Oy | Trie-rakenteeseen perustuva funktionaalinen muisti |
US20010049677A1 (en) * | 2000-03-30 | 2001-12-06 | Iqbal Talib | Methods and systems for enabling efficient retrieval of documents from a document archive |
US7526425B2 (en) * | 2001-08-14 | 2009-04-28 | Evri Inc. | Method and system for extending keyword searching to syntactically and semantically annotated data |
US7502765B2 (en) * | 2005-12-21 | 2009-03-10 | International Business Machines Corporation | Method for organizing semi-structured data into a taxonomy, based on tag-separated clustering |
JP5346156B2 (ja) | 2006-11-08 | 2013-11-20 | 株式会社日立製作所 | プログラム、転置インデックスの格納方法及び検索方法 |
US7496568B2 (en) * | 2006-11-30 | 2009-02-24 | International Business Machines Corporation | Efficient multifaceted search in information retrieval systems |
CN101647021B (zh) * | 2007-04-13 | 2013-03-27 | 麻省理工学院 | 语音数据检索装置、语音数据检索方法、语音数据检索程序和包含有语音数据检索程序的计算机可用介质 |
US7609000B1 (en) * | 2007-10-22 | 2009-10-27 | Google Inc. | Variable-length compression technique for encoding or decoding a sequence of integers |
JP5376163B2 (ja) * | 2007-11-15 | 2013-12-25 | 日本電気株式会社 | 文書管理・検索システムおよび文書の管理・検索方法 |
US8073869B2 (en) * | 2008-07-03 | 2011-12-06 | The Regents Of The University Of California | Method for efficiently supporting interactive, fuzzy search on structured data |
US8060580B2 (en) * | 2008-10-03 | 2011-11-15 | Seomoz, Inc. | Index rank optimization system and method |
CN102246172A (zh) * | 2008-10-13 | 2011-11-16 | 法卢资产有限公司 | 用于电子内容的分布式索引搜索的系统及方法 |
US9218380B2 (en) * | 2009-12-30 | 2015-12-22 | Telecom Italia S.P.A. | Method and system for carrying out searches in a database comprising taxonomic classification of digital information contents |
US8423489B2 (en) * | 2010-05-20 | 2013-04-16 | Intellectual Property Control Corp. | Method for creating association index for the analysis of documents classified in a hierarchical structure using frequency distribution, a taxonomic structure, normalizing and weighting |
-
2011
- 2011-06-16 US US13/824,740 patent/US9600565B2/en active Active
- 2011-06-16 JP JP2012538590A patent/JP5812007B2/ja active Active
- 2011-06-16 WO PCT/JP2011/063792 patent/WO2012049883A1/ja active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10301959A (ja) * | 1997-02-28 | 1998-11-13 | Fujitsu Ltd | データ圧縮/復元装置およびデータ圧縮/復元方法 |
JP2001216314A (ja) * | 2000-02-01 | 2001-08-10 | Nippon Telegr & Teleph Corp <Ntt> | 時空間に適応する情報検索方法及び装置及び時空間に適応する情報検索プログラムを格納した記憶媒体 |
JP2007102786A (ja) * | 2005-09-30 | 2007-04-19 | Internatl Business Mach Corp <Ibm> | 大規模フルテキスト・インデックスでタキソノミーにインデックスを付け、探索することをサポートする方法、装置およびシステム |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2021517306A (ja) * | 2017-03-15 | 2021-07-15 | センシェア ゲーエムベーハー | データベースにおけるトライデータ構造の有効使用 |
US11275740B2 (en) | 2017-03-15 | 2022-03-15 | Censhare Gmbh | Efficient use of trie data structure in databases |
US11347741B2 (en) | 2017-03-15 | 2022-05-31 | Censhare Gmbh | Efficient use of TRIE data structure in databases |
JP7364581B2 (ja) | 2017-03-15 | 2023-10-18 | センシェア ゲーエムベーハー | データベースにおけるトライデータ構造の有効使用 |
US11899667B2 (en) | 2017-03-15 | 2024-02-13 | Censhare Gmbh | Efficient use of trie data structure in databases |
Also Published As
Publication number | Publication date |
---|---|
US9600565B2 (en) | 2017-03-21 |
JPWO2012049883A1 (ja) | 2014-02-24 |
US20130262470A1 (en) | 2013-10-03 |
JP5812007B2 (ja) | 2015-11-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP3849279B2 (ja) | インデクス作成方法および検索方法 | |
JP4848317B2 (ja) | データベースのインデックス作成システム、方法及びプログラム | |
US7962524B2 (en) | Computer program, device, and method for sorting dataset records into groups according to frequent tree | |
JP3566111B2 (ja) | 記号辞書作成方法及び記号辞書検索方法 | |
JP5138046B2 (ja) | 検索システム、検索方法およびプログラム | |
US8193954B2 (en) | Computer product, information processing apparatus, and information search apparatus | |
CN101809567A (zh) | 文本串的二次散列提取 | |
JP2005025763A (ja) | 構造化文書の分割プログラム、分割装置、及び分割方法 | |
Gog et al. | Fast and lightweight LCP-array construction algorithms | |
US20140229484A1 (en) | Extraction method, computer product, extracting apparatus, and extracting system | |
US6735600B1 (en) | Editing protocol for flexible search engines | |
JP5812007B2 (ja) | インデックス作成装置、データ検索装置、インデックス作成方法、データ検索方法、インデックス作成プログラムおよびデータ検索プログラム | |
Amir et al. | Managing unbounded-length keys in comparison-driven data structures with applications to online indexing | |
CN111562920A (zh) | 小程序代码相似度确定方法、装置、服务器及存储介质 | |
JP2019204246A (ja) | 学習データ作成方法及び学習データ作成装置 | |
JP6428615B2 (ja) | 多次元範囲検索装置及び多次元範囲検索方法 | |
Chien et al. | Geometric BWT: compressed text indexing via sparse suffixes and range searching | |
Ahmed et al. | ISSP-tree: an improved fast algorithm for constructing a complete prefix tree using single database scan | |
JP3859044B2 (ja) | インデクス作成方法および検索方法 | |
JPWO2018012413A1 (ja) | 類似データ検索装置、類似データ検索方法および記録媒体 | |
JP6123372B2 (ja) | 情報処理システム、名寄せ判定方法及びプログラム | |
JP5867208B2 (ja) | データモデル変換プログラム、データモデル変換方法およびデータモデル変換装置 | |
JP6802109B2 (ja) | ソフトウェア仕様分析装置、及びソフトウェア仕様分析方法 | |
JP2020135530A (ja) | データ管理装置、データ検索方法及びプログラム | |
JP5582358B2 (ja) | 文書検索システム、文書検索方法、及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11832325 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2012538590 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13824740 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11832325 Country of ref document: EP Kind code of ref document: A1 |