JP5601121B2 - Transposed index generation method and generation apparatus for N-gram search, search method and search apparatus using the inverted index, and computer program - Google Patents

Description

  The present invention relates to N-gram search, and more particularly to a transposed index generation method and generation apparatus for N-gram search, a search method and search apparatus using the transposed index, and a computer program.

  With the increasing digitization of documents, the importance of search technology that finds a desired document from a large number of document groups accumulated so far has increased.

  In many languages such as English, it is common to create an index file by using a word as an index unit and implement high-speed search processing using the index file. However, in Japanese, word breaks are not explicitly indicated by spaces or the like, and therefore, a method using N-grams as index units is often used.

  An N-gram is a partial character string composed of consecutive N characters. The creation of an N-gram index file (hereinafter referred to as a transposed index) is based only on a character string, so that it is not necessary to recognize a word. However, since the search term to be searched is divided and processed into a plurality of N-grams, there is a problem that the search time increases when the search processing is performed with a long search term.

  In order to solve such a problem, Non-Patent Document 1 discloses a technique for speeding up search processing. Specifically, in Non-Patent Document 1, the sum of the N-gram document frequencies is calculated as an estimated value for speeding up the processing, and is used to select N-grams that are actually used for document search processing. Speed up.

Yasuaki Ogawa, Toru Matsuda, “Efficient Document Retrieval Method Using n-gram Index”, IEICE Transactions (DI), Vol.J82-DI, No.1, pp.121-129, 1999 January

  In such a search process using N-grams, there is a demand for speeding up by a simpler process. However, the configuration disclosed in Patent Document 1 has a problem that the search time becomes longer as the amount of data of words and documents to be searched increases. For example, in an environment where usable resources are limited, such as a small electronic device such as a mobile phone or an electronic dictionary, the performance of the device is limited, so that the search time may be long. Therefore, there is a demand for a new method for performing high-speed search processing by processing search efficiently.

  The present invention is to solve the above-described problems, and a transposed index generation method and a generation apparatus suitable for efficiently narrowing down search target words and the like, and the transposed index It is an object to provide a search method and a search apparatus using a computer, and a computer program.

In order to achieve the above object, a transposed index generation method according to the first aspect of the present invention includes:
A method for generating an inverted index in an information processing device,
The information processing apparatus generates a transposing index for the plurality of document data stored in the storage unit, and a storage unit that stores a plurality of document data each composed of an entry word and a corresponding explanatory sentence Control means to
The control means is
For each of a plurality of document data each composed of the headword and the explanatory text corresponding to the headword stored in the storage means, the number of characters of the headword and the explanatory text is counted, An order change step for changing the order of a plurality of document data;
A generation step of generating a transposed index by associating appearance positions in a plurality of document data in which the order is changed for each of “N-grams (N is a natural number)” that is a character string of N characters;
It is characterized by performing .

The control means includes
Document data including character strings of headwords and explanations of each of the plurality of document data in which the order is changed is extracted, and character strings are included in the extracted document data and the extracted document data. Moreover perform the document data, the correspondence step for associating inclusion relation,
In the generation step executed by the control unit , the inclusion relation is further stored, and the transposed index is generated.
And this is desirable.

In the generating step executed by the control means , the transposition index is generated by further storing the correspondence relationship between the plurality of document data whose order has been changed and the plurality of document data before the order has been changed. To
And this is desirable.

In order to achieve the above object, a search method according to the second aspect of the present invention includes:
Replaced document data storage means for storing a plurality of document data whose order has been changed in the order changing step of the above transposed index generation method,
An inverted index storage means for storing the inverted index generated in the generating step of the above-described inverted index generating method;
A search method in an information processing apparatus having control means for searching for the document data using the transposed index,
The control means includes
N-gram extraction step for extracting N-gram from the search character string;
From the inverted index, the N-gram extraction to get the appearance position associated with the N-gram extracted in step, on the basis of the occurrence position, the search character from among the plurality of document data in which the sequence has been replaced A document identification step for identifying document data including columns;
It is characterized by performing .

In order to achieve the above object, a search method according to the third aspect of the present invention includes:
Replaced document data storage means for storing a plurality of document data whose order has been changed in the order changing step of the above transposed index generation method,
An inverted index storage means for storing the inverted index generated in the generating step of the above-described inverted index generating method;
A search method in an information processing apparatus having control means for searching for the document data using the transposed index,
The control means includes
N-gram extraction step for extracting N-gram from the search character string;
From the inverted index, the N-gram extraction to get the appearance position associated with the N-gram extracted in step, on the basis of the occurrence position, the search character from among the plurality of document data in which the sequence has been replaced A document specifying step of specifying document data including a column, and further specifying document data associated with the specified document data based on the stored inclusion relationship;
It is characterized by performing .

In order to achieve the above object, a search method according to the fourth aspect of the present invention includes:
Replaced document data storage means for storing a plurality of document data whose order has been changed in the order changing step of the above transposed index generation method,
An inverted index storage means for storing the inverted index generated in the generating step of the above-described inverted index generating method;
A search method in an information processing apparatus having control means for searching for the document data using the transposed index,
The control means includes
N-gram extraction step for extracting N-gram from the search character string;
From the inverted index, the N obtains the occurrence position associated with the N-gram extracted in grams extraction step, and the occurrence position, the sequence plurality of document data and the order before it is replaced, which is replaced Based on the correspondence relationship with a plurality of document data, the document data including the search character string is identified from the plurality of document data before the order is changed, and based on the stored inclusion relationship, A document specifying step for further specifying the document data associated with the specified document data;
It is characterized by performing .

In order to achieve the above object, an inverted index generation device according to a fifth aspect of the present invention provides:
For each of a plurality of document data composed of a headword and a corresponding explanatory text, the number of characters between the headword and the explanatory text is counted, and the order of the plurality of document data is changed in the order of the smaller number of characters. Replacement means,
Generating means for generating an inverted index by associating appearance positions in a plurality of document data in which the order is changed for each of “N-grams (N is a natural number)” that is a character string of N characters;
It is characterized by providing.

In order to achieve the above object, a search device according to the sixth aspect of the present invention provides:
N-gram extracting means for extracting N-gram from the search character string;
The appearance positions associated with the N-grams extracted in the N-gram extraction step are acquired from the transposed index generated by the above generation method, and the plurality of documents in which the order is changed based on the appearance positions Document specifying means for specifying document data including the search character string from the data;
It is characterized by providing.

In order to achieve the above object, a computer program according to the seventh aspect of the present invention provides:
Computer
For each of a plurality of document data composed of a headword and a corresponding explanatory text, the number of characters between the headword and the explanatory text is counted, and the order of the plurality of document data is changed in the order of the smaller number of characters. Replacement means,
Generating means for generating a transposed index by associating appearance positions in a plurality of document data in which the order is changed with respect to each of “N-grams (N is a natural number)” that is a character string of N characters;
To function as.

In order to achieve the above object, a computer program according to the eighth aspect of the present invention provides:
Computer
N-gram extraction means for extracting N-gram from the search character string;
The appearance positions associated with the N-grams extracted in the N-gram extraction step are acquired from the transposed index generated by the above generation method, and the plurality of documents in which the order is changed based on the appearance positions Document specifying means for specifying document data including the search character string from the data;
To function as.

  According to the present invention, a transposed index generation method and a generation apparatus suitable for efficiently narrowing down a word or the like to be searched, a search method and a search apparatus using the transposed index, and a computer A program can be provided.

It is a figure which shows one example of schematic structure of the production | generation apparatus which produces | generates the transposed index which concerns on this invention. It is a figure which shows one example of schematic structure of the search device carrying the transposition index concerning this invention. It is a flowchart which shows the flow of the production | generation process of an inverted index. It is a figure which shows the example of the document data which replaced the order in order with few characters. It is a figure which shows the example of the document data which matched the replacement | exchange document data in which a character string corresponds. It is a figure which shows the specific structure of an inverted index. It is a flowchart which shows the flow of the search process of a search device. It is a figure which shows the example of the document data which provided the position pointer and the number pointer. It is a figure which shows the example of the document data which matched the replacement | exchange document data in which a character string corresponds in multiple layers. It is a figure which shows another example of schematic structure of the production | generation apparatus which produces | generates the transposed index which concerns on this invention. It is a figure which shows another example of the outline | summary structure of the search device carrying the transposition index concerning this invention.

  Hereinafter, a transposed index generation method and generation apparatus, a search method using the transposed index, and a search apparatus according to an embodiment of the present invention will be described. The embodiments described below are for illustrative purposes and do not limit the scope of the present invention.

  In the present embodiment, the computer apparatus is configured as a transposed index generating apparatus as shown in FIG. Moreover, the generating apparatus 10 shown in FIG. 1 realizes the transposed index generating method according to the present embodiment.

  The generation device 10 includes a central processing unit (CPU) 11, a read only memory (ROM) 12, a random access memory (RAM) 13, a hard disk drive (HDD) 14, an input device 15, an output device 16, and a communication control device 17. Composed. Each component is connected to each other by a system bus which is a transmission path for transferring commands and data.

The CPU 11 controls the operation of the entire generation apparatus 10 and is connected to each component to exchange control signals and data.
The ROM 12 stores computer programs and various data necessary for operation control of the entire generation apparatus 10. The CPU 11 operates by a computer program stored in the ROM 12 and executes various controls.
The RAM 13 is for temporarily storing data and computer programs, and holds computer programs and data read from the ROM 12 and other data necessary for the progress of processing.
The HDD 14 stores data necessary for the operation of the inverted index generation process. The HDD 14 stores a plurality of document data 18. The generation device 10 generates an inverted index based on the plurality of document data 18.
The input device 15 is configured by a keyboard, a touch panel, or the like, for example, and accepts various inputs from the user.
The output device 16 is configured by a display, for example, and outputs various processing results of the generation device 10.
The communication control device 17 is used for connecting the generation device 10 to a computer communication network such as the Internet, and is necessary when connecting to the computer communication network to exchange data.

  In the present embodiment, the generation device 10 includes order changing means, association means, and generation means. These are realized by the above-described CPU 11 interacting with the outside using the input device 15, the output device 16, and the communication control device 17 while accessing the data stored in the HDD 14 in cooperation with the ROM 12 and the RAM 13. Is done.

  Specifically, the order changing unit of the generation device 10 counts the number of characters of the headword and the explanatory text for each of a plurality of document data 18 each composed of the headword and the corresponding explanatory text, The order of the plurality of document data 18 is changed in ascending order of the number of characters.

  Here, the headword is a word or document to be searched. An explanatory sentence is a document in which the meaning of a headword is explained. For example, when the headword is the word “memory”, the explanatory text is, for example, “(1) remember or remember things without forgetting. (2) The past influence remains on the organism. (3) Retain the contents of past experiences and remember them later. ”, The headword and the explanatory text are associated with each other and stored as one document data 18 in, for example, the HDD 14.

  The order changing means counts the number of characters in the character string for each document data 18 composed of such headwords and explanations by the function of the CPU 11 or the like. Further, the order changing means changes the order of the document data 18 by changing the order of the document data 18 in the order of the number of characters of the document data 18 among the plurality of document data 18 stored in the HDD 14 by the function of the CPU 11 or the like. A plurality of replacement document data 19 is stored in, for example, the RAM 12 and held.

  In addition, the associating unit of the generation apparatus 10 extracts replacement document data 19 including character strings of headwords and explanatory texts of the plurality of replacement document data 19 in which the order is changed, and the extracted document data 19 is extracted. The inclusion relationship between the replacement document data 19 and the replacement document data 19 in which a character string is included in the extracted replacement document data 19 is associated.

  The associating means extracts a plurality of replacement document data 19 including a predetermined headword represented by the replacement document data 19 and a character string of a predetermined explanatory text by a function of the CPU 11 or the like. Here, as a specific example, consider a case where there are three headwords and three explanatory texts corresponding to the three headwords as the plurality of replacement document data 19. Document data A (headword: “diagram”, explanatory text corresponding to the headword: “diamond”), document data B (headword: “diagram”, explanatory text corresponding to the headword: “train operation table, Also, schedules of buses and airplanes, schedules, train schedules ”), document data C (headword“ memory ”, explanation corresponding to the headword:“ remember or remember things ” ")". In this case, the replacement document data A includes a character string “diagram”, and the replacement document data B also includes character strings “diagram”, “diagram”, and “train diagram”. Are in a relationship including the replacement document data A. On the other hand, since the replacement document data C does not have the character string “diamond”, the replacement document data C does not include the replacement document data A and the replacement document data B.

  That is, the associating means determines whether or not there is a character string of another headword or other explanatory text that matches the character string of the headword or explanatory text, so that the replacement document including the character string is determined. Data 19 is specified. If there is replacement document data 19 including a character string, the association unit associates the replacement document data 19 with each other and stores the associated inclusion relation in, for example, the RAM 12 or the like and holds the relation. .

  Further, the generation unit of the generation apparatus 10 associates the appearance positions in the plurality of replacement document data 19 with respect to each of “N gram (N is a natural number) that is a character string of N characters”, and the association unit Further, the inclusive relationship associated with is stored and a transposed index is generated. The generated inverted index is output to the HDD 14 or via the output device 16 or the communication control device 17.

Specifically, when one document data is composed of a character string of N _doc characters, the generation unit extracts N _doc −N + 1 N-grams (N character strings), and further, a plurality of documents. N-grams are similarly extracted from the data, and a transposed index describing each appearance position is generated for N-grams of the same pattern. Further, the generation unit similarly extracts N-grams from the document data associated by the association unit, and generates a transposed index describing each appearance position for the N-grams of the same pattern. The generated transposed index is stored and saved in, for example, the HDD 14.

  The inverted index generated by such a generation device 10 is mounted on the search device and used for search processing. In the present embodiment, the computer device is configured as a search device as shown in FIG. Also, the search apparatus 20 shown in FIG. 2 implements the inverted index search method according to the present embodiment.

  The search device 20 includes a CPU 21, ROM 22, RAM 23, HDD 24, input device 25, output device 26, and communication control device 27. Each component is connected to each other by a system bus which is a transmission path for transferring commands and data.

  These constituent elements are basically equivalent to the constituent elements of the generation apparatus 10 shown in FIG. That is, in FIG. 1, each component that functions to generate a transposed index from the document data 18 functions to perform a search process using the generated transposed index here.

That is, the CPU 21 controls the overall operation of the search device 20 and is connected to each component to exchange control signals and data.
The ROM 22 stores computer programs and various data necessary for operation control of the entire search device 20. The CPU 11 operates by a computer program stored in the ROM 12 and executes various controls.
The RAM 23 is for temporarily storing data and computer programs, and holds computer programs and data read from the ROM 22 and other data necessary for the progress of processing.
The HDD 24 stores data and the like necessary for the search processing operation. The HDD 24 stores a transposed index 30 generated by the generation apparatus 10 and a plurality of replacement document data 19 in which the order is changed from the plurality of document data 18 at that time. Based on the transposed index 30, the search device 20 specifies in which replacement document data 19 a plurality of replacement document data 19 the search character string designated by the user appears.
The input device 25 is configured by a keyboard, a touch panel, or the like, for example, and accepts various inputs from the user.
The output device 26 is configured by a display or the like, for example, and outputs various processing results of the search device 20.
The communication control device 27 is for connecting the search device 20 to a computer communication network such as the Internet, and is necessary when connecting to the computer communication network to exchange data.

  In the present embodiment, the search device 20 includes N-gram extraction means and document identification means. These are realized by the above-described CPU 11 interacting with the outside using the input device 15, the output device 16, and the communication control device 17 while accessing the data stored in the HDD 14 in cooperation with the ROM 12 and the RAM 13. Is done.

  Specifically, the N-gram extraction means of the search device 20 extracts N-gram from the search character string. That is, for example, the input device 25 of the search device 20 receives the search character string input by the user. Then, the N-gram extracting means extracts an extractable one of N-grams constituting the search character string by the CPU 21 of the search device 20 or the like. Specifically, when the user inputs a search character string of M characters, the N-gram extraction unit 13 extracts all N-grams (N character strings) that can be extracted from the search character string. That is, when M is larger than N, M−N + 1 N-grams are extracted.

  Then, the document specifying unit of the search device 20 acquires the appearance position associated with the N-gram extracted by the N-gram extracting unit from the transposed index 30, and based on the appearance position, a plurality of replacement document data. The replacement document data 19 including the search character string is specified from 19. Further, the document specifying unit specifies the replacement document data 19 associated with the specified replacement document data 19. The specified replacement document data 19 is output via the output device 26 and the communication control device 27.

  That is, since the transposed index 30 generated by the generation device 10 is associated with document data including a character string, the document specifying unit can replace one replacement document data among a plurality of replacement document data 19. When 19 is specified, the replacement document data 19 associated with the replacement document data 19 is further specified.

  Details of the flow of processing in the generation device 10 and the search device 20 realized by such a configuration will be described. Here, first, with reference to FIG. 3, the process of generating the inverted index 30 will be described with reference to a flowchart.

  For example, the generation device 10 receives the instruction for starting the generation process from the user via the input device 15, and starts the generation process of the transposed index 30. When the generation process of the transposed index 30 is started, the order changing unit of the generation apparatus 10 counts the number of characters in the character string for each of the plurality of document data 18 stored in the HDD 14 by the function of the CPU 11 ( Step S101). Here, the document data 18 is composed of a headword and an explanatory text of the headword. The order changing means counts the number of characters in the character string of the headword and the character string of the explanatory text in order to change the order of the plurality of document data 18 according to the number of characters of the document data 18. For example, when the number of characters in the headword is 5 and the number of characters in the explanatory text is 15, the order changing unit counts the number of characters in the document data 18 as 20 (= 5 + 15). Since the number of characters is counted for each of the plurality of document data 18 by the order changing means, the number of characters for each document data 18 is derived.

  Next, the order changing means of the generating apparatus 10 changes the order of the plurality of document data 18 in ascending order of the number of characters counted by the order changing means by the function of the CPU 11 (step S102). More specifically, as shown in FIG. 4, the plurality of document data 18 is initially written in the order of headwords, such as headword 1, headword 2, headword 3, and so on. Are arranged regardless of the number of characters of the document data 18. For example, the document data 18 of the headword 1 (30 characters) with the larger number of characters is positioned ahead of the document data 18 of the headword 2 (15 characters) with the smaller number of characters.

  In this embodiment, the order changing means rearranges the plurality of document data 18 from the state in the ascending order of the number of characters to obtain the plurality of replacement document data 19. More specifically, among the plurality of document data 18, the document data 18 of the entry word Z having the smallest number of characters is placed at the beginning, and the document data 18 of entry word 3 having the largest number of characters is placed at the end. In addition, since the document data 18 of the headword 1 that was initially at the head has the fifth smallest number of characters among the plurality of document data 18, it is arranged fifth in the plurality of replacement document data 19.

  As described above, the order changing unit generates the exchange document data 19 by changing the order of the document data 18 in the order of the smallest number of characters among the plurality of document data 18 counted for each document data 18. To do. The order changing unit stores the replacement document data 19 in the RAM 13 or the HDD 14.

  Next, the associating means of the generating apparatus 10 pays attention to the first replacement document data 19 by the function of the CPU 21 (step S103). Then, the associating means includes, among the plurality of replacement document data 19, replacement document data 19 that includes character strings of headwords and explanatory texts constituting the focused excess demarcation data 19. It is determined whether or not there is (step S104). For example, when the character string of the headword in the replacement document data 19 of interest is “AB” and the character string of the corresponding explanatory text is “CDE”, the associating means determines that the character string “AB” It is determined whether there is any other replacement document data 19 that includes the character string “CDE”. As the replacement document data 19 including such a character string, for example, replacement document data 19 including a character string “ABCDE”, “ABXYZCDE”, and “CDEAB” in an entry word or an explanation corresponds.

  When it is determined that there is a plurality of replacement document data 19 that includes the character string of the replacement document data 19 of interest (step S104; YES), the associating means The replacement document data 19 including the character string is associated with the replacement document data 19, and the associated inclusion relation is held (step S105). More specifically, as shown in FIG. 5, for example, replacement document data 19 a having headword 20 is associated with four replacement document data 19 c to 19 f and replacement document data having headword 57. 19b is associated with two replacement document data 19g to 19h. The associating means retains the inclusion relationship thus associated in the RAM 13 or the like.

  On the other hand, when it is not determined that there is a plurality of replacement document data 19 that includes the character string of the focused replacement document data 19 (step S104; NO), the processing of the search device 20 is performed. It does not pass through step S105 mentioned above. That is, the above-described inclusion relationship is not maintained.

  Then, the associating unit of the search device 20 determines whether there is the next replacement document data 19 (step S106). That is, the associating means determines whether the replacement document data 19 currently focused on is the last replacement document data 19. If there is the next replacement document data 19 (step S106; YES), the associating means pays attention to the next replacement document data 19 (step S107), and then the process returns to step S104 again.

  Such processing in steps S104 to S107 is performed for each replacement document data 19. Then, until the next replacement document data 19 disappears, the replacement document data 19 including the character string of the replacement document data 19 focused in step S103 is determined. If included, the inclusion relationship is maintained. To do.

  Here, since the plurality of replacement document data 19 are replaced in ascending order of the number of characters, the replacement document data 19 having a larger number of characters than the number of characters of the replacement document data 19 focused in step S103 is obtained in steps S104 to S107. In this process, attention will be paid one after another. For this reason, it is possible to specify other replacement document data 19 including the character string of the replacement document data 19 focused in step S103 by simply paying attention to the plurality of replacement document data 19 sequentially from the top. .

  Next, the generation unit of the generation device 10 uses the function of the CPU 11 to display the appearance position in the replacement document data 19 for each of the N-grams extracted from the replacement document data 19 stored by being replaced in ascending order of the number of characters. And the inclusion relationship held in step S105 is stored, and the transposed index 30 is generated (step S108). The generated transposed index 30 is stored in the HDD 14 or is output via the output device 16 and the communication control device 17. Then, the search process ends.

  Hereinafter, a specific configuration of the transposed index 30 according to the present embodiment will be described with reference to FIG. As shown in the figure, the transposed index 30 includes a file (pattern.idx) in which an N-gram character string pattern and an appearance position information storage address are described, and a file (in which an appearance position for each N-gram character string pattern is described ( position.idx), a file (number.idx) in which the document number and the first character position of each document are described, and a file related to inclusion (relation.idx).

  Here, the appearance position is a position based on the first character position of the text in which the document group to be searched is arranged in the document number order. Similarly, the first character position of each document number in the figure is also a position based on the first character position of the text in which the document groups to be searched are arranged in document number order.

  The file (relation.idx) relating to the inclusion relationship describes the inclusion relationship held in step S105 of the generation process described above. Specifically, in FIG. 5, the replacement document data 19a of the headword 20 is associated with the replacement document data 19c to 19f of the headwords 45, 85, 456, and 775. Four inclusion document numbers 45, 85, 456, and 775 are associated with the document number 20 (headword 20). Similarly, two inclusion document numbers 203 and 360 are associated with the document number 57 (headword 57).

  The transposed index 30 created in step S108 is used for search processing performed by the search device 20 described later.

  Through the above processing, the generating apparatus 10 for the inverted index 30 in the present embodiment creates replacement document data 19 in which the order of the plurality of document data 18 is changed in ascending order of the number of characters for each document data 18, and the replacement document is created. An N-gram in the data 19 is associated with the appearance position in the replacement document data 19 to generate a transposed index 30. Further, the transposed index data 30 is generated by associating the replacement document data 19 having the relationship including the character string and further storing the inclusion relationship. Since the replacement document data 19 having the relationship including the character string is associated (linked), when the replacement document data 19 including the search character string is specified, the replacement document data 19 is linked to the replacement document data 19. The replacement document data 19 thus specified is also specified. This leads to efficient search processing described later.

  Next, a search process performed by the search device 20 according to the present embodiment will be described. FIG. 7 is a flowchart showing the flow of search processing.

  First, when the processing of the search device 20 is started, for example, the input device 25 of the search device 20 receives a search character string from the user (step S201).

  Next, the N-gram extraction means extracts N-grams from the search character string received in step S201 by the function of the CPU 21 (step S202). Here, the value of N is a value determined in advance in the search device 20 and takes a value of N = 2, N = 3, or other natural numbers. In the following, for the sake of explanation, N = 2 each time. A case where N = 3 or the like is used will be described.

  Specifically, it is assumed that the user inputs a 9-character search character string “accelerated full-text search process”. At this time, in the case of search processing with N = 2, the extracted N-grams (bigrams) are “high speed”, “speed-up”, “general text”, “full text”, “text check”, “search” in order from the front. ”,“ Search process ”, and“ process ”(9-2 + 1). Further, for example, in the case of a search process with N = 3, the extracted N-grams (trigrams) are “accelerated”, “accelerated all”, “according full sentence”, “full text check”, “sentence search” in order from the front. ”,“ Search process ”, and“ search process ”(9-3 + 1).

  Next, the document specifying means pays attention to the first replacement document data 19 by the function of the CPU 21 (step S203). Then, the document specifying unit determines whether or not the searched replacement document data 19 includes a search character string (step S204). Here, the transposed index 30 is used to determine whether or not the search character string is included in the replacement document data 19. Specifically, in step S202, the appearance position associated with each N-gram is acquired from the transposed index 30 using the N-gram extracted from the search character string.

  In step S204, the document specifying unit determines whether there is a continuous appearance position that constitutes the search character string among the appearance positions of the acquired N-gram, and the replacement in which the search character string is focused. It is determined whether it is included in the document data 19. Then, when the replacement character data 19 of interest includes a search character string (step S204; YES), the document specifying unit temporarily holds the replacement document data 19 in the RAM 13 or the like (step S205). And output to the user in later processing.

  On the other hand, if the focused replacement document data 19 does not contain a search character string (step S204; NO), the document specifying means has the replacement document data 19 next to the focused replacement document data 19. It is determined whether or not (step S208). Since the plurality of replacement document data 19 are replaced in ascending order of the number of characters, the document specifying unit pays attention to the replacement document data 19 in the order in which the number of characters gradually increases. Then, the document specifying unit determines whether the replacement document data 19 currently focused on is the last replacement document data 19.

  Next, the document specifying unit determines whether or not there is replacement document data 19 associated with the replacement document data 19 determined to include the search character string by the function of the CPU 21 (step S206).

  In the file (relation.idx) relating to the inclusion relation of the transposed index 30, the document numbers corresponding to the character strings (including the character strings) are associated with each other. For this reason, the document specifying unit determines whether there is an inclusion document number associated with the document number of the replacement document data 19 determined to include the search character string. When there is an associated document number associated with the document, the document specifying unit corresponds to the replacement document data 19 in which the replacement document data 19 with the included document number is determined to include the search character string. Specify that the document data is attached.

  If there is the replacement document data 19 to be associated (step S206; YES), the document specifying unit determines in step S204 all the replacement document data 19 determined to include the search character string. The replacement document data 19 is temporarily stored in the RAM 13 or the like (step S207).

  On the other hand, when there is no associated replacement document data 19 (step S206; NO), the document specifying unit determines whether there is replacement document data 19 next to the focused replacement document data 19. (Step S208). If there is the next replacement document data 19 (step S208; YES), the document specifying means determines that the next replacement document data 19 is determined to be associated in step S204. It is determined whether or not 19 (step S209).

  When the next replacement document data 19 is the replacement document data 19 determined to be associated (step S209; YES), the document specifying unit does not pay attention to the next replacement document data 19. In step S210, it is determined whether or not there is the next replacement document data 19 of the next replacement document data 19 (step S208). This is because the replacement document data 19 determined to be associated in step S204 includes the character string of the replacement document data determined to include the search character string. This is because it is not necessary to determine again whether or not it is included. For this reason, by performing the processing of steps S208 to S210, the number of times of determining whether or not the search character string is included in the replacement document data 19 can be reduced.

  If the next replacement document data 19 is not the replacement document data 19 determined to be associated (step S209; NO), the document specifying unit pays attention to the next replacement document data 19 ( Step S211), and then the process returns to Step S204 again.

  If there is no next replacement document data 19 (step S208; NO), the document specifying unit determines whether or not the search character string is included in all the replacement document data 19, and the above-described steps S204 to S204 are performed. Exit the repeat process of S211. Then, the document specifying unit outputs the replacement document data 19 including the search character string held in steps S205 and S207 to the user (step S212). That is, the replacement document data 19 specified as including the search character string in the process of step S204 and all replacements determined to be associated with the replacement document data 19 in the process of step S206. Document data 19 is output. Thereafter, the search process is terminated.

  Such processing in steps S204 to S211 is performed for each replacement document data 19. When there is replacement document data 19 associated with the replacement document data 19 determined to include the search character string, one replacement document data 19 including the search character string is first identified, and the replacement document data 19 is identified. The replacement document data 19 associated with the document data 19 is further specified. Therefore, the replacement document data 19 including the search character string can be specified at a time.

  In step S212, if there is no replacement document data 19 specified to include the search character string, the document specifying means does not output any replacement document data 19, and typically "search character string" Is not found "is output to the user and the process is terminated.

  As described above, when one replacement document data 19 including a search character string is identified from among a plurality of replacement document data 19, the search device 20 according to the present embodiment corresponds to the replacement document data 19. The attached replacement document data 19 with a matching character string (including the character string) can be specified at the same time. Thereby, it is possible to reduce the number of times of processing for determining whether or not a search character string is included in the plurality of replacement document data 19 and to perform search processing efficiently. For this reason, this embodiment is particularly useful in an environment where the resources that can be used are limited in small electronic devices such as mobile phones and electronic dictionaries.

  In addition, this invention is not limited to said embodiment, A various deformation | transformation and application are possible.

  For example, in the present embodiment, when the order changing unit creates the replacement document data 19, a number may be attached so that the correspondence with the state before the replacement can be recognized. In FIG. 8, a heading number is assigned to each of the plurality of document data 18 before the rearrangement in ascending order, and in this state, the document data 18 is replaced in the descending order of the number of characters and the replacement document data 19 is created. Show. Here, the heading numbers that were initially arranged in ascending order are rearranged in the replacement document data 19 so as to be separated. On the other hand, in FIG. 8, the heading numbers after replacement are assigned in ascending order.

  The generating unit stores the post-replacement heading number assigned in this way and the pre-replacement heading number in association with each other, for example, in the transposed index 30, so that each replacement document data 19 is replaced. It becomes possible to attach the order in which they were arranged before.

  Thus, even when the document specifying means of the search device 20 specifies the replacement document data 19 including the search character string based on the transposed index 30 generated from the plurality of replacement document data 19, a plurality of input documents Not only which replacement document data 19 is specified in the replacement document data 19 but also which document data 18 is specified in the plurality of document data 18 arranged in the order before the replacement. It becomes possible to do.

  The association unit can also associate the replacement document data 19 with each other so that the correspondence structure between the replacement document data has a multilayer structure. In FIG. 9, the replacement document data 19c (headword 45) is associated with the replacement document data 19a (headword 20), and the replacement document data 19c (headword 45) A state in which three replacement document data 19i to 19k are associated with each other is shown. Two replacement document data 19l to 19m are associated with the replacement document data 19f (headword 775), and replacement document data 19n is associated with the replacement document data 19l. ing. As described above, the association unit can also associate the replacement document data 19 having matching character strings in multiple layers. As a result, when one replacement document data 19 is specified, a plurality of replacement document data 19 associated with the replacement document data 19 can be specified. Can do.

  Further, in step S108, the transposed index 30 from which symbols and special characters such as “&”, “*”, “+” are removed can be generated. Since the symbol or the like is rarely input from the user as a search character string, a transposed index 30 from which the symbol or the like has been removed is generated, and a search process is performed based on the transposed index, so that the word to be searched Etc. can be narrowed down efficiently. Further, in the transposed index 30 from which symbols and the like are removed, since the ratio of the constituent elements indicating the correspondence relationship in which the replacement document data 19 that includes the character string is associated with each other is relatively large, an efficient search is performed. It can be performed.

  Further, the search character string accepted from the user may be one word, or may be a plurality of search character strings. When a plurality of search character strings are accepted, there are types of calculation methods such as logical product, logical sum, and negative logical product of the plurality of search character strings, and the search may be performed by any of the calculation methods.

  Further, the constituent elements of the document data 18 are not limited to headwords and explanatory texts. For example, the document data 18 may be composed of a headword, an explanation, a drawing in which the headword is explained, a headword having the opposite meaning that the headword means, and the like.

  And the component of the transposition index 30 is not restricted to a component as shown in the said FIG. For example, the appearance frequency in the plurality of document data 18 to be searched for the extracted N-gram may be further used as a constituent element. In this case, the document specifying unit can specify the document data 18 including the search character string more efficiently by using the appearance frequency information.

  Further, the order changing means is not limited to counting the number of characters when changing the order of the document data 18, and can also measure the data amount of the document data 18 composed of headwords and explanatory texts. The order changing means can also create the replacement document data 19 by changing the order of the plurality of document data 18 in the order of the measured data amount.

  In the generation apparatus 10 according to the present embodiment, the document data 18 is not limited to exist in the generation apparatus 10 by being stored in the HDD 14 as shown in FIG. That is, for example, as shown in FIG. 11, the document data 18 may exist on the Internet instead of in the generation device 10 and can be acquired via the communication control device 17.

  Further, in the search device 20 in the present embodiment, the document data 18 is limited to exist in the search device 20 by being stored in the HDD 14 as shown in FIG. I can't. That is, for example, as shown in FIG. 11, the document data 18 may exist on the Internet, not in the search device 20, and may be acquired via the communication control device 17.

  By adopting such a configuration, in the embodiment of FIG. 11, it is not necessary to store the document data 18 in the search device 20 as compared with the one in FIG. It becomes easy to realize even in a device having a limited capacity such as a small electronic dictionary.

  Further, the embodiment of the present invention may be a computer program for causing a computer device to function as the generating device 10 in addition to the above-described embodiment. Moreover, the computer program for functioning a computer apparatus as the said search apparatus 20 may be sufficient.

  The computer program can be stored in a computer-readable information storage medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  Further, the computer program can be distributed and sold via a computer communication network independently of a computer device on which the computer program is executed. The information storage medium can be distributed and sold independently of the computer device.

  DESCRIPTION OF SYMBOLS 10 ... Generating device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... HDD, 15 ... Input device, 16 ... Output device, 17 ... Communication control device, 18 ... Document data, 19 ... Replacement document data, 20 ... Search device, 21 ... CPU, 22 ... ROM, 23 ... RAM, 24 ... HDD, 25 ... Input device, 26 ... Output device, 27 ... Communication control device, 30 ... Transposition index

Claims (10)

A method for generating an inverted index in an information processing device,
The information processing apparatus generates a transposing index for the plurality of document data stored in the storage unit, and a storage unit that stores a plurality of document data each composed of an entry word and a corresponding explanatory sentence Control means to
The control means is
For each of a plurality of document data each composed of the headword and the explanatory text corresponding to the headword stored in the storage means, the number of characters of the headword and the explanatory text is counted, An order change step for changing the order of a plurality of document data;
A generation step of generating a transposed index by associating appearance positions in a plurality of document data in which the order is changed for each of “N-grams (N is a natural number)” that is a character string of N characters;
Method for generating an inverted index, which comprises the run. The control means includes
Document data including character strings of headwords and explanations of each of the plurality of document data in which the order is changed is extracted, and character strings are included in the extracted document data and the extracted document data. Moreover perform the document data, the correspondence step for associating inclusion relation,
In the generation step executed by the control unit , the inclusion relation is further stored, and the transposed index is generated.
Method for generating an inverted index of claim 1, wherein the this. In the generating step executed by the control means , the transposition index is generated by further storing the correspondence relationship between the plurality of document data whose order has been changed and the plurality of document data before the order has been changed. To
Method for generating an inverted index according to claim 1 or 2, characterized and this. Replaced document data storage means for storing a plurality of document data whose order is changed in the order changing step of the transposed index generation method according to claim 1;
Inverted index storage means for storing the inverted index generated in the generating step of the inverted index generating method according to claim 1;
A search method in an information processing apparatus having control means for searching for the document data using the transposed index,
The control means includes
N-gram extraction step for extracting N-gram from the search character string;
From the inverted index, the N-gram extraction to get the appearance position associated with the N-gram extracted in step, on the basis of the occurrence position, the search character from among the plurality of document data in which the sequence has been replaced A document identification step for identifying document data including columns;
The search method characterized by performing . Replaced document data storage means for storing a plurality of document data whose order has been changed in the order changing step of the transposed index generation method according to claim 2;
Inverted index storage means for storing the inverted index generated in the generating step of the inverted index generating method according to claim 2;
A search method in an information processing apparatus having control means for searching for the document data using the transposed index,
The control means includes
N-gram extraction step for extracting N-gram from the search character string;
From the inverted index, the N-gram extraction to get the appearance position associated with the N-gram extracted in step, on the basis of the occurrence position, the search character from among the plurality of document data in which the sequence has been replaced A document specifying step of specifying document data including a column, and further specifying document data associated with the specified document data based on the stored inclusion relationship;
The search method characterized by performing . Replaced document data storage means for storing a plurality of document data whose order has been changed in the order changing step of the transposed index generation method according to claim 3;
Transposed index storage means for storing the inverted index generated in the generating step of the inverted index generating method according to claim 3;
A search method in an information processing apparatus having control means for searching for the document data using the transposed index,
The control means includes
N-gram extraction step for extracting N-gram from the search character string;
From the inverted index, the N obtains the occurrence position associated with the N-gram extracted in grams extraction step, and the occurrence position, the sequence plurality of document data and the order before it is replaced, which is replaced Based on the correspondence relationship with a plurality of document data, the document data including the search character string is identified from the plurality of document data before the order is changed, and based on the stored inclusion relationship, A document specifying step for further specifying the document data associated with the specified document data;
The search method characterized by performing . For each of a plurality of document data composed of a headword and a corresponding explanatory text, the number of characters between the headword and the explanatory text is counted, and the order of the plurality of document data is changed in the order of the smaller number of characters. Exchange means;
Generating means for generating an inverted index by associating appearance positions in a plurality of document data in which the order is changed for each of “N-grams (N is a natural number)” that is a character string of N characters;
Generator inverted index, characterized in that it comprises a. N-gram extracting means for extracting N-gram from the search character string;
An appearance position associated with the N-gram extracted in the N-gram extraction step is acquired from the transposed index generated by the generation method according to claim 1, and the order is changed based on the appearance position. Document specifying means for specifying document data including the search character string from a plurality of document data;
Search apparatus comprising: a. Computer
For each of a plurality of document data composed of a headword and a corresponding explanatory text, the number of characters between the headword and the explanatory text is counted, and the order of the plurality of document data is changed in the order of the smaller number of characters. Replacement means,
Generating means for generating a transposed index by associating appearance positions in a plurality of document data in which the order is changed with respect to each of “N-grams (N is a natural number)” that is a character string of N characters;
Computer program for functioning as a. Computer
N-gram extraction means for extracting N-gram from the search character string;
An appearance position associated with the N-gram extracted in the N-gram extraction step is acquired from the transposed index generated by the generation method according to claim 1, and the order is changed based on the appearance position. Document specifying means for specifying document data including the search character string from a plurality of document data;
Computer program for functioning as a.

Images