JP2012003355A - Retrieval device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the size of a retrieval index without decreasing retrieval accuracy in the case of creating the retrieval index in response to a retrieval request for uniquely acquiring a specific position of a specific document.SOLUTION: A position where an index key is extracted is determined based on the feature of a character code expressing a character from the whole or partial area of an inputted document, and an index key configured of the combination of one or more characters lying at the position is extracted from the whole or partial area of the document to associate the index key with an appearance position in the document, where the index key appears, to output to an index DB. Furthermore, a partial area in a certain document is accepted as a search query, and from the search query, a position where the query key is extracted is determined based on the feature of the character code expressing the character and a query key configured of the combination of one or more characters is extracted. The index DB is searched based on the query key, and a search result is outputted.

Description

  The present invention relates to a search apparatus, method, and program, and in particular, a document in which a region appears and a position in the document using a captured image of a partial region in a document in which a page break or a line break position is determined as a search query. The present invention relates to a search apparatus, a method, and a program for creating a document and an index of each position in the document in response to a search request for acquiring a document.

  Specifically, this is applied to a document that may include the area in a document where the page break or line feed position has been determined, and a position in the document that is not comprehensively acquired, but is intended to uniquely identify the position. The present invention relates to a search apparatus, method, and program.

  There are not a few scenes in which it is necessary to uniquely identify from which document a part of a document is included in which document or at which position in which document.

  For example, if there is a magazine cut out at hand, there may be a case where the original cut out magazine is searched and the continuation of the cut out is read. In this case, it is necessary to uniquely identify which magazine the clipping was part of.

  The above example can be regarded as a search system that uses a partial area of a document as a query and inquires about a document name including the area or a document name and a position in the document from a huge amount of documents.

  In order to construct a system that responds to a search request for acquiring information from a document group, it is necessary to analyze the document group in advance and create a search index.

  For example, as shown in FIG. 25, a continuous character string of N characters appearing in a document is extracted, and the character string is used as a search index key, and the document name including the character string or the document name and the document There is a method in which the position where the character string appears is used as a search index value.

  In addition, the N-gram method has been recognized as useful in a wide range of situations, and many extension methods have been proposed even now. In addition to the normal N-gram method, a method of changing the value of N according to the situation has been implemented (see, for example, Non-Patent Document 1).

"A realization method of N-gram index using Unicode and its evaluation", Information Processing Society of Japan, 2000-NL-136-17, pp.135-142.

  However, when a query is made to a search index created by the above method using a partial region of a document as a query, it is difficult to reduce the index size without reducing the search accuracy.

  For example, as shown in FIG. 26, a two-page partial area of “document 1” is photographed, the photographed partial image is converted into partial text by OCR (optical character recognition), and a search key is extracted from the partial text. Consider a case in which a search query is made to the search index based on the key. Note that the key extraction method at the time of creating a search index and at the time of a search inquiry is a method of extracting a continuous character string of N characters as described above, and N = 2. Also, as shown in FIG. 27, the search query results of each search key are totaled to identify the original document name and the position in the original document with the largest number of cases.

  First, consider the case of creating a search index by performing key extraction while shifting character by character in the reading direction as shown in FIG. In this case, as shown in FIG. 28, inquiries including correct search results (in this case, two pages of “Document 1”) for all search keys extracted while shifting one character at a time from the upper left end of the partial text. Since the result is obtained, the search query results are aggregated to obtain the position in the document / document having the largest number of cases (in this case, “Document 1”, page 2), it is the correct position in the original document / original document. . However, since this method creates a search index by extracting keys while shifting character by character, there is a problem that the data amount of the search index becomes enormous, the search query speed decreases, and the search index storage hard disk capacity increases.

  On the other hand, in order to reduce the data amount of the search index, consider a case where a search index is created by performing key extraction while shifting by two characters in the reading direction as shown in FIG. In this case, as shown in FIG. 30, inquiries including correct search results (in this case, two pages of “Document 1”) for all search keys extracted while shifting one character at a time from the upper left end of the partial text. Results may not be obtained. In other words, among the keys used for search queries, “A”, “P”, “Den”, “Chi”, “Through”, and “Ji” are created as search indexes for two pages of Document 1. Therefore, the inquiry result of these keys does not include two pages of “document 1” which is a correct inquiry result. For this reason, when the search query results are aggregated to obtain the position in the document / document having the largest number (in this case, page 43 of “Document 5”), it may not be the correct position in the original document / original document. .

  At this time, a correct query result may be obtained if key extraction is performed while shifting by two characters in the reading direction from the upper left end of the partial text as shown in FIG. However, when key extraction is performed while shifting two characters from a partial text, a search query cannot always be made correctly as shown in FIG. That is, when an input of a partial area of the original document is used as an input, it is difficult to determine which area is to be captured, and a correct search cannot be performed simply by shifting the captured partial area by one character. End up. That is, as shown in FIG. 32, when the key is extracted from the upper left corner of the partial text shifted by one character from FIG. 31, the keys “A”, “P”, “Den”, “Chi” ”,“ Through ”, and“ Jiji ”are not created with search indexes for the two pages of“ Document 1 ”in the first place, so that the correct search result for the search key (in this case,“ Document 1 ”, page 2)) Inquiry results including are not obtained at all.

  As described above, when key extraction is performed while shifting two characters at a time in order to reduce the data amount of the search index (FIG. 29), the search cannot be performed correctly depending on the method of extracting keys from partial text (FIGS. 30 and 32). ), And the search accuracy is low. In this example, the key extraction is performed by shifting two characters at a time, but the problem is essentially the same when the keys are extracted by shifting M characters (M> 2).

  The present invention has been made in view of the above points, and at the time of creating a search index for responding to a search request for uniquely acquiring a specific position of a specific document from a document group, the search index is not reduced. It is an object of the present invention to provide a search apparatus, method, and program capable of reducing the size of a program.

In order to solve the above problems, the present invention (Claim 1) uses a partial area in a document in which a page break or a line break position is determined as a search query, and the document in which the area appears and the document A search device for creating a search index for responding to a search request for acquiring a position and performing a search,
A document input means for accepting input of documents to be indexed;
Index key extraction position determining means for determining a position for extracting an index key from the whole or a partial area of the document based on characteristics of a character code representing a character;
Index key extracting means for extracting an index key composed of a combination of one or more characters at the position from the whole or a partial area of the document;
Index output means for associating an appearance position in the document in which the index key appears with the index key and outputting it to the index storage means.

The present invention (Claim 2) is the index key extraction position determination means according to Claim 1,
Means for determining a position for extracting an index key composed of a combination of one or more characters from the whole or a partial area of the document based on the relationship between the characters and the character codes of the characters existing in the vicinity thereof;

Further, the present invention (Claim 3) includes query input means for accepting a partial area in a document as a search query,
Query key extraction position determination means for determining a position from which the query key is extracted from the search query based on the characteristics of the character code representing the character;
Query key extraction means for extracting a query key consisting of a combination of one or more characters from the search query;
Search means for searching the index storage means based on the query key and outputting the search result is further included.

Further, the present invention (Claim 4) is the query key extraction position determination means according to Claim 3,
Means for determining a position from which a query key consisting of a combination of one or more characters is extracted from the search query based on the relationship between the characters and the character codes of the characters existing in the vicinity thereof;

Further, the present invention (Claim 5) uses a partial area in a document in which a page break or a line break position is fixed as a search query, and makes a search request for acquiring a document in which the area appears and a position in the document. A search method for creating a search index and performing a search,
A document input step in which the input means receives input of a document to be indexed;
An index key extraction position determination means for determining an index key extraction position from the whole or a partial area of the document based on the characteristics of the character code representing the character;
An index key extracting unit that extracts an index key composed of a combination of one or more characters at the position from the whole or a partial area of the document; and
The index output means performs an index output step of associating the index key with an appearance position in the document in which the index key appears, and outputting to the index storage means.

Further, according to the present invention (Claim 6), in the index key extraction position determination step of Claim 5,
A position for extracting an index key composed of a combination of one or more characters from the whole or a partial area of the document is determined based on the relationship between the characters and the character codes of the characters existing in the vicinity.

Further, according to the present invention (claim 7), the query input means accepts a partial area in a document as a search query;
A query key extraction position determining means for determining a position from which the query key is extracted from the search query based on a characteristic of a character code representing the character;
A query key extracting means for extracting a query key comprising a combination of one or more characters from the search query;
The search means further performs a search step of searching the index storage means based on the query key and outputting the search result.

The present invention (Claim 8) is characterized in that in the query key extraction position determination step of Claim 7,
A position where a query key composed of a combination of one or more characters is extracted from the search query is determined based on the relationship between the characters and the character codes of the characters existing in the vicinity thereof.

  Moreover, this invention (Claim 9) is a program for functioning a computer as each means which comprises the search device of any one of Claim 1 thru | or 4.

  As described above, according to the present invention, when specifying the key extraction position, the specific position of the specific document is uniquely identified from the document group by using the query position and the language-independent information called the character code pattern. When creating a search index for responding to a search request acquired at the same time, it is possible to reduce the size of the search index without reducing the search accuracy.

  In particular, when the key extraction position is determined by the character code pattern, the search key is extracted from the position where the search index key is created, regardless of the query position (the position in the book taken in the first embodiment). Therefore, even if the search index is not comprehensively created in the book, the search can be executed with high accuracy.

  In addition, when the key extraction position is determined by the character code pattern, the effect of the present technique can be exhibited without worrying about differences in language characteristics in each country.

It is a block diagram of the server part in the 1st Embodiment of this invention. It is an example of the document in the 1st Embodiment of this invention. It is a flowchart of the search index creation process in the 1st Embodiment of this invention. It is an example of the correlation data in the 1st Embodiment of this invention. It is FIG. (1) which shows the key extraction position determination process in the 1st Embodiment of this invention. It is an example of the data of the key extraction position in the 1st Embodiment of this invention. It is an example of the extracted key in the 1st Embodiment of this invention. It is an example of search index DB in the 1st Embodiment of this invention. It is a flowchart of the search inquiry process in the 1st Embodiment of this invention. It is an example of the partial area image | photographed in the 1st Embodiment of this invention. It is FIG. (2) which shows the key extraction determination process in the 1st Embodiment of this invention. It is a figure which shows the key extraction position determined by the search inquiry creation process in the 1st Embodiment of this invention. It is an example of the key extracted by the search inquiry process in the 1st Embodiment of this invention. It is a totaling example of the search inquiry result in 1st Embodiment in this invention. It is an example of content DB in the 1st Embodiment of this invention. It is a display example of the Web browser in the 1st Embodiment of this invention. It is a flowchart of the search index creation process in the 2nd Embodiment of this invention. It is a figure which shows the key extraction position determination process at the time of the search index preparation in the 2nd Embodiment of this invention. It is a flowchart of the search inquiry process in the 2nd Embodiment of this invention. It is a figure which shows the determination process of the key extraction position at the time of the search inquiry preparation in the 2nd Embodiment of this invention. It is a flowchart of the search index creation process in the 3rd Embodiment of this invention. It is a figure which shows the key extraction position determination process at the time of the search index preparation in the 3rd Embodiment of this invention. It is a flowchart of the search inquiry process in the 3rd Embodiment of this invention. It is a figure which shows the key extraction position determination process at the time of the search inquiry in the 3rd Embodiment of this invention. It is an example of search index creation by a prior art. It is an example of the search inquiry by a prior art. It is a figure which shows the key extraction method at the time of the search inquiry by a prior art. In this example, key extraction is performed while shifting one character at a time in the reading direction according to the prior art. This is an example in which key extraction is performed while shifting two characters at a time for publicity reading according to the prior art. This is an example in which key extraction is performed while shifting one character at a time in the reading direction from the upper left corner of a partial text according to the prior art. This is an example (part 1) in which key extraction is performed while shifting by two characters in the reading direction from the upper left corner of a partial text according to the prior art. This is an example (part 2) in which key extraction is performed while shifting by two characters in the reading direction from the upper left corner of a partial text according to the prior art.

  Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1 shows the configuration of a server according to the first embodiment of the present invention.

  The server 300 shown in the figure is used as a search device of the present invention.

  The server unit 300 can be realized by a device such as a PC server, and includes a data input unit 301, a key extraction position determination unit 302, a key extraction unit 303, a search index output unit 304, a search index DB 305, a server side data transmission / reception unit 306, a search inquiry unit. 307 and content DB 308.

  The client unit 400 in FIG. 1 can be realized by a camera-equipped mobile phone or the like, and includes a document photographing unit 401, a client-side data transmission / reception unit 402, and a content display unit 403.

  The document reading device 200 in FIG. 1 is connected to the data input unit 301 and the server-side data transmission / reception unit 306 of the server unit 300 and is an external device such as a general scanner, and is a paper medium document on which characters are written. Is input, and the document is scanned and converted to an electronic image file as output. The optical character recognition device 201 is an external device such as general OCR software, which takes an image file containing characters as input, and outputs the converted character as electronic text data.

  A document 100 in the figure is one page of a paper medium book including sentences as shown in FIG. Note that FIG. 2 is an example of a page composed only of text, but the page may include information other than characters such as a figure and a table. Further, the document may be composed of a part of one page or may be composed of a plurality of pages.

  Hereinafter, processing in the above configuration will be described.

  The present invention is divided into search index creation processing and search query creation processing.

<Search index creation process>
FIG. 3 is a flowchart of search index creation processing according to the first embodiment of this invention.

  Step 1) Index creation input step: In the data input unit 301, an index creation document is input as text data.

    Step 10) The document reading apparatus 200 receives the document 100 as an input, and outputs a document 100 converted into an image file.

    Step 11) The optical character recognition apparatus 201 receives the output of Step 10 as an input, and outputs an image file converted into text data. The text data also holds the line feed position of the text in the image file.

    Step 12) The data input unit 301 accepts the document name of the document 100, the position in the document, and the output of step 11 as input, and outputs them to the key extraction position determination unit 302 in association with each other as shown in FIG. Here, the document name is a book name, and the position in the document is a page.

  Step 2) Index creation key extraction step: The key position extraction position determination unit 302 and the key extraction unit 303 extract a key from text data according to a certain rule.

    Step 13) The key extraction position determination unit 302 receives the output from the data input unit 301 in step 12 as an input, and determines the key extraction position. The key extraction position is determined using a character code pattern. Here, as shown in FIG. 5, each character of the original text data (FIG. 5A) is converted into a Unicode code (FIG. 5B), and “Character A code <appears to the right of character A”. Character A that is “character code” is set as the key extraction position (FIG. 5C), and the key extraction position (number of characters when counting from the upper left corner to the right, number of characters when counting from the upper left corner to the lower) The key extraction position information is added to the data shown in FIG. 4 and output as shown in FIG.

    Step 14) The key extraction unit 303 receives the output of Step 13 as an input, and performs key extraction based on a certain rule. Here, two characters obtained by concatenating the character at the key extraction position and the character to the right of the character are output as keys as shown in FIG. In FIG. 7, the extracted key, the document name from which the key is extracted, and the position (page) in the document are output to the search index output unit 304.

  Step 3) Key generation step during index creation: The extracted key is output as a search index.

    Step 15) The search index output unit 304 accepts the output of step 14 as input and stores it in the search index DB 305 in the format of FIG. Here, it is assumed that the processing of steps 10 to 15 is repeated for a plurality of documents, and the search index DB 305 stores data as shown in FIG.

<Search query creation process>
FIG. 9 is a flowchart of search query creation processing according to the first embodiment of this invention.

  Step 4) Inquiry input step: A document to be inquired is inputted as text data.

    Step 16) The document photographing unit 401 of the client unit 400 photographs a partial area of the document 100 and outputs it as an image file as shown in FIG.

    Step 17) The client-side data transmitting / receiving unit 402 accepts the image file of the partial area output in Step 16 as an input, and outputs the image file as it is to the server unit 300 through a network or the like.

    Step 18) In step 17, the server-side data transmission / reception unit 306 accepts the image file output from the client unit 400 as an input, and outputs the image file converted into text data using the optical character recognition device 201. The text data also holds the line feed position of the text in the image file.

  Step 5) Inquiry key extraction step: A key is extracted from text data according to a certain rule.

    Step 19) The key extraction position determination unit 302 receives the text data output in Step 18 as an input, and determines the key extraction position. The key extraction position is determined using the same method as in step 13. That is, each character is converted into a Unicode code as shown in FIG. 11 (FIG. 11 (b)), and the character A with “character A code <character code appearing right next to character A” is set as the key extraction position. (FIG. 11 (c)), the key extraction position is expressed in the coordinate format of (number of characters when counting from the upper left corner to the right, number of characters when counting from the upper left corner to the lower), and output as shown in FIG. To do.

    Step 20) The key extraction unit 303 receives the output of Step 19 as an input, and performs key extraction using the same method as in Step 14. That is, two characters obtained by connecting the character at the key extraction position and the character on the right side of the character are output as keys as shown in FIG.

  Step 6) Inquiry step during inquiry: An inquiry is made using the extracted key.

    Step 21) The search query unit 307 receives the key output in Step 20 as an input, and queries the search index DB 305 for the document name corresponding to each key and the position in the document.

    Step 22) As shown in FIG. 14, the search query results for each search key are tabulated to identify the original document name and the position in the original document with the largest number of cases. Since the key extraction position determination method at the time of search index creation (step 13) and search inquiry (step 19) is the same, all the search keys extracted from the two pages of “document 1” at the time of search inquiry are all “document 1”. Because it is included in the search index in association with these two pages, when the query results are aggregated to obtain the maximum number of results, this is always the correct search result (in this case, page 2 of “Document 1”). Become.

    Step 23) The search inquiry unit 307 makes an inquiry to the content DB 308, and acquires and outputs the content (here http: //content_1_2.html) corresponding to the original document name specified in Step 22 and the position in the original document. . Here, it is assumed that the data shown in FIG. 15 is stored in the content DB 308 in advance.

  Step 7) Inquiry result output step: The inquiry result is displayed.

    Step 24) The server-side data transmission / reception unit 306 receives the output of step 23 as an input and outputs it to the client unit 300 through the network.

    Step 25) The client side data transmitting / receiving unit 402 receives and outputs the output of Step 24 as an input.

    Step 26) The content display unit 403 receives the output of step 25 as an input and displays it as content. Here, it is assumed that the content display unit 403 is a mobile phone display, and the content of http: //content_1_2.html, which is the content, is displayed by a Web browser as shown in FIG.

[Second Embodiment]
This embodiment is different from the first embodiment in that only the key extraction position determination at the time of search index creation (step 13) and the key extraction position determination at the time of search inquiry (step 19) are changed. The method is the same as in the first embodiment.

  FIG. 17 is a flowchart of search index creation processing according to the second embodiment of this invention. Here, only step 1013 in which step 13 is changed and step 1019 in which step 19 is changed will be described.

  Step 1013) The key extraction position determination unit 302 receives the output of Step 12 as an input, and determines the key extraction position. The key extraction position is determined using a character code pattern. Here, each character is converted into a Unicode code as shown in FIG. 17 (FIG. 18 (b)), and the key A is extracted from the character A, which is “character A code × 2 <character code appearing right next to character A”. The position is specified (FIG. 18C). Thereafter, the same output as in step 13 is performed.

  FIG. 19 is a flowchart of search inquiry processing in the second embodiment of this invention.

  Step 1019) The key extraction position determination unit 302 receives the output of Step 18 as an input, and determines the key extraction position. The key extraction position is determined using the same method as in step 1013. That is, each character is converted into a Unicode code as shown in FIG. 18 (FIG. 20B), and a character A that is “character A code × 2 <character code appearing to the right of character A” is extracted as a key. The position is specified (FIG. 20B). Thereafter, the same output as in step 19 is performed.

[Third Embodiment]
This embodiment is different from the first embodiment in that only the key extraction position determination at the time of search index creation (step 13) and the key extraction position determination at the time of search inquiry (step 19) are changed. The method is the same as in the first embodiment.

  FIG. 21 is a flowchart of search index creation processing according to the third embodiment of the present invention.

  Here, only step 2013 in which step 13 is changed will be described.

  Step 2013) The key extraction position determination unit 302 receives the output of step 12 as an input, and determines the key extraction position. The key extraction position is determined using a character code pattern. Here, each character is converted into a Unicode code as shown in FIG. 22 (FIG. 22B), and the character A with “the code of character A is odd” is specified as the key extraction position (FIG. 22C). Thereafter, the same output as in step 13 is performed.

  FIG. 23 is a flowchart of search inquiry processing according to the third embodiment of the present invention. Here, only step 2019 in which step 19 is changed will be described.

  Step 2019) The key extraction position determination unit 302 receives the output of Step 18 as an input, and determines the key extraction position. The key extraction position is determined using the same method as in step 2013. That is, as shown in FIG. 24, each character is converted into a Unicode code (FIG. 24B), and the character A with “the code of character A is an odd number” is specified as the key extraction position (FIG. 24C). Thereafter, the same output as in step 19 is performed.

  In the present invention, the operation of the server unit 300 in the first to third embodiments described above is constructed as a program, installed in a computer used as the server unit and executed, or distributed via a network. It is possible to make it.

  In the present invention, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  In addition, this invention is not limited to said point, A various change and application are possible within a claim.

100 Document 200 Document Reading Device 201 Optical Character Recognition Device 300 Server Unit 301 Data Input Unit 302 Key Extraction Position Determination Unit 303 Key Extraction Unit 304 Search Index Output Unit 305 Search Index DB
306 Server-side data transmission / reception unit 307 Search inquiry unit 308 Content DB
400 Client unit 401 Document photographing unit 402 Client-side data transmission / reception unit 403 Content display unit

Claims (9)

Create a search index for responding to a search request to obtain a document in which the area appears and a position in the document by using a partial area in the document in which the page break or line break position is fixed as a search query. A search device to perform,
A document input means for accepting input of documents to be indexed;
Index key extraction position determining means for determining a position for extracting an index key from the whole or a partial area of the document based on characteristics of a character code representing a character;
Index key extracting means for extracting an index key composed of a combination of one or more characters at the position from the whole or a partial area of the document;
An index output means for associating the index key with an appearance position in a document in which the index key appears, and outputting to the index storage means;
A search device comprising: The index key extraction position determining means includes
Including means for determining a position for extracting an index key composed of a combination of one or more characters from the whole or a partial area of the document based on the relationship between the characters and the character codes of the characters existing in the vicinity thereof
The search device according to claim 1. Query input means for accepting a partial area in a document as a search query,
Query key extraction position determination means for determining a position from which the query key is extracted from the search query based on the characteristics of the character code representing the character;
Query key extraction means for extracting a query key consisting of a combination of one or more characters from the search query;
Search means for searching the index storage means based on the query key and outputting the search results;
The search device according to claim 1, further comprising: The query key extraction position determining means includes
Means for determining a position from which a query key consisting of a combination of one or more characters is extracted from the search query based on the relationship between the character and the character code of the character existing in the vicinity thereof,
The search device according to claim 3. Create a search index for responding to a search request to obtain a document in which the area appears and a position in the document by using a partial area in the document in which the page break or line break position is fixed as a search query. A search method to perform,
A document input step in which the input means receives input of a document to be indexed;
An index key extraction position determination means for determining an index key extraction position from the whole or a partial area of the document based on the characteristics of the character code representing the character;
An index key extracting unit that extracts an index key composed of a combination of one or more characters at the position from the whole or a partial area of the document; and
An index output means for associating the index key with an appearance position in the document in which the index key appears, and outputting to the index storage means;
The search method characterized by performing. In the index key extraction position determination step,
The search according to claim 5, wherein a position for extracting an index key composed of a combination of one or more characters from the whole or a partial area of the document is determined based on a relationship between a character and a character code of a character existing in the vicinity thereof. Method. A query input step in which the query input means accepts a partial area in a document as a search query;
A query key extraction position determining means for determining a position from which the query key is extracted from the search query based on a characteristic of a character code representing the character;
A query key extracting means for extracting a query key comprising a combination of one or more characters from the search query;
A search means for searching the index storage means based on the query key and outputting the search results;
The search method according to claim 5, further comprising: In the query key extraction position determination step,
The search method according to claim 7, wherein a position for extracting a query key composed of a combination of one or more characters from the search query is determined based on a relationship between the characters and character codes of characters existing in the vicinity thereof.   The program for functioning a computer as each means which comprises the search device of any one of Claims 1 thru | or 4.

Images