JP2023111074A - Search support apparatus, search support method, and program - Google Patents

Abstract

To provide a search support apparatus which reduces time and effort of a user for search.SOLUTION: A search support apparatus includes: a character extraction unit which extracts a character string based on input image data; a selection unit which selects, from among a group of character strings extracted by the character extraction unit, a part of a character string displayed with the image data, as a search character string; and a query output unit which outputs the search character string selected by the selection unit, in a form of search query. Preferably, the selection unit selects as many search character strings as possible up to an upper limit number of character strings of a search system. The query output unit outputs as many selected search character strings as possible in a form of search query.SELECTED DRAWING: Figure 6

Description

The present invention relates to a search support device, a search support method, and a program.

For example, Patent Document 1 discloses a search keyword input means for inputting a search keyword, an image information storage means for storing a plurality of image information, and a recognition target among the plurality of image information stored in the image information storage means. character recognition means for sequentially recognizing characters included in the image information read by said recognition target image reading means; and said search keyword input means for recognition results of said character recognition means. If the search keyword input by is included, the sequential recognition of the character recognition means is stopped, and the image information read by the recognition target image reading means or information for specifying the image information is output. A search system is disclosed comprising: means.

Further, Patent Document 2 discloses a device that scans text of image media, generates text data corresponding to it, generates a phrase list from the text data, and starts information retrieval corresponding to the phrase list. ing.

Patent No. 3486168 U.S. Pat. No. 7,151,864

An object of the present invention is to provide a search support device that reduces the user's trouble in searching.

A search support device according to the present invention includes a character extraction unit for extracting a character string based on input image data, and a character string extracted from a group of character strings extracted by the character extraction unit and displayed with the image data. It has a selection unit that selects part of a character string as a search character string, and a query output unit that outputs the search character string selected by the selection unit in the form of a search query.

Preferably, the selection unit selects as many search character strings as possible up to the upper limit of the number of characters of a search system, and the query output unit converts as many of the selected search character strings as possible into a search query. output in the format

Preferably, the selection unit preferentially selects proper nouns, numeric strings, or alphanumeric strings from among the character strings extracted by the character extraction unit.

Preferably, the character extraction section is an OCR processing section, and the selection section selects the search character string based on the OCR score output from the OCR processing section.

Preferably, the selection unit selects a search character string using a predetermined selection logic, and an operation detection unit that detects a user's editing operation on the search query output by the query output unit; A logic update unit that updates the selection logic used by the selection unit based on a detection result by the operation detection unit.

Preferably, the selection unit selects the item value as the search character string based on the combination of the items of the form and the item value.

Preferably, the system further includes an excluded character string definition unit that defines character strings with a high appearance frequency as excluded character strings, and the selection unit selects the excluded character strings defined by the excluded character string definition unit as search character strings. Select a string for search while excluding from .

Preferably, the selection unit determines the priority of selection based on the position of the character string in the image.

Further, a search support method according to the present invention includes a character extraction step of extracting a character string based on input image data, and displaying the image data from among the character strings extracted by the character extraction step. and a query output step of outputting the search character string selected by the selection step in the form of a search query.

Further, the program according to the present invention includes a character extraction step of extracting a character string based on input image data, and a character string extracted from the character string group extracted by the character extraction step, and displaying the image data. A computer is caused to execute a selection step of selecting a part of a character string as a search character string, and a query output step of outputting the search character string selected by the selection step in the form of a search query.

It is possible to reduce the user's trouble in searching.

1 is a diagram illustrating the overall configuration of a file management system 1; FIG. 4 is a diagram illustrating a search screen of the storage service 9; FIG. 2 is a diagram illustrating a hardware configuration of a search support device 2; FIG. 2 is a diagram illustrating a functional configuration of a search support device 2; FIG. 4 is a flowchart for explaining search processing (S10) in the file management system 1; FIG. 6 is a flowchart illustrating in more detail the search query generation process (S20) of FIG. 5; FIG. It is a figure which illustrates the search assistance program 32 in a modification.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating the overall configuration of a file management system 1. As shown in FIG.
As illustrated in FIG. 1, the file processing system 1 includes a storage service 9 that stores data files and a computer terminal 60 that accesses the storage service 9, which are connected to each other via a communication line such as the Internet.
The storage service 9 stores data files in association with search keywords, and searches for data files according to search queries entered in the search window illustrated in FIG.
The computer terminal 60 is a computer terminal operated by a user. The user may use the storage service 9 using the smartphone 62 instead of the computer terminal 60 .

In the above configuration, for example, when there is a printed form at hand and the data file of the original of this form is searched from the storage service 9, it is necessary to extract and input the wording described in the form so that it can be retrieved. There is However, words and phrases that are also used in other data files are meaningless because the search cannot be narrowed down, and it is necessary for a human to determine which words and phrases are effective for the search. In addition, it takes time to manually input multiple words in order to improve the search accuracy, and a string of alphanumeric characters such as a form number may be copied incorrectly.

Therefore, the file management system 1 of this embodiment further includes a search support device 2 and a scanner 4, and uses the scanner 4 as search input means. More specifically, a form to be searched is scanned by the scanner 4, and the search support device 2 performs OCR (Optical Character Recognition) processing on the description content, extracts effective words and phrases as a search query, and uses the words and phrases. to search.
As a result, the user can retrieve the original document with high accuracy by simply setting the form on the scanner 4 without the trouble of manual input.
The search support device 2 is a computer terminal connected to the scanner 4 and generates search queries from image data read by the scanner 4 . The search support device 2 of this example is connected to a computer terminal 60, a smartphone 62, and a storage service 9 via a communication line such as the Internet, and in response to a request from the computer terminal 60 or smartphone 62, the storage service 9 , and returns search results from the storage service 9 to the computer terminal 60 or the smart phone 62.
The scanner 4 is an image reading device that optically reads an image from a document.

FIG. 3 is a diagram illustrating the hardware configuration of the search support device 2. As illustrated in FIG.
As illustrated in FIG. 3, the search support device 2 has a CPU 200, a memory 202, an HDD 204, a network interface 206 (network IF 206), a display device 208, and an input device 210. These components are connected via a bus 212. connected to each other.
CPU 200 is, for example, a central processing unit.
The memory 202 is, for example, a volatile memory and functions as a main memory.
The HDD 204 is, for example, a hard disk drive device, and stores computer programs (eg, search support program 3 in FIG. 4) and other data files as a non-volatile recording device.
The network IF 206 is an interface for wired or wireless communication, and realizes connection to the scanner 4, the computer terminal 60, and the storage service 9, for example.
The display device 208 is, for example, a liquid crystal display.
Input device 210 is, for example, a keyboard and mouse.

FIG. 4 is a diagram illustrating the functional configuration of the search support device 2. As illustrated in FIG.
As illustrated in FIG. 4, a search support program 3 is installed and operates in the search support device 2 of this example. The search support program 3 is stored in a recording medium such as a CD-ROM, for example, and installed in the search support device 2 via this recording medium.
The search support program 3 has a scanner control section 300 , an OCR processing section 310 , an excluded character string definition section 320 , an item value extraction section 330 , a selection section 340 and a query output section 350 .
Part or all of the search support program 3 may be realized by hardware such as ASIC, or may be realized by partially borrowing functions of an OS (Operating System).

In the search support program 3 , the scanner control section 300 controls the scanner 4 to acquire image data read by the scanner 4 .
The OCR processing unit 310 performs OCR processing on the image data acquired by the scanner control unit 300, and extracts the character strings displayed on the document.

The exclusion character string definition unit 320 defines character strings with a high appearance frequency as exclusion character strings to be excluded from the search query.
Also, the excluded character string definition unit 320 defines a character string shorter than the predetermined number of characters (N characters) as an excluded character string.

The item value extraction unit 330 extracts a combination of items and item values of the form from the OCR result of the OCR processing unit 310 . The item value extraction unit 330 of this example extracts a key-value combination from the OCR result of the OCR processing unit 310 .

The selection unit 340 selects part of the character strings displayed in the image data from among the character strings extracted by the OCR processing unit 310 as search character strings. More specifically, the selection unit 340 excludes the exclusion character strings defined by the exclusion character string definition unit 320 from among the character strings extracted by the OCR processing unit 310, while the search system of the storage service 9 Select as many strings as possible for the search string, up to the upper limit of the number of characters in .
In addition, the selection unit 340 preferentially selects character strings corresponding to nouns such as proper nouns, numeric strings, and alphanumeric character strings from the character string group extracted by the OCR processing unit 310 as search character strings. to select. Furthermore, the selection unit 340 selects a search character string based on the OCR score output from the OCR processing unit 310 . Note that the OCR score is an index indicating the certainty of character recognition in OCR processing.

The query output unit 350 outputs the search character string selected by the selection unit 340 in the form of a search query. The query output unit 350 of this example outputs the search character string selected by the selection unit 340 in the form of a search query for the storage service 9 .

FIG. 5 is a flow chart for explaining the search processing (S10) in the file management system 1. As shown in FIG.
As exemplified in FIG. 5, in step 100 (S100), when the user sets the form to be searched on the scanner 4 and instructs the search by the scanner from the computer terminal 60, the scanner control section 300 of the search support device 2 (see FIG. 4) sends a scan instruction to the scanner 4 according to the instruction from the computer terminal 60 .
At step 105 ( S<b>105 ), the scanner control section 300 acquires image data scanned by the scanner 4 .
At step 20 ( S<b>20 ), the search support program 3 executes search query generation processing based on the image data acquired from the scanner 4 .
At step 115 (S115), the search support program 3 inputs the generated search query into the search window (FIG. 2) of the storage service 9 to execute the search.

FIG. 6 is a flowchart explaining in more detail the search query generation process (S20) of FIG.
As shown in FIG. 6, in step 200 (S200), the OCR processing unit 310 performs OCR processing on the image data acquired by the scanner control unit 300, and outputs the result of the OCR processing and the OCR score. .

At step 205 ( S<b>205 ), the selection unit 340 performs morphological analysis on the OCR processing result output from the OCR processing unit 310 .
At step 210 (S210), the selection unit 340 extracts only nouns as candidate character strings from the OCR processing results output from the OCR processing unit 310 based on the morphological analysis results.

At step 215 (S215), the item value extraction section 330 performs key-value extraction from the OCR processing result, and the selection section 340 selects an alphanumeric string corresponding to the form number from the key-value extraction result. Note that the alphanumeric string corresponding to the form number may be extracted by named entity extraction.
At step 220 (S220), the query output unit 350 sets the alphanumeric string selected by the selection unit 340 as part of the search query.

At step 225 (S225), the selection unit 340 determines whether the number of characters selected as the search query exceeds the character limit in the search system of the storage service 9. If the character limit is not exceeded, The process proceeds to S230, and when the character count limit is exceeded, the search query generation process ends.
At step 230 (S230), the selection unit 340 determines whether or not there is a character string that has not been selected as part of the search query in the candidate character string group. If so, the process proceeds to S235, and if there is no character string that has not been selected, the search query generation process ends.

At step 235 (S235), the selection unit 340 selects a character string corresponding to a proper noun such as a company name, name, address, etc. from the result of key-value extraction by the item value extraction unit 330. FIG. Character strings corresponding to proper nouns may be extracted by named entity extraction or word dictionary extraction.
At step 240 (S240), the query output unit 350 sets the character string selected by the selection unit 340 as part of the search query.

At step 245 (S245), the selection unit 340 determines whether the number of characters selected as the search query exceeds the character limit in the search system of the storage service 9. If the character limit is not exceeded, The process proceeds to S250, and when the character count limit is exceeded, the search query generation process ends.
At step 250 (S250), the selection unit 340 determines whether or not there is a character string that has not been selected as part of the search query in the candidate character string group. If so, the process proceeds to S255, and if there is no character string that has not been selected, the search query generation process ends.

At step 255 (S255), the exclusion character string definition unit 320 defines a character string having less than N characters (for example, N=4) as an exclusion character string, and the selection unit 340 selects , to exclude strings with less than N characters.
At step 260 (S260), the selection unit 340 sorts the candidate character string group in descending order of the OCR score based on the OCR score output from the OCR processing unit 310. FIG.

At step 265 (S265), the selection unit 340 selects a character string with the highest OCR score from the candidate character string group based on the sort result.
At step 270 (S270), the query output unit 350 sets the character string selected by the selection unit 340 as part of the search query.

At step 275 (S275), the selection unit 340 determines whether the number of characters selected as the search query exceeds the character limit in the search system of the storage service 9. If the character limit is not exceeded, The process proceeds to S280, and when the number of characters exceeds the character limit, the search query generation process ends.
At step 280 (S280), the selection unit 340 determines whether or not there is a character string that has not been selected as part of the search query in the candidate character string group. If so, the process returns to S265, and if there is no character string that has not been selected, the search query generation process ends.

The search query generation process (S20) roughly includes three extraction processes: a form number extraction process, a proper noun extraction process, and a recognition score order extraction process.
Since the extraction accuracy of these processes may not be 100% and there may be cases where there is no description in the first place, the processes are performed sequentially regardless of the success or failure of the extraction. The extracted character strings are set as a search query by enumerating them separated by spaces (eg, "ABC bill ABC12345678").
The extraction process ends as soon as the character limit for the search query is exceeded in each extraction process, or as soon as the candidate character string group is exhausted.

The first form number extraction process (S215 and S220) is a process for extracting a form number. Since the form number is a unique character string that is given when the form is issued, if it can be extracted, the search accuracy will be greatly improved. However, since the uniqueness is only within the management range of the issuing company, the possibility of duplication with that of other companies is not 0, so other information is also required in the subsequent processing.
Extraction methods include key-value extraction and named entity extraction.

The second proper noun extraction processing (S235 and S240) is processing for extracting proper nouns written in the form. Search accuracy can be improved by extracting multiple pieces of information such as a company name, name, address, etc., for which a form cannot be uniquely determined.
Extraction methods include key-value extraction, named entity extraction, and extraction using a word dictionary, as in the form number extraction process.

The third recognition score order extraction process (S255 to S270) is a process of extracting a highly accurate character string based on the recognition score in the OCR process. It can be expected that there will be omissions in the above extraction process and extraction of other character strings written in the form.
As an extraction method, first, the group of candidate character strings is narrowed down to words of N characters or more (for example, N=4). As a result, words that are too short to make sense and words that are noise due to misrecognition are excluded from the search query.
After that, sorting is performed according to the OCR recognition score, and character strings with high recognition scores are sequentially extracted.

As described above, according to the file management system 1 of the present embodiment, it is possible to automatically generate a search query from image data. can be searched. In addition, since the number of character strings included in the search query is automatically entered as much as possible as long as it is allowed by the search system, the user can search by simply deleting unnecessary keywords (character strings) as appropriate. Queries can be tuned.
Further, according to the file management system 1, it becomes easy to search for presentation materials, sales promotion materials (flyers), papers, patent documents, contracts, and the like.

Next, a modification of the above embodiment will be described.
FIG. 7 is a diagram illustrating the search support program 32 in the modified example. In addition, the same code|symbol is attached|subjected to the substantially same thing as the structure shown by FIG. 4 among each structure illustrated by this figure.
As illustrated in FIG. 7, the search support program 32 of the modified example adopts a configuration in which an operation detection unit 360 and a logic update unit 370 are added to the search support program 3 of FIG.
The operation detection unit 360 detects a user's editing operation on the search query output by the query output unit 350 . For example, the operation detection unit 360 detects when the user performs a deletion operation on the search query input to the search window (FIG. 2) by the query output unit 350 .
The logic update unit 370 updates the selection logic used by the selection unit 340 based on the detection result of the operation detection unit. For example, the logic update unit 370 may change the adoption or rejection of the form number extraction process, the proper noun extraction process, and the recognition score order extraction process, or their order, or may change the value of the number of characters N.
In other words, the search support device 2 learns the success or failure of the extraction content by providing an interaction that allows the user to confirm the result of the search query extraction process. This makes it possible to prioritize the words to be extracted and improve the extraction accuracy.

Also, the storage service 9 may be of any form, such as an in-house server, as long as it can store content and has a search function.
When extracting proper nouns using a word dictionary, not complete matching but partial matching may be accepted. This makes it possible to correct OCR recognition errors when setting search queries.
Further, when extracting a query, the selection unit 340 may use position information of words in the form, font (size, bold), and the like. This makes it possible to preferentially extract words of higher importance as search keywords.
In the above embodiment, the purpose is to identify the original document, but by appropriately changing the priority order and contents of query extraction, it is possible to search for similar content (delete the last N digits of the form number) instead of the original document itself, Category search (invoice and date) will also be possible.
The search method may be a real-time search targeting only one form, or a batch search targeting a plurality of sheets. Search results can also be output as text such as a csv file.
Further, in the above-described embodiment, the scanner 4 is used as means for reading a document, but the scanner 4 may be replaced by a photographing device such as a smart phone. For example, the search support device 2 creates a search query based on image data captured by a camera built into a smartphone.

1 File Management System 2 Search Support Device 3 Search Support Program 4 Scanner 9 Storage Service

Claims (10)

a character extraction unit for extracting a character string based on input image data;
a selection unit that selects, as a search character string, a part of the character string displayed in the image data from the character string group extracted by the character extraction unit;
and a query output unit that outputs the search character string selected by the selection unit in the form of a search query. The selection unit selects as many search character strings as possible up to the upper limit of the number of characters of the search system,
The search support device according to claim 1, wherein the query output unit outputs as many of the selected search character strings as possible in the form of a search query. 3. The search support device according to claim 2, wherein the selection unit preferentially selects proper nouns, numeric strings, or alphanumeric strings from among the character strings extracted by the character extraction unit. The character extraction unit is an OCR processing unit,
The search support device according to claim 3, wherein the selection unit selects a search character string based on the OCR score output from the OCR processing unit. The selection unit selects a search string using a predetermined selection logic,
An operation detection unit that detects a user's editing operation on the search query output by the query output unit;
5. The search support device according to claim 4, further comprising: a logic update unit that updates the selection logic used by the selection unit based on the detection result of the operation detection unit. 4. The search support device according to claim 3, wherein the selection unit selects an item value as a search character string based on a combination of items and item values of a form. further comprising an excluded character string definition section that defines frequently occurring character strings as excluded character strings;
7. The search support device according to claim 6, wherein the selection unit selects the search character string while excluding the exclusion character string defined by the exclusion character string definition unit from the search character string. 8. The search support device according to claim 7, wherein the selection unit determines the priority of selection based on the position of the character string in the image. a character extraction step of extracting a character string based on the input image data;
a selection step of selecting, as a search character string, part of the character strings displayed in the image data from the character string group extracted by the character extraction step;
and a query output step of outputting the search character string selected in the selection step in the form of a search query. a character extraction step of extracting a character string based on the input image data;
a selection step of selecting, as a search character string, part of the character strings displayed in the image data from the character string group extracted by the character extraction step;
A program for causing a computer to execute a query output step of outputting the search character string selected in the selection step in the form of a search query.

Images