JP2015072533A - Information processing apparatus and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus that performs, when a display character string included in an image is recognized, processing corresponding to the display character string.SOLUTION: An information processing apparatus includes reception means for receiving a result of character recognition, storage means for storing the display character string and processing information in association with each other, search means for searching for the display character string in the result of the character recognition, extraction means for extracting, when the search means has successfully searched the display character string in the result of the character recognition, processing information corresponding to the display character string from the storage means, and processing means for performing processing on the basis of the processing information extracted by the extraction means.

Description

  The present invention relates to an information processing apparatus and an information processing program.

  Patent Document 1 has an object of providing a portable information terminal device that reads a URI described in a printed matter and acquires and displays the corresponding content from an Internet WWW server, and is of a size that can be grasped with one hand. A display device, an operation switch, an execution button, a speaker, an antenna, and a reading start button are provided on the top surface of the information terminal device, and a reading window for a scanner is provided on the bottom surface, and an image of an area including a URI on the printed material. Is read by the internal scanner through the reading window, the URI is extracted from the image, the built-in browser is activated, and the browser transmits the content corresponding to the URI via the provider connected by the wireless communication line on the Internet. Obtaining from a server and displaying on a display device is disclosed.

  Japanese Patent Laid-Open No. 2004-26883 has a problem that even a person unfamiliar with the operation can quickly and surely start a predetermined application program. The name of a given application program and its arguments are printed, such as a card-like object that is printed as “patent specification” as the file name to be opened by Santaro (capture data to start processing). After the card-like object is imaged by the CCD video camera, the personal computer performs character recognition processing on the image data imaged by the CCD video camera, and after starting the application program of the character name obtained as a result of recognition, Import data of recognized file names as arguments Causing operations such descriptions set automatically executed is disclosed.

  In Patent Document 3, it is an object to make it easy to use the Internet, and a card-like object on which a URL for uniquely specifying an information source on the Internet spreading worldwide is printed as a CCD video. When the personal computer recognizes the URL by image recognition processing from the image data captured by the CCD video camera, the personal computer automatically accesses the URL obtained by character recognition after starting the Internet browser. Displaying information is disclosed.

JP 2001-184141 A JP 2000-207485 A JP 2000-207338 A

  An object of the present invention is to provide an information processing apparatus and an information processing program that perform processing corresponding to a display character string when a display character string included in an image is recognized.

The gist of the present invention for achieving the object lies in the inventions of the following items.
The invention of claim 1 is a receiving means for receiving a character recognition result, a storage means for storing a display character string and processing information in association with each other, a search means for searching for the display character string from the character recognition result, When the display character string can be searched from the character recognition result by the search means, the extraction means for extracting the processing information corresponding to the display character string from the storage means, and the processing information extracted by the extraction means An information processing apparatus comprising processing means for performing processing based on the information processing apparatus.

  The invention of claim 2 further includes candidate search means for searching for candidates as the display character string from the character recognition result, wherein the search means displays the display character from the candidates searched by the candidate search means. The information processing apparatus according to claim 1, wherein a column is searched.

  According to a third aspect of the present invention, the candidate in the candidate search means is information described in a regular expression indicating a part or all of the display character string, and the candidate search means The information processing apparatus according to claim 2, wherein a search is performed by performing a regular expression search for the candidate.

  According to a fourth aspect of the present invention, the candidate in the candidate search means is information indicating a range of positions where display characters exist, and the character recognition result includes a position of each character, and the candidate search The information processing apparatus according to claim 2 or 3, wherein the means searches the character recognition result by extracting a character whose recognized character position falls within the candidate range.

  According to a fifth aspect of the present invention, the candidate in the candidate search means is information indicating a range of display character size, and the character recognition result includes the size of each character, and the candidate search 5. The method according to claim 2, wherein the means searches the character recognition result by extracting a character whose recognized character size corresponds to the candidate range. 6. Information processing apparatus.

  According to a sixth aspect of the present invention, the candidate in the candidate search means is information indicating a color range of display characters, and the character recognition result includes a color of each character. The information processing apparatus according to claim 2, wherein the character recognition result is searched by extracting a character whose color corresponds to the candidate range. It is.

  The invention of claim 7 is a computer for receiving a character recognition result, a storage unit for storing a display character string and processing information in association with each other, and a search unit for searching for the display character string from the character recognition result. And when the display character string can be searched from within the character recognition result by the search means, an extraction means for extracting processing information corresponding to the display character string from the storage means, and an extraction means extracted by the extraction means It is an information processing program for functioning as a processing means for performing processing based on processing information.

  According to the information processing apparatus of the first aspect, when a display character string included in the image is recognized, a process corresponding to the display character string can be performed.

  According to the information processing apparatus of the second aspect, the processing information corresponding to the display character string can be searched at a higher speed than in the case where the present configuration is not provided.

  According to the information processing apparatus of the third aspect, it is possible to reduce the number of search objects by performing a regular expression search on the character recognition result.

  According to the information processing apparatus of the fourth aspect, it is possible to reduce the number of search objects by extracting the character recognition result whose position of the recognized character falls within the candidate range.

  According to the information processing apparatus of the fifth aspect, it is possible to reduce the number of search objects by extracting the character recognition result whose character size corresponds to the candidate range.

  According to the information processing apparatus of the sixth aspect, it is possible to reduce the number of search objects by extracting the character recognition result whose color of the recognized character falls within the candidate range.

  According to the information processing program of the seventh aspect, when a display character string included in the image is recognized, a process corresponding to the display character string can be performed.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is a flowchart which shows the process example by 1st Embodiment. It is explanatory drawing which shows the example of the text string which 1st Embodiment makes object. It is explanatory drawing which shows the example of a data structure of a display character string and command table. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is a flowchart which shows the process example by 2nd Embodiment. It is explanatory drawing which shows the example of the text string which 2nd Embodiment makes object. It is explanatory drawing which shows the example of a data structure of a display character string and command table. It is a notional module block diagram about the structural example of 3rd Embodiment. It is a flowchart which shows the process example by 3rd Embodiment. It is explanatory drawing which shows the process example by 3rd Embodiment. It is explanatory drawing which shows the example which a portable information terminal with a camera image | photographs a document. It is explanatory drawing which shows the example of the coordinate system in an image. It is explanatory drawing which shows the example of a character position coordinate. It is explanatory drawing which shows the example of character candidate position information. It is a notional module block diagram about the structural example of 4th Embodiment. It is a flowchart which shows the process example by 4th Embodiment. It is a notional module block diagram about the structural example of 5th Embodiment. It is a flowchart which shows the process example by 5th Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

First, before explaining the present embodiment, a technique that is a premise thereof will be described. This description is intended to facilitate understanding of the present embodiment.
For example, in the technique described in Patent Document 1, an image is captured with a PDA (Personal Digital Assistants) or PC (Personal Computer) attached camera, characters in the image are recognized, and a URL ( Uniform Resource Locator (including URI: Uniform Resource Identifier)). A keyword such as “http: //” is registered in advance, and a character string starting with the keyword is extracted. The recognized URL (URI) is automatically displayed by a web browser.
Further, for example, in the technique described in Patent Document 2, an image is taken with a PDA or PC-attached camera, characters in the image are recognized, and a command described in the image is extracted. For example, the word “word processor” is registered in advance as a command name, and if the character recognition result matches the registered command name, it is extracted as a command. Then, the character string next to the command is extracted as an argument, and an application (computer program) is started using the command and the argument. For example, when the command is “word processor” and the argument is “document”, an application “word processor” is started to open “document”.
In these techniques, a command such as a web browser can be automatically activated only when a URL or a command name is described as it is in an image.
If the URL is directly described in the document, the image becomes complicated and the design is often lost. For example, a long string of meaningless alphanumeric characters is often described in the URL, and the appearance is not beautiful.
In some cases, it is desired to change only the URL and the command without changing the image itself. Therefore, if the URL or command is to be changed, the design of a document such as a flyer or a catalog must be changed. Designs are time-consuming and costly and cannot be changed easily. Therefore, there is a problem that the URL and the command cannot be easily changed.
The prior art is based on the premise that URLs and commands can be easily retrieved from a text string that has been subjected to character recognition. For example, when the number of registered URLs and commands is enormous, the search takes time. If the number of registered URLs and commands is N, N searches must be performed on the text string. When N is a large number, it does not end in a realistic time.

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

The information processing apparatus according to the first embodiment performs character recognition on an image obtained by photographing a document in which a display character string is described, and performs processing corresponding to the display character string. As shown, it has a character recognition module 110, a character string search module 120, a display character string / command table 140, and a command processing module 150. Examples of the processing corresponding to the display character string include automatically displaying a web page and automatically starting an application. Specifically, a document including a display character string is photographed by a portable information terminal with a camera, the image is recognized, and processing corresponding to the display character string is performed.
In addition, the display character string does not include URLs and commands that are uncomfortable in design in the document. That is, a command such as a web browser is operated without describing a URL or a command.

The character recognition module 110 is connected to the character string search module 120. The character recognition module 110 accepts an image 105 on which characters are described, and recognizes the image 105 as characters. Accepting the image 105 means, for example, reading the image 105 with a camera, a scanner, etc., receiving the image 105 from an external device via a communication line with a fax, etc., a hard disk (in addition to what is built in the computer, Etc.) including reading out the image 105 stored in the network (including those connected via a network). The image 105 may be a binary image or a multi-value image (including a color image). One image may be received or a plurality of images may be received. Further, the content of the image 105 may be an advertisement pamphlet, a document used in business, or the like. The existing technology may be used for character recognition.
Then, the text string 115 is passed to the character string search module 120 as a character recognition result of the character recognition module 110.

  The display character string / command table 140 is connected to the character string search module 120. The display character string / command table 140 stores display character strings and processing information (hereinafter also referred to as commands) in association with each other. For example, a display character string / command table 400 is stored. FIG. 4 is an explanatory diagram showing an example of the data structure of the display character string / command table 400. The display character string / command table 400 includes a number column 410, a display character string column 420, and a command column 430. The number column 410 stores a number for uniquely identifying the correspondence between the display character string and the command in the present embodiment. The display character string column 420 stores a display character string. That is, the display character string which may be described in the image 105 used as the object of character recognition is memorize | stored. The command column 430 stores a command corresponding to the display character string. The command stored in the command column 430 may be a URL, and the command processing in that case is to start a web browser for accessing the URL. The command stored in the command column 430 may be an application name and its argument (there may be no argument), and the command processing in that case is to start the application.

The character string search module 120 is connected to the character recognition module 110, the display character string / command table 140, and the command processing module 150. The character string search module 120 searches for a display character string stored in the display character string / command table 140 from the text string 115 as a character recognition result. Specifically, matching processing between the display character string stored in the display character string column 420 of the display character string / command table 400 and the text string 115 is performed.
If the display character string can be searched from the text string 115 by the search, the character string search module 120 extracts processing information corresponding to the display character string from the display character string / command table 140. Specifically, the command 125 stored in the command column 430 of the display character string / command table 400 corresponding to the retrieved display character string is extracted.

  The command processing module 150 is connected to the character string search module 120. The command processing module 150 performs processing based on the command 125 that is processing information extracted by the character string search module 120. As described above, when the command 125 is a URL, a web browser for accessing the URL is activated. If the command 125 is an application name and its argument, the application is activated.

FIG. 2 is a flowchart illustrating a processing example according to the first exemplary embodiment.
In step S202, the character recognition module 110 receives the image 105. For example, a photograph is acquired by a portable information terminal such as a digital camera or a smartphone with a camera. Take pictures of products in catalogs and other documents. The photograph is captured as a digital image (image 105).
In step S204, the character recognition module 110 recognizes the image 105 as characters. That is, the image 105 is recognized, and the characters in the image 105 are converted into a text string 115 and output. For example, as the text string 115 as a character recognition result, the text string 315 illustrated in FIG. 3 is “Product description: This product (product number 123-456) is very bargain, has a good sense, and is reasonably priced. Besides, the color and shape are outstanding. There is no hand to buy. ”

In step S <b> 206, the character string search module 120 extracts a display character string from the display character string / command table 140. For example, the display character string / command table 400 in the display character string / command table 140 stores a display character string (display character string column 420) and a command (command column 430) as a pair. The display character string is a character string that may be entered in the image 105. In the example of FIG. 4, the product number is stored as the display character string. A URL is stored as a command. The display character strings are extracted from the display character string / command table 400 in order from the first number (number field 410).
In step S208, the character string search module 120 searches the display character string from the character recognition result. That is, the text string 115 is searched using the display character string extracted in step S206 as a query. The search may use a general full-text search algorithm or the like. In the example of FIGS. 3 and 4, the first “123-456” in the display character string / command table 400 is searched from the text string 315.

In step S210, the character string search module 120 determines whether or not there is a display character string in the character recognition result. If there is, the process proceeds to step S212. Otherwise, the process proceeds to step S216. If the search is hit (Y in step S210), the extracted display character string exists in the text string 115.
In step S212, the character string search module 120 extracts a command 125 corresponding to the display character string searched from the display character string / command table 140. For example, the command 125 paired with the display character string is extracted from the display character string / command table 400.
In step S214, the command processing module 150 interprets the command 125 and processes the command 125. In the example of FIGS. 3 and 4, the URL (http: //www.abc.def/xxx of the display character string / command table 400: 1) corresponding to the retrieved display character string is displayed on the web browser. To do.
In step S216, error processing is performed. For example, it may be displayed on the display device that “there is no display character string corresponding to the command in the photographed image”. Further, it is not necessary to perform processing such as error display only by photographing a document.

<Second Embodiment>
FIG. 5 is a conceptual module configuration diagram of a configuration example according to the second embodiment.
In addition, the same code | symbol is attached | subjected to the site | part of the same kind as the above-mentioned embodiment, and the overlapping description is abbreviate | omitted (hereinafter the same).
The information processing apparatus according to the second embodiment performs character recognition on an image obtained by photographing a document in which a display character string is described, and performs processing corresponding to the display character string. As shown, a character recognition module 110, a regular expression search module 520, a character string DB search module 530, a display character string / command table 140, and a command processing module 150 are included. In particular, even if the number of registered display character strings is enormous, it operates in a realistic time.

The correspondence between the display character string stored in the display character string / command table 140 and the processing information may be many. In the first embodiment, an example of searching for a character string in a text string using a display character string as a query for each row has been shown. However, in this method, a time proportional to the number of display character strings is required for the search. . If 1 ms is required for one search and there are 1 million display character strings, 1000 seconds are required.
Therefore, the following operation is performed in the second embodiment.
First, a regular expression 517 that matches all display character strings (or most display character strings) is generated. For example, the display character string in the display character string / command table 140 is analyzed, and the regular expression 517 “three-digit number—three-digit number” is generated. The regular expression 517 may be generated using an existing method. Using the regular expression 517 as a query, the regular expression search module 520 performs a regular expression search on the text string 115. The regular expression search can be realized by using a regular expression library or a grep command.
The regular expression search module 520 passes the character string (extracted character string 525) when the regular expression search is hit to the character string DB search module 530. In the example of FIG. 7, in the case of the regular expression “three-digit number—three-digit number”, two character strings “123-456” and “333-333” are extracted character strings 525.
Using these extracted character strings 525 as queries, the character string DB search module 530 searches the display character string / command table 140 for a display character string that matches the extracted character string 525. As this search method, a general character string search can be used.
As a result of the search in the character string DB search module 530, a command 125 corresponding to the display character string hit is output.
By doing so, it is not necessary to use all the display character strings in the display character string / command table 140 as queries, and the search speeds up.

The relationship and processing between modules shown in the example of FIG. 5 are as follows.
The character recognition module 110 is connected to the regular expression search module 520.
The regular expression search module 520 is connected to the character recognition module 110 and the character string DB search module 530. The regular expression search module 520 searches for a candidate for a display character string from the text string 115 that is a character recognition result. The candidate here is information described in a regular expression indicating a part or all of the display character string, and the regular expression search module 520 searches the text string 115 by performing a candidate regular expression search. To do.
The character string DB search module 530 is connected to the regular expression search module 520, the display character string / command table 140, and the command processing module 150. The character string DB search module 530 searches the display character string from the candidates searched by the regular expression search module 520.
The display character string / command table 140 is connected to the character string DB search module 530. The command processing module 150 is connected to the character string DB search module 530.

FIG. 6 is a flowchart illustrating a processing example according to the second exemplary embodiment.
In step S602, the character recognition module 110 receives the image 105. Processing equivalent to that in step S202 is performed.
In step S604, the character recognition module 110 recognizes characters in the image 105. Processing equivalent to that in step S204 is performed. The text column 715 illustrated in FIG. 7 is “Product description: This product (product number 123-456) is very bargain, has a good sense, is reasonably priced, and is excellent in color and shape. There is no phone number 333-3333 ".

In step S606, the regular expression search module 520 receives the regular expression 517 generated from the display character string in the display character string / command table 140. For example, the display character string / command table 800 illustrated in FIG. 8 is stored in the display character string / command table 140. The display character string / command table 800 is equivalent to the display character string / command table 400 illustrated in FIG. However, the number of lines in the display character string / command table 800 (the number of correspondence between the display character string and the command) is large. By analyzing the display character string in the display character string field 820 of the display character string / command table 800, a regular expression 517 of “three digits—three digits” is generated.
In step S608, the regular expression search module 520 searches for a character string that matches the regular expression 517 from the character recognition result. The search here is a regular expression search. In the example of FIG. 7, “123-456” and “333-333” are searched, and these character strings become the extracted character string 525.
In step S610, the character string DB search module 530 searches the display character string / command table 140 for the extracted character string 525. In the above example, only “123-456” and “333-333” are targeted, and “123-456” is searched.

In step S612, the character string search module 120 determines whether or not there is a display character string in the character recognition result (extracted character string 525). If there is, the process proceeds to step S614. Otherwise, the process proceeds to step S618. move on. Processing equivalent to that in step S210 is performed.
In step S614, the character string search module 120 extracts a command 125 corresponding to the display character string searched from the display character string / command table 140. Processing equivalent to that in step S212 is performed.
In step S616, the command processing module 150 interprets the command 125 and processes the command 125. Processing equivalent to that in step S214 is performed.
In step S618, error processing is performed. Processing equivalent to that in step S216 is performed.

  In the second embodiment, the image 105 may be either a still image or a moving image. In that case, the character recognition module 110 determines whether the image 105 is a moving image. When the image 105 is a moving image, the character recognition module 110 recognizes a plurality of frame images constituting the moving image. As a result, a text string 115 is generated for each frame image. Then, the regular expression search module 520 performs processing on the text string 115 in the first frame image having a character string that matches the regular expression 517. For example, even if an operator mistakenly operates a smartphone with a camera and shoots a moving image instead of a still image, high-speed processing can be performed.

<Third Embodiment>
FIG. 9 is a conceptual module configuration diagram of a configuration example according to the third embodiment.
The information processing apparatus according to the third embodiment performs character recognition on an image obtained by photographing a document in which a display character string is described, and performs processing corresponding to the display character string. As shown, a character recognition module 110, a candidate position character extraction module 920, a character string DB search module 530, a display character string / command table 140, and a command processing module 150 are included. In particular, even if the number of registered display character strings is enormous, it operates in a realistic time.

This will be described with reference to the examples of FIGS. As shown in the example of FIG. 11A, it is assumed that the document 1100 describes a plurality of products (including product names, product descriptions, product numbers, and the like). Here, the processing performed by the command processing module 150 displays a URL in which detailed product information is described.
As shown in the example of FIG. 12, the operator uses the camera-equipped portable information terminal 1210 to photograph the document 1100. At that time, the operator often shoots such that a product for which detailed information is to be acquired is the center, such as a photographed image 1110 of the user shown in the example of FIG. This is because the operator expects that a web page related to the product photographed by the operator is automatically displayed. In addition, it may be presented that photographing is performed so that a product (an image to be processed in the command processing module 150) for which detailed information is to be acquired is centered.
In the third embodiment, first, the character recognition module 110 performs character recognition on the user's photographed image 1110 (image 105) to obtain a character recognition result text string 1120. However, since the operator simply “photographs so as to be centered”, it is not always possible to extract only the display character string to be targeted. There is a high possibility that an image or description of another adjacent product in the document will be reflected. In the example of FIG. 11B, the image of the right shirt, the description B thereof, and the like are applicable.
Therefore, the candidate position character extraction module 920 is a candidate that can be used in the subsequent character string DB search module 530 from the character recognition result text string 1120 (text string / each character position 915) shown in the example of FIG. An extraction result 1125 (extracted character string 525) that is a character string is extracted.
In the third embodiment, position information in the user's captured image 1110 is used. The candidate position character extraction module 920 receives character candidate position information 917. In addition to the text string, each character position information (text string / character position 915) is obtained as a character recognition result of the character recognition module 110. Character candidate position information 917 and detailed examples of character positions will be described later with reference to FIGS.
The candidate position character extraction module 920 extracts a character string that matches the character candidate position information 917 from the character recognition result text string 1120. The extraction result is like an extraction result 1125 shown in the example of FIG. In this way, the contents of the text string of the character recognition result can be limited, and the influence of adjacent images (products etc.) can be avoided.
Further, the extracted character string 525 may be re-extracted with a regular expression by the regular expression search module 520 of the second embodiment.

The relationship and processing between modules shown in the example of FIG. 9 are as follows.
The character recognition module 110 is connected to the candidate position character extraction module 920.
The candidate position character extraction module 920 is connected to the character recognition module 110 and the character string DB search module 530. The candidate position character extraction module 920 searches for a candidate as a display character string from the text string / character position 915 that is a character recognition result. The candidate here is information (character candidate position information 917) indicating a range of positions where the display character exists, and the text string / character position 915 includes the position of each character. The extraction module 920 searches the text string / each character position 915 by extracting a character string whose recognized character position falls within the candidate range. Note that the character candidate position information 917 may be a value indicating a predetermined range, or information on a focus position included as attribute information in the image 105 may be used. For example, it may be character candidate position information 917 defined by a rectangle with a predetermined length around the focus position.
The character string DB search module 530 is connected to the candidate position character extraction module 920, the display character string / command table 140, and the command processing module 150. The character string DB search module 530 searches the display character string from the candidates (extracted character string 525) searched by the candidate position character extraction module 920.
The display character string / command table 140 is connected to the character string DB search module 530. The command processing module 150 is connected to the character string DB search module 530.

FIG. 10 is a flowchart illustrating a processing example according to the third exemplary embodiment.
In step S1002, the character recognition module 110 receives the image 105. Processing equivalent to that in step S202 is performed.
In step S1004, the character recognition module 110 recognizes the image 105 and extracts the position of each character. Processing equivalent to that in step S204 is performed, but the character position is extracted together with the text information of each character.
In step S1006, the candidate position character extraction module 920 receives the character candidate position information 917.
In step S1008, the candidate position character extraction module 920 extracts a character string (extracted character string 525) corresponding to the character candidate position information 917 in the text string / character position 915 as the character recognition result.
In step S1010, the character string DB search module 530 extracts a display character string from the display character string / command table 140. Processing equivalent to that in step S206 is performed.

In step S1012, the display character string is searched from the extracted character string 525. That is, the extracted character string 525 is searched using the display character string extracted in step S1010 as a query. The search may use a general search algorithm. In step S1014, the character string search module 120 determines whether there is a display character string in the character recognition result (extracted character string 525). Otherwise, go to step S1020.
In step S1016, the character string search module 120 extracts a command 125 corresponding to the display character string searched from the display character string / command table 140. Processing equivalent to that in step S212 is performed.
In step S1018, the command processing module 150 interprets the command 125 and processes the command 125. Processing equivalent to that in step S214 is performed.
In step S1020, error processing is performed. Processing equivalent to that in step S216 is performed.

FIG. 13 is an explanatory diagram illustrating an example of a coordinate system in an image. FIG. 14 is an explanatory diagram showing an example of character position coordinates 1420. FIG. 15 is an explanatory diagram illustrating an example of the character candidate position information 1530.
A detailed example of character candidate position information 917 and a text string / character position in each character position 915 will be described. The image 105 can be represented by a shape in which pixels are two-dimensionally arranged. Each pixel position can be represented by X and Y coordinates. Here, with the upper left corner of the image 1300 as the origin 1310, the X axis and the Y axis are defined as in the example shown in FIG. This is an example, and the origin 1310 and the axis defining method may be performed in any way as long as the pixel position can be specified.
As a result of character recognition, the character position in the XY coordinate system can be obtained. As in the example shown in FIG. 14, the upper left coordinate (character position coordinate 1420) of the circumscribed rectangle of the character image targeted for character recognition can be set as the character position. Of course, not the upper left coordinates but the lower right coordinates, the center position of the circumscribed rectangle, or the like may be used as the character position coordinates.
As the character candidate position information 917, a relative positional relationship with respect to the size of the image 105 is designated. That is, in the normalized coordinate system in which the size of the image 1300 is set to 1 in the X-axis direction and 1 in the Y-axis direction as in the example shown in FIG. specify.
As the character candidate position information 917, for example, a range of 0.2 to 0.8 in the X axis direction and a range of 0.3 to 0.7 in the Y axis direction can be specified.
Similarly, the text string obtained by character recognition and the character position of each character position 915 need only be expressed in a coordinate system normalized in the X-axis direction and the Y-axis direction. A text string in which the character position exists in the area indicated by the character candidate position information 917 is extracted.
Or it may not be a normalized coordinate. The character position of the character candidate position information 917, the text string, and each character position 915 may be the pixel position itself in which the X coordinate and the Y coordinate are defined by integers.

<Fourth embodiment>
FIG. 16 is a conceptual module configuration diagram of a configuration example according to the fourth embodiment.
The information processing apparatus according to the fourth embodiment performs character recognition on an image obtained by photographing a document in which a display character string is described, and performs processing corresponding to the display character string. As shown, it has a character recognition module 110, a candidate size character extraction module 1620, a character string DB search module 530, a display character string / command table 140, and a command processing module 150. In particular, even if the number of registered display character strings is enormous, it operates in a realistic time.
In the third embodiment, the character candidate position information 917 is used for limitation, but in the fourth embodiment, character candidate size information 1617 is used for limitation.

What is necessary is just to prescribe | regulate a character size by the coordinate system which prescribes | regulates in 3rd Embodiment. As a result of character recognition by the character recognition module 110, the size (number of vertical pixels and number of horizontal pixels) of the circumscribed rectangle of each character is extracted. The number of pixels is normalized by the size of the image 105 (the number of vertical pixels and the number of horizontal pixels) to obtain a character size that can be handled in this embodiment.
The candidate size character extraction module 1620 receives the character size together with the text string as a character recognition result.
As the character candidate size information 1617, for example, threshold values such as a horizontal size range of 0.01 to 0.1 and a vertical size range of 0.01 to 0.1 are set.
If the character size as the character recognition result is within the range determined by the character candidate size information 1617, it is output to the character string DB search module 530 as the extracted character string 1625. Those not within this range are not output as the extracted character string 1625 (excluded as not including the display character string).
The relationship between the range defined by the character candidate size information 1617 and the character size may be any of the following.
(1) Only when the character size is within the range both vertically and horizontally, the corresponding character recognition result text string is output as the extracted character string 1625.
(2) If either the vertical or horizontal character size is within the range, the corresponding character recognition result text string is output as the extracted character string 1625.
In addition, as the character candidate size information 1617, the area of the circumscribed rectangle may be designated, or the size of the character string may be designated. When the character string is written horizontally, the range of the character string height may be specified. When the character string is vertically written, the range of the width of the character string may be specified.

The relationship and processing between modules shown in the example of FIG. 16 are as follows.
The character recognition module 110 is connected to the candidate size character extraction module 1620.
The candidate size character extraction module 1620 is connected to the character recognition module 110 and the character string DB search module 530. The candidate size character extraction module 1620 searches for a candidate as a display character string from the text string / each character size 1615 as a character recognition result. The candidates here are information (character candidate size information 1617) indicating the size range of display characters, and the text string / character size 1615 includes the size of each character. The extraction module 1620 searches the text string / each character size 1615 by extracting those whose recognized character size falls within the candidate range.
The character string DB search module 530 is connected to the candidate size character extraction module 1620, the display character string / command table 140, and the command processing module 150. The character string DB search module 530 searches the display character string from the candidates (extracted character string 1625) searched by the candidate size character extraction module 1620.
The display character string / command table 140 is connected to the character string DB search module 530. The command processing module 150 is connected to the character string DB search module 530.

FIG. 17 is a flowchart illustrating a processing example according to the fourth exemplary embodiment.
In step S1702, the character recognition module 110 receives the image 105. Processing equivalent to that in step S202 is performed.
In step S1704, the character recognition module 110 recognizes characters in the image 105 and extracts the size of each character. A process equivalent to step S204 is performed, but the size of the character is extracted together with the text information of each character.
In step S1706, the candidate size character extraction module 1620 receives the character candidate size information 1617.
In step S1708, the candidate size character extraction module 1620 extracts a character string (extracted character string 1625) corresponding to the character candidate size information 1617 in the text string / character size 1615 as a character recognition result.
In step S1710, the character string DB search module 530 extracts a display character string from the display character string / command table 140. Processing equivalent to that in step S206 is performed.

In step S1712, the display character string is searched from the extracted character string 1625. That is, the extracted character string 1625 is searched using the display character string extracted in step S1710 as a query. A general search algorithm may be used for the search. In step S1714, the character string search module 120 determines whether or not there is a display character string in the character recognition result (extracted character string 1625). Otherwise, the process proceeds to step S1720.
In step S <b> 1716, the character string search module 120 extracts a command 125 corresponding to the display character string searched from the display character string / command table 140. Processing equivalent to that in step S212 is performed.
In step S1718, the command processing module 150 interprets the command 125 and processes the command 125. Processing equivalent to that in step S214 is performed.
In step S1720, error processing is performed. Processing equivalent to that in step S216 is performed.

<Fifth embodiment>
FIG. 18 is a conceptual module configuration diagram of a configuration example according to the fifth embodiment.
The information processing apparatus according to the fifth embodiment performs character recognition on an image obtained by photographing a document in which a display character string is described, and performs processing corresponding to the display character string. As shown, it has a character recognition module 110, a candidate color character extraction module 1820, a character string DB search module 530, a display character string / command table 140, and a command processing module 150. In particular, even if the number of registered display character strings is enormous, it operates in a realistic time.
In the third embodiment, the character candidate position information 917 is used, and in the fourth embodiment, the character candidate size information 1617 is used. However, in the fifth embodiment, the character candidate color information 1817 (for example, black, red, etc.) is used. And the like).

The candidate color character extraction module 1820 receives the character color (text string / character color 1815) together with the text string as a character recognition result from the character recognition module 110. The candidate color character extraction module 1820 may extract the color of each character as a character recognition result from the character position obtained from the character position information in the character recognition result and the image 105.
A character color range is defined as the character candidate color information 1817. Only characters having a color within the range defined by the character candidate color information 1817 are output as the extracted character string 1825 to the character string DB search module 530. For example, the signal intensity of the color difference component is defined as the character candidate color information 1817. The character color is converted to YCbCr.
The color difference component intensity is defined as C = sqrt (Cb ² + Cr ² ). (Sqrt is a function for obtaining the square root)
As character candidate color information 1817, luminance information TY and a color component intensity threshold TC are set. When Y <TY and C <TC, the candidate color character extraction module 1820 extracts the corresponding text string. For example, this case corresponds to extracting only black characters.

The relationship and processing between modules shown in the example of FIG. 18 are as follows.
The character recognition module 110 is connected to the candidate color character extraction module 1820.
The candidate color character extraction module 1820 is connected to the character recognition module 110 and the character string DB search module 530. The candidate color character extraction module 1820 searches for a candidate as a display character string from the text string / character color 1815 as a character recognition result. The candidate here is information (character candidate color information 1817) indicating the color range of the display character, and the text string / each character color 1815 includes the color of each character. 1820 searches for a text string / each character color 1815 by extracting those whose recognized character color falls within the candidate range.
The character string DB search module 530 is connected to the candidate color character extraction module 1820, the display character string / command table 140, and the command processing module 150. The character string DB search module 530 searches the display character string from the candidates (extracted character string 1825) searched by the candidate color character extraction module 1820.
The display character string / command table 140 is connected to the character string DB search module 530. The command processing module 150 is connected to the character string DB search module 530.

FIG. 19 is a flowchart illustrating a processing example according to the fifth exemplary embodiment.
In step S1902, the character recognition module 110 receives the image 105. Processing equivalent to that in step S202 is performed.
In step S1904, the character recognition module 110 recognizes the image 105 and extracts the color of each character. A process equivalent to step S204 is performed, but the character color is extracted together with the text information of each character.
In step S1906, the candidate color character extraction module 1820 accepts the character candidate color information 1817.
In step S1908, the candidate color character extraction module 1820 extracts a character string (extracted character string 1825) corresponding to the character candidate color information 1817 in the text string / each character color 1815 as the character recognition result.
In step S1910, the character string DB search module 530 extracts a display character string from the display character string / command table 140. Processing equivalent to that in step S206 is performed.

In step S1912, the display character string is searched from the extracted character string 1825. That is, the extracted character string 1825 is searched using the display character string extracted in step S1910 as a query. The search may use a general search algorithm. In step S1914, the character string search module 120 determines whether or not there is a display character string in the character recognition result (extracted character string 1825). Otherwise, the process proceeds to step S1920.
In step S1916, the character string search module 120 extracts a command 125 corresponding to the display character string searched from the display character string / command table 140. Processing equivalent to that in step S212 is performed.
In step S 1918, the command processing module 150 interprets the command 125 and processes the command 125. Processing equivalent to that in step S214 is performed.
In step S1920, error processing is performed. Processing equivalent to that in step S216 is performed.

  A hardware configuration example of the information processing apparatus according to the present embodiment will be described with reference to FIG. The configuration illustrated in FIG. 20 is configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 2017 such as a scanner and a data output unit 2018 such as a printer.

  A CPU (Central Processing Unit) 2001 includes various modules described in the above-described embodiments, that is, a character recognition module 110, a character string search module 120, a command processing module 150, a regular expression search module 520, a character string DB search module. 530, a candidate position character extraction module 920, a candidate size character extraction module 1620, a candidate color character extraction module 1820, and the like are control units that execute processing according to a computer program describing an execution sequence of each module.

  A ROM (Read Only Memory) 2002 stores programs, calculation parameters, and the like used by the CPU 2001. A RAM (Random Access Memory) 2003 stores programs used in the execution of the CPU 2001, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 2004 including a CPU bus.

  The host bus 2004 is connected via a bridge 2005 to an external bus 2006 such as a PCI (Peripheral Component Interconnect / Interface) bus.

  A keyboard 2008 and a pointing device 2009 such as a mouse are input devices operated by an operator. The display 2010 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 2011 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 2001 and information. The hard disk includes an image 105, a text string 115, a command 125, a display character string / command table 140, a regular expression 517, an extracted character string 525, a text string / character positions 915, character candidate position information 917, a text string / characters. A size 1615, character candidate size information 1617, an extracted character string 1625, a text string / each character color 1815, character candidate color information 1817, an extracted character string 1825, and the like are stored. Further, various computer programs such as various other data processing programs are stored.

  The drive 2012 reads data or a program recorded in a removable recording medium 2013 such as a mounted magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read from the interface 2007 and an external bus 2006. , The bridge 2005, and the RAM 2003 connected via the host bus 2004. The removable recording medium 2013 can also be used as a data recording area similar to the hard disk.

  The connection port 2014 is a port for connecting the external connection device 2015, and has a connection unit such as USB, IEEE1394. The connection port 2014 is connected to the CPU 2001 and the like via the interface 2007, the external bus 2006, the bridge 2005, the host bus 2004, and the like. The communication unit 2016 is connected to a communication line and executes data communication processing with the outside. The data reading unit 2017 is a scanner, for example, and executes document reading processing. The data output unit 2018 is, for example, a printer, and executes document data output processing.

  Note that the hardware configuration of the information processing apparatus illustrated in FIG. 20 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 20, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 20 may be connected to each other via a communication line so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions of a scanner, a printer, a copying machine, a fax machine, etc.).

Note that the above-described various embodiments may be combined (for example, adding or replacing a module in one embodiment in another embodiment), and processing contents of each module The technique described in the background art may be employed. For example, two or more of the regular expression search module 520, the candidate position character extraction module 920, the candidate size character extraction module 1620, and the candidate color character extraction module 1820 may be combined.
Specifically, candidates may be extracted by combining regular expressions, character sizes, character positions, and character colors.
Restriction condition 1: Regular expression Restriction condition 2: Character size Restriction condition 3: Character position Restriction condition 4: When character color is used, only characters that match all restriction conditions are extracted as candidates. Of course, the limiting condition may be one, two, three, four, or more.
When a regular expression is not used, the character string DB search module 530 may search the display character string / command table 140 using words separated by morphological analysis or the like and using the separated words.

The character string search module 120 (character string DB search module 530) may output all of the commands 125 corresponding to the display character strings to the command processing module 150 when a plurality of display character strings hits the search. Then, only the first command 125 may be output to the command processing module 150.
In the above example, the URL is shown as the command 125. However, instead of the URL, an application name, an argument, or the like may be used as long as it can start a computer application.
Numbers are assigned to the rows of the tables shown in the examples of FIGS. 4 and 8, but this number may be omitted.
As the information processing apparatus, the character recognition module 110 is not essential. If the character recognition result can be received, similar processing and effects can be obtained.
By rewriting the contents of the display character string / command table 140, only the command (only the URL) is changed without changing the display character string (that is, without changing the content of the document to be photographed). Is possible.
If the regular expression does not represent all of the display character string, the display character string that is not expressed may be individually searched as a query.
In the above example, a regular expression is used, but a similar character string limiting method may be used. For example, a simple method of extracting only numbers and extracting them may be used.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 110 ... Character recognition module 120 ... Character string search module 140 ... Display character string and command table 150 ... Command processing module 520 ... Regular expression search module 530 ... Character string DB search module 920 ... Candidate position character extraction module 1620 ... Candidate size character extraction Module 1820 ... Candidate color character extraction module

Claims (7)

A receiving means for receiving the character recognition result;
Storage means for storing display character strings and processing information in association with each other;
Search means for searching for the display character string from the character recognition result;
An extraction means for extracting processing information corresponding to the display character string from the storage means when the display character string can be searched from the character recognition result by the search means;
An information processing apparatus comprising processing means for performing processing based on the processing information extracted by the extraction means. Candidate search means for searching for candidates as the display character string from the character recognition result further comprises:
The information processing apparatus according to claim 1, wherein the search unit searches the display character string from candidates searched by the candidate search unit. The candidate in the candidate search means is information described in a regular expression indicating a part or all of the display character string,
The information processing apparatus according to claim 2, wherein the candidate search unit searches the character recognition result by performing a regular expression search for the candidate. The candidate in the candidate search means is information indicating a range of positions where display characters exist, and the character recognition result includes the position of each character,
The information processing apparatus according to claim 2, wherein the candidate search unit searches the character recognition result by extracting a character whose recognized character position falls within the range of the candidate. . The candidate in the candidate search means is information indicating a range of display character size, and the character recognition result includes the size of each character,
The candidate search means searches for the character recognition result by extracting a character whose recognized character size falls within the candidate range. The information processing apparatus described in 1. The candidate in the candidate search means is information indicating a display character color range, and the character recognition result includes the color of each character,
The said candidate search means searches by extracting the thing from which the color of a recognition character corresponds to the said candidate range with respect to the said character recognition result. The Claim 1 characterized by the above-mentioned. The information processing apparatus described. Computer
A receiving means for receiving the character recognition result;
Storage means for storing display character strings and processing information in association with each other;
Search means for searching for the display character string from the character recognition result;
An extraction means for extracting processing information corresponding to the display character string from the storage means when the display character string can be searched from the character recognition result by the search means;
An information processing program for causing a processing unit to perform processing based on processing information extracted by the extraction unit.

Images