JP2015045984A - Information processing apparatus, character recognition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To further improve recognition accuracy.SOLUTION: Provided is an information processing apparatus 10 including: a storage unit 11 storing therein a plurality of first character strings L1 each expressed by characters in a set character group CG, and a first group G1 that is a collection of characters in the character group CG similar in shapes of character parts clipped from a first image P1 including the first character string L1 for each of the first character strings L1 while making each of the first character strings L1 correspond to the first group G1; and an arithmetic unit 12 generating a second group G2 that is a collection of characters in the character group CG similar in shapes of character parts clipped from a second image P2, selecting the first character string L1 based on the second character group G2, calculating an evaluation value representing a difference between the first group G1 corresponding to the selected first character group G1 and the second group G2, and determining the selected first character string L1 as a recognition result when the evaluation value is smaller than a set threshold.

Description

  The present invention relates to an information processing apparatus, a character recognition method, and a program.

  In recent years, many terminal devices such as mobile phones and smartphones have a camera function. Such terminal devices are equipped with various application software that reads information from images captured using the camera function. One of them is character recognition software that recognizes characters by cutting out character regions included in an image. For example, there is application software that reads character information such as names and telephone numbers from business card images. A technique for recognizing characters included in an image is called OCR (Optical Character Recognition).

  The OCR function is often installed in a reading device such as a document scanner. There is also application software that realizes the OCR function in a personal computer. The OCR function is a function of cutting out a character area from an image, collating an image pattern of the cut out character area with an image pattern of a registered character prepared in advance, and outputting the most similar registered character as a character recognition result. The accuracy and recognition rate of character recognition varies depending on the sharpness of the image and the type of registered characters. For this reason, various techniques have been proposed to increase the accuracy and recognition rate of character recognition.

  For example, a first method has been proposed in which a character recognition result and a registered word group are collated under a condition that allows N character recognition errors. Also, a second method has been proposed in which a handwritten character string recognition system that integrates character segmentation, character recognition, and language processing is constructed, and language information is fed back to character segmentation using the recognition system. In the second method, during character recognition, a plurality of character strings are generated as recognition result candidates, and a linguistically acceptable character string is output as a final recognition result.

  In addition, a search character string group X is created by associating a character string (hereinafter referred to as a misrecognized character string) that may include erroneous recognition of OCR with a correct character string (hereinafter referred to as a correct character string). A third method for performing character recognition using X has been proposed. In the third method, a search character string group Y is created by replacing one character from a character string included in the search character string group X with a wild card (a symbol representing an arbitrary character string), and a partial error is generated. A character string that matches the recognized character string can be searched. These techniques can be expected to improve character recognition accuracy and recognition rate.

JP-A-4-328682

IEICE Transactions, J69-D, no. 9, pp. 1292 (1986) Proceedings of the 51st IPSJ National Convention, pp. 4-247 (1995)

  However, in the above method, a character string written on a sheet is assumed to be recognized on the assumption that a person reads it as a document. Therefore, if the above method is applied to specially shaped characters (hereinafter referred to as design characters) including design elements described in signs, posters, etc., there may be an increase in character recognition failures and false detections. . This is due to the fact that many character recognition errors occur because the shape of the registered character and the shape of the design character differ beyond the allowable error range. Even when character recognition is performed on a character string described in a font different from the registered character font by the above-described method, there is a possibility that the number of character recognition failures and false detections increase similarly.

  For example, if the number N of characters that allow a character recognition error is increased in the first method, the number of characters to be compared with registered characters decreases, and the registered characters output as recognition results cannot be narrowed down. In addition, if the number of character strings generated as recognition result candidates in the second method is increased, the possibility of extracting linguistically acceptable character strings from candidates with low similarity is increased. Increased risk of false detection. In addition, when the third method is applied, if the same character string as the erroneously recognized character string is accidentally present in the recognition target image, the correct character string associated with the erroneously recognized character string is output as the recognition result. The That is, erroneous detection occurs.

  Therefore, according to one aspect, an object of the present invention is to provide an information processing apparatus, a character recognition method, and a program that can further improve recognition accuracy.

  According to one aspect of the present disclosure, a plurality of first character strings expressed by characters of a set character group and a plurality of first character strings are cut out from the first image including the first character string. A first collection of characters in a character group similar to the shape of the character portion, a storage unit in which the characters are stored in association with each other, and characters in a character group similar to the shape of the character portion cut out from the second image are collected A second set is generated, a first character string is selected based on the second set, and an evaluation value representing a difference between the first set and the second set corresponding to the selected first character string is calculated and set An information processing apparatus is provided that includes an arithmetic unit that uses a selected first character string as a recognition result when the evaluation value is smaller than a threshold value.

  Further, according to another aspect of the present disclosure, a computer that can access the storage unit stores a character string for each of a plurality of character strings represented by characters of a set character group and each of the plurality of character strings. The first set of characters in the character group similar to the shape of the character part cut out from the first image included is referred to the information in the storage unit stored in association with each other, and the character set cut out from the second image An evaluation representing a difference between the first set and the second set corresponding to the selected character string by generating a second set of characters of a character group similar to the shape, selecting a character string based on the second set A character recognition method is provided in which a value is calculated and the selected character string is a recognition result when the evaluation value is smaller than a set threshold value.

  Further, according to another aspect of the present disclosure, a computer that can access the storage unit is provided with a plurality of character strings represented by characters of a set character group and a character string for each of the plurality of character strings. The first set of characters in the character group similar to the shape of the character part cut out from the first image included is referred to the information in the storage unit stored in association with each other, and the character set cut out from the second image An evaluation representing a difference between the first set and the second set corresponding to the selected character string by generating a second set of characters of a character group similar to the shape, selecting a character string based on the second set A program is provided that calculates a value and executes a process of using a selected character string as a recognition result when the evaluation value is smaller than a set threshold value.

  According to the present disclosure, the recognition accuracy can be further improved.

It is the figure which showed an example of the information processing apparatus which concerns on 1st Embodiment. It is the figure which showed an example of the information provision system which concerns on 2nd Embodiment. It is the figure which showed an example of the hardware which can implement | achieve the function which the terminal device which concerns on 2nd Embodiment has. It is the block diagram which showed an example of the function which the terminal device which concerns on 2nd Embodiment has. It is the block diagram which showed an example of the function which the server apparatus which concerns on 2nd Embodiment has. It is the figure which showed an example of the information database which concerns on 2nd Embodiment. It is the figure which showed an example of the word database which concerns on 2nd Embodiment. It is the figure which showed an example of the error character string database which concerns on 2nd Embodiment. It is a 1st figure for demonstrating the word recognition process by the server apparatus which concerns on 2nd Embodiment. It is a 2nd figure for demonstrating the word recognition process by the server apparatus which concerns on 2nd Embodiment. It is a 3rd figure for demonstrating the word recognition process by the server apparatus which concerns on 2nd Embodiment. It is a 4th figure for demonstrating the word recognition process by the server apparatus which concerns on 2nd Embodiment. It is the 1st figure which showed the flow of the word recognition process which the server apparatus which concerns on 2nd Embodiment performs. It is the 2nd figure which showed the flow of the word recognition process which the server apparatus which concerns on 2nd Embodiment performs. It is the 3rd figure which showed the flow of the word recognition process which the server apparatus which concerns on 2nd Embodiment performs. It is the 4th figure which showed the flow of the word recognition process which the server apparatus which concerns on 2nd Embodiment performs. It is the 1st figure for explaining the verification method concerning one modification of a 2nd embodiment. It is a 2nd figure for demonstrating the verification method which concerns on the modification of 2nd Embodiment. It is a 3rd figure for demonstrating the verification method which concerns on the modification of 2nd Embodiment. It is the 1st figure which showed the flow of the update process of the error character database which concerns on 2nd Embodiment. It is the 2nd figure which showed the flow of the update process of the error character database which concerns on 2nd Embodiment. It is the 3rd figure which showed the flow of the update process of the error character database which concerns on 2nd Embodiment.

  Embodiments according to the present disclosure will be described below with reference to the accompanying drawings. In addition, about the element which has the substantially same function in this specification and drawing, duplication description may be abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. First Embodiment>
A first embodiment will be described.
An information processing apparatus 10 according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of an information processing apparatus according to the first embodiment.

As illustrated in FIG. 1, the information processing apparatus 100 includes a storage unit 11 and a calculation unit 12.
The storage unit 11 is a volatile storage device such as a RAM (Random Access Memory) or a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The arithmetic unit 12 is a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). However, the arithmetic unit 12 may be an electronic circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). For example, the calculation unit 12 executes a program stored in the storage unit 11 or another memory.

  The storage unit 11 stores a plurality of first character strings L1 expressed by characters of the set character group CG. In the example of FIG. 1, a character group CG including characters such as “snow”, “communication”, “disease”, “wealth”, “、” is set. The first character string L1 is a character string obtained by combining characters included in the character group CG. FIG. 1 shows an example of the first character string L1 such as a character string “Fujitsu”, which is a combination of “wet” and “tetsu” included in the character group CG. In the description according to the first embodiment, for convenience of description, the character string “Fujitsu” is given as an example of the first character string L1, and the description may be made with this example in mind.

  In addition, the storage unit 11 collects the characters of the character group CG similar to the shape of the character portion cut out from the first image P1 including the first character string L1 for each of the plurality of first character strings L1. A set G1 is stored.

  FIG. 1 exemplifies a first image P1 including the character string “Fujitsu”. The first image P1 is, for example, a captured image obtained by capturing a character string “Tomitori” described on a signboard installed on a street with a camera, or a character string “Tomitori” printed on paper with a set font. This is a captured image. A character string portion including the character string “Fujitsu” can be cut out from the first image P1 by image processing. Furthermore, the character part can be cut out character by character from the character string part. In the example of FIG. 1, the first character portion (character portion including “wealth”) and the second character portion (character portion including “to”) are cut out.

  When the image of the character part is cut out from the first image P1, for example, a character similar to the shape of the cut out character part can be extracted from the character group CG by a method such as pattern matching. For example, a method of preparing a template image of a character included in a character group CG or a feature amount thereof, calculating a similarity between a character part image and a template image, and extracting a character having a high similarity from the character group CG And so on. In addition, there is a method of extracting a character from the character group CG using the reliability indicating the high possibility that a certain character is a character described in the character portion (for example, expressed by a probability). Conceivable.

  In the method of extracting characters from the character group CG using index values such as similarity and reliability, a plurality of candidate characters are extracted from the character group CG in descending order of the index value. Then, the character having the largest index value is output as a result. The first set G1 is, for example, a set in which a set number of characters are collected for each extracted character portion in descending order of index values. The storage unit 11 stores a first set G1 obtained when extracting characters corresponding to the character portion of the first image P1 including the first character string L1 in association with the first character string L1. Yes.

  The calculation unit 12 generates a second set G2 in which characters of the character group CG similar to the shape of the character part cut out from the second image P2 are collected. The second image P2 is a captured image captured by a camera, for example. Further, the character string (“Futtsu”) included in the second image P2 is unknown. For example, the calculation unit 12 extracts a plurality of candidate characters from the character group CG in descending order of the index value by the same method as the method of generating the first set G1 from the first image P1 including the first character string L1. Then, a second set G2, which is a set of extracted characters, is generated.

  In addition, the calculation unit 12 selects the first character string L1 based on the second set G2. For example, the calculation unit 12 generates a character string by combining characters included in the second set G2, and selects a first character string L1 that matches the generated character string. Note that when the first character string L1 is not included in the second image P2, the calculation unit 12 may not be able to select the first character string L1 from the second set G2. However, when the first character string L1 is included in the second image P2, the calculation unit 12 is likely to be able to select the first character string L1 from the second set G2. In the example of FIG. 1, the character string “Tottori” can be generated by combining the characters included in the second image P2. In this case, the character string “Tottori” is selected as the first character string L1.

  In addition, the calculation unit 12 calculates an evaluation value representing a difference between the first set G1 and the second set G2 corresponding to the selected first character string L1. As the evaluation value, for example, the number of characters included in common between the first set G1 and the second set G2 can be used. If the number of characters included in common is small, it can be evaluated that the difference between the first set G1 and the second set G2 is large. In addition, when an index value is obtained for each character included in the first set G1 and the second set G2, the difference between the index values corresponding to the characters included in common can be used as the evaluation value. For example, if the difference in index value per character is large, it can be evaluated that the difference between the first set G1 and the second set G2 is large.

  In addition, when the evaluation value is smaller than the set threshold, the calculation unit 12 sets the selected first character string L1 as a recognition result. That is, when the difference between the first set G1 and the second set G2 is within the allowable range, the first character string L1 recognized from the second image P2 is output. If a character string is to be recognized from an image including special fonts or design characters, there is a possibility that erroneous detection or erroneous recognition occurs. However, sets of candidate characters obtained in the process of recognition processing for images containing the same character string have similar properties. Therefore, by performing the verification process based on the evaluation value as described above, it becomes possible to reduce the risk of erroneous detection and erroneous recognition, and improve the recognition accuracy of the character string.

The first embodiment has been described above.
<2. Second Embodiment>
Next, a second embodiment will be described.

[2-1. system]
First, an information providing system according to the second embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of an information providing system according to the second embodiment.

  As illustrated in FIG. 2, the information providing system according to the second embodiment includes a terminal device 100 and a server device 200. The terminal device 100 can communicate with the server device 200. For example, the terminal device 100 can perform data communication with the server device 200 using a wireless communication line such as a wireless local area network (LAN) or a mobile phone line.

  In this information providing system, when the photo PT is transmitted from the terminal device 100 to the server device 200, the registration information A related to the word included in the photo PT is provided from the server device 200 to the terminal device 100.

  The server apparatus 200 holds an information database 201a in which registration information A is recorded in association with words. Registered information A includes, for example, place name, shop name, sightseeing information, facility information (for example, facility name, entrance fee, opening time, address, telephone number, etc.), map information, product information, and discount information (for example, discount coupon) , Coupons, etc.). For example, in the information database 201a, registration information A such as a facility name “Fujien”, an entrance fee “500 yen”, and an opening time “9:00 to 17:00” is stored in association with the word “Fujien”. Yes.

  For example, when a photo PT such as a signboard or a poster is transmitted from the terminal device 100 to the server device 200, the server device 200 cuts out an image area WA including a character string from the received photo PT, and characters included in the character string are extracted. recognize. Furthermore, the server device 200 detects a word corresponding to the recognized combination of characters.

  The server device 200 holds a word database 201b in which words registered in the information database 201a are recorded. The server device 200 refers to the word database 201b and detects a word corresponding to the recognized combination of characters. The server device 200 that detects the corresponding word extracts the registration information A corresponding to the detected word from the information database 201a. Then, the server device 200 transmits the extracted registration information A to the terminal device 100. The terminal device 100 that has received the registration information A displays the received registration information A on the screen.

  As described above, when the information providing system according to the second embodiment is used, the user simply transmits a captured image obtained by capturing a character string such as a signboard to the server apparatus 200, and the registration information A related to the signboard or the like. Can be easily obtained. Special information display such as a two-dimensional bar code is not used, and general character strings need only be described. Therefore, signs installed in various places can be used as information sources. Therefore, the registration information A can be easily obtained by using a signboard that happens to be displayed, a word described in an advertisement, or the like, which is highly convenient.

  However, characters such as signs are often drawn with special fonts and design characters. When using the OCR function that reads character strings written on books or written documents such as application documents and records character information as digital data, characters with special shapes drawn on signs etc. There is a high possibility that it cannot be recognized correctly. It is assumed that the word cannot be recognized or a word different from the correct word is detected. In view of this, the second embodiment proposes a mechanism for correctly recognizing words including characters that are prone to character recognition errors, such as special fonts and design characters. By applying this mechanism, word recognition accuracy is improved, and by applying to the information providing system illustrated in FIG. 2, the user's convenience is improved.

  The information providing system according to the second embodiment has been described above. Of the functions of the server device 200, a function related to character recognition may be incorporated in the terminal device 100 so that the character string can be recognized from the captured image by the terminal device 100 alone. Further, the information in the information database 201a of the server device 200 may be held in the terminal device 100 so that information such as tourism information can be obtained from the recognized character string by the terminal device 100 alone. In the following, the description will proceed with the information providing system shown in FIG. 2 in mind, but such a modification is also included in the technical scope of the second embodiment.

[2-2. hardware]
Next, the hardware of the terminal device 100 will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of hardware capable of realizing the functions of the terminal device according to the second embodiment. The functions of the terminal device 100 can be realized using, for example, hardware resources of the information processing apparatus illustrated in FIG. That is, the functions of the terminal device 100 are realized by controlling the hardware shown in FIG. 3 using a computer program.

  As shown in FIG. 3, this hardware mainly includes a CPU 902, a ROM (Read Only Memory) 904, a RAM 906, a host bus 908, and a bridge 910. The hardware further includes an external bus 912, an interface 914, an input unit 916, an output unit 918, a storage unit 920, a drive 922, a connection port 924, and a communication unit 926.

  The CPU 902 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation of each component or a part thereof based on various programs recorded in the ROM 904, the RAM 906, the storage unit 920, or the removable recording medium 928. . The ROM 904 is an example of a storage device that stores a program read by the CPU 902, data used for calculation, and the like. The RAM 906 temporarily or permanently stores, for example, a program read by the CPU 902 and various parameters that change when the program is executed.

  These elements are connected to each other via, for example, a host bus 908 capable of high-speed data transmission. On the other hand, the host bus 908 is connected to an external bus 912 having a relatively low data transmission speed via a bridge 910, for example. As the input unit 916, for example, a mouse, a keyboard, a touch panel, a touch pad, a button, a switch, a lever, or the like is used. Furthermore, as the input unit 916, a remote controller capable of transmitting a control signal using infrared rays or other radio waves may be used.

  As the output unit 918, for example, a display device such as a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), or an ELD (Electro-Luminescence Display) is used. As the output unit 918, an audio output device such as a speaker or headphones, or a printer may be used. In other words, the output unit 918 is a device that can output information visually or audibly.

  The storage unit 920 is a device for storing various data. As the storage unit 920, for example, a magnetic storage device such as an HDD is used. Further, as the storage unit 920, a semiconductor storage device such as an SSD (Solid State Drive) or a RAM disk, an optical storage device, a magneto-optical storage device, or the like may be used.

  The drive 922 is a device that reads information recorded on a removable recording medium 928 that is a removable recording medium or writes information on the removable recording medium 928. As the removable recording medium 928, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is used.

  The connection port 924 is a port for connecting an external connection device 930 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, and an optical audio terminal. Examples of the external connection device 930 include an earphone, a camera, a flash, and a printer. Note that a device such as a camera or a flash may be connected to the interface 914 without using the connection port 924.

  The communication unit 926 is a communication device for connecting to the network 932. Examples of the communication unit 926 include a wired or wireless LAN communication circuit, a WUSB (Wireless USB) communication circuit, an optical communication circuit or router, an ADSL (Asymmetric Digital Subscriber Line) communication circuit or router, A communication circuit for a cellular phone network is used. A network 932 connected to the communication unit 926 is a wired or wireless network, and includes, for example, the Internet, a LAN, a broadcast network, a satellite communication line, and the like.

  The hardware of the terminal device 100 has been described above. Note that the functions of the server device 200 can also be realized using the hardware illustrated in FIG. 3, similarly to the terminal device 100. Therefore, detailed description of the hardware of the server device 200 is omitted.

[2-3. Terminal device functions]
Next, functions of the terminal device 100 will be described with reference to FIG. FIG. 4 is a block diagram illustrating an example of functions of the terminal device according to the second embodiment.

  As illustrated in FIG. 4, the terminal device 100 includes an image acquisition unit 101, an image transmission unit 102, an information reception unit 103, and an information display unit 104. The functions of the image transmission unit 102 and the information reception unit 103 can be realized using the connection port 924, the communication unit 926, and the like described above. The functions of the image acquisition unit 101 and the information display unit 104 can be realized using the above-described CPU 902 or the like.

  The image acquisition unit 101 uses an image (such as a captured image captured using the camera function of the terminal device 100 or a captured image stored in a removable recording medium 928 connected to the terminal device 100) to recognize words ( Hereinafter, the target image) is acquired. The target image acquired by the image acquisition unit 101 is input to the image transmission unit 102. The image transmission unit 102 transmits the target image input from the image acquisition unit 101 to the server device 200.

  When a word is detected from the target image in the server device 200, registration information A corresponding to the detected word is transmitted to the terminal device 100. The information receiving unit 103 receives the registration information A transmitted from the server device 200. The registration information A received by the information receiving unit 103 is input to the information display unit 104. The information display unit 104 displays the registration information A input from the information receiving unit 103 on the screen of the terminal device 100.

The function of the terminal device 100 has been described above.
[2-4. Server device functions]
Next, functions of the server device 200 will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of functions of the server device according to the second embodiment. In the description, FIGS. 6 to 12 are appropriately referred to.

  As illustrated in FIG. 5, the server device 200 includes a storage unit 201, an image reception unit 202, a word recognition unit 203, an information transmission unit 204, and a database update unit 205. The function of the storage unit 201 can be realized using the above-described RAM 906, the storage unit 920, or the like. The functions of the image reception unit 202 and the information transmission unit 204 can be realized using the connection port 924, the communication unit 926, and the like described above. The function of the word recognition unit 203 can be realized using the above-described CPU 902 or the like.

(Storage unit 201)
The storage unit 201 includes an information database 201a, a word database 201b, a registered character database 201c, and an error character string database 201d.

  The information database 201a is a database in which registration information A is stored in association with words. The word database 201b is a database that stores information for detecting words from the result of character recognition. The registered character database 201c is a database in which templates of characters used for character recognition (hereinafter referred to as registered characters) are stored. The error character string database 201d is a database that stores information on recognition results when characters are recognized for an image in which an erroneous character string is easily detected.

(Information database 201a)
Here, the information database 201a will be further described with reference to FIG. FIG. 6 is a diagram illustrating an example of an information database according to the second embodiment.

  The information database 201a illustrated in FIG. 6 has columns of “word” and “information”. Also, in the “word” column, words such as “Fujien”, “Snow Country”, “Aizen Myoo Statue” and the like are described. However, the word described in the “word” column is a word stored in the word database 201b.

  Furthermore, in the “information” column, registration information A such as “admission fee 500 yen, opening hours 9 to 17:00” is described for each word. The numbers described in the “No.” column are numbers for identifying records in the information database 201a. As in this example, the information database 201a stores words and registration information A in association with each other.

(Word database 201b)
Next, the word database 201b will be further described with reference to FIG. FIG. 7 is a diagram showing an example of a word database according to the second embodiment.

  The word database 201b illustrated in FIG. 7 includes columns of “word”, “candidate rank”, “first character”, “second character”, “third character”,. In the following, the “first character”, “second character”, and “third character” fields may be referred to as “candidate character group” fields. In the “word” column, words such as “Fujien” and “Snow Country” are written. In the “candidate ranking” column, “first place”,..., “Fourth place” is described for each word. In the “candidate character group” column, the character type and cost are described.

  The word described in the “word” column is a word stored in the word database 201b. The character type and cost described in the “candidate character group” column are the characters extracted as candidates of the recognition result (hereinafter referred to as candidate characters) and the recognition cost when the character recognition is performed from the image including the corresponding word. Represents a value.

  Note that the recognition cost is an example of an index value indicating a high possibility that a candidate character is erroneous. That is, the recognition cost is an amount indicating that the smaller the value is, the more likely it is to be a correct character (hereinafter, correct character), and it is sometimes called the difference degree. In the second embodiment, an example in which the recognition cost is used as an index value will be mainly described. However, for example, a recognition score can be used as an index value. The recognition score is an example of an index value that represents a high possibility that the candidate character is a correct character. That is, the recognition score is an amount that indicates that the correct character is more likely to increase as the value increases, and is sometimes referred to as similarity. The degree of similarity may be represented by a certainty factor representing likelihood, a probability value representing the probability of being a correct character, or the like.

  In the example of FIG. 7, four candidate characters are described for each character included in the word. For example, “wealth”, “snow”, “window”, “冨” are described as candidate characters corresponding to the first character “wealth” of the word “Fujien”. In the example of FIG. 7, the top four candidate characters are listed in ascending order of cost. The rank number described in the “candidate rank” column is a rank representing the order in ascending order of cost.

  In the example of FIG. 7, candidate characters up to the fourth place are described, but it is possible to set the candidate characters up to the third place, and candidate characters lower than the fourth place are displayed. It is also possible to make the setting described. As described above, the word database 201b stores candidate character groups together with words.

(Error character string database 201d)
Next, the error character string database 201d will be further described with reference to FIG. FIG. 8 is a diagram showing an example of an error character string database according to the second embodiment.

  The error character string database 201d illustrated in FIG. 8 includes columns of “word”, “error character string”, “candidate rank”, “first character”, “second character”, “third character”,. Is provided. The “first character”, “second character”, and “third character” fields are “candidate character group” fields. In the “word” column, a word such as “Fujien” is written. In the “error character string” column, a character string is described for each word. In the “candidate rank” column, “first place”,..., “Fourth place” is described for each word. In the “candidate character group” column, the character type and cost are described.

  The word described in the “word” column is a word stored in the word database 201b. In the “word” column, the number of error character strings is described for each word. The character string described in the “error character string” column represents a character string (hereinafter referred to as an error character string) erroneously recognized when character recognition is performed from an image including the corresponding word.

  For example, if there is no correct character in the candidate character group, a combination of candidate characters with the highest candidate ranking is obtained as an error character string. The character type and cost described in the “candidate character group” column represent candidate characters and recognition cost values extracted when character recognition is performed from an image including a corresponding word. In addition, it can replace with recognition cost and can utilize a recognition score as an index value.

  In the example of FIG. 8, two error character strings “Shizukuni” and “Snow Country” are described in association with the word “Fujien”. In addition, four candidate characters are described for each character included in the error character string “蔦 contend country”. For example, “蔦”, “dream”, “word”, “snow” with respect to the first character “の” of the error character string “Tsukukoku” corresponding to the first character “Wealth” of the word “Fujien” Is described.

  In the example of FIG. 8, candidate characters up to the fourth place are described, but it is possible to set the candidate characters up to the third place, and candidate characters lower than the fourth place are displayed. It is also possible to make the setting described. Further, the number of candidate characters (number of ranks) described in the word database 201b and the number of candidate characters (number of ranks) described in the error character string database 201d may be set to different numbers. For example, the word database 201b can be set so that candidate characters up to the fourth place are considered, and the error character string database 201d can be set up to consider the candidate characters up to the eighth place.

(Image receiving unit 202)
Refer to FIG. 5 again. The image receiving unit 202 receives a photo PT that is a word recognition target from the terminal device 100. The photo PT received by the image receiving unit 202 is input to the word recognition unit 203. Note that the photo PT received by the image receiving unit 202 may be temporarily stored in the storage unit 201.

(Word recognition unit 203)
The word recognition unit 203 detects a word from the photo PT input by the image reception unit 202. For example, the word recognition unit 203 cuts out the image area WA including the character string from the photograph PT. For example, the image area WA may be cut out by binarizing the image data of the photograph PT and cutting out a character string from the binary image. The word recognition unit 203 that cuts out the image area WA uses the registered character database 201c to cut out a portion including each character (hereinafter, a character area) from the image area WA. Any character recognition technique disclosed at present or in the future can be applied as a method for extracting a character region.

  The word recognition unit 203 that cuts out the character area matches each registered character with the character shape of the character area using the registered character template stored in the registered character database 201c, and calculates the recognition cost for each registered character. To do. As a method for calculating the recognition cost, any method disclosed at present or in the future can be applied. If the index values described in the word database 201b and the error character string database 201d are recognition scores, the recognition scores are calculated instead of the recognition costs.

  The word recognizing unit 203 extracts a preset number of registered characters as candidate characters in ascending order of recognition cost. That is, a candidate character group including a plurality of candidate characters for each character region is generated. The word recognizing unit 203 that has generated the candidate character group combines the candidate characters included in the candidate character group and detects the words described in the word database 201b. For example, the word recognition unit 203 detects a word by a method illustrated in FIG. FIG. 9 is a first diagram for explaining word recognition processing by the server device according to the second embodiment.

  FIG. 9 shows an example of processing for detecting a word from an image P01 that is an image area WA cut out from the photograph PT. In the image P01, the word “Fujien” is described. Further, “冨”, “wealth”, “snow”, and “word” are obtained in order from the top as candidate characters for the character region including the first character “wealth”. Similarly, “shi”, “sat”, “engineering”, and “large” are obtained in order from the top as candidate characters for the character area including the second character “shi”. Furthermore, “country”, “garden”, “times”, and “酉” are obtained in order from the top as candidate characters for the character area including the third character “garden”.

  Combining the candidate characters positioned at the top of the candidate character group, which is the character recognition result, results in “Fujishikuni” and not “Fujien” which is the correct character string. However, if the candidate character group including the lower candidate characters is referred to, the first character “wet”, the second character “shi”, and the third character “ Garden "is obtained. As illustrated in FIG. 9, the word recognition unit 203 detects a word described in the word database 201 b from a candidate character group including lower candidate characters. In the example of FIG. 9, “Fujien” is correctly detected as the word recognition result.

  When the word described in the word database 201b cannot be detected by the above method, the word recognizing unit 203 combines the candidate characters included in the candidate character group to obtain the error character string described in the error character string database 201d. To detect. For example, the word recognition unit 203 detects an error character string by a method illustrated in FIG. FIG. 10 is a second diagram for explaining word recognition processing by the server device according to the second embodiment.

  FIG. 10 shows an example of processing for detecting a word from the image P02 that is the image area WA cut out from the photograph PT. In the image P02, the word “Fujien” is described. However, each character described in the image P02 is unclear.

  Therefore, for example, “蔦”, “dream”, “word”, and “snow” are obtained in order from the top as candidate characters for the character region including the first character “wealth”. Further, “plan”, “shi”, “description”, and “engineer” are obtained in order from the top as candidate characters for the character region including the second character “shi”. Furthermore, “country”, “酉”, “field”, and “tool” are obtained in order from the top as candidate characters for the character region including the third character “garden”.

  Combining the candidate characters positioned at the top of the candidate character group, which is the character recognition result, results in a “contemporary country” and not the correct character string “Fujien”. Furthermore, even if the candidate character group including the lower candidate characters is referred to, the first character “Wealth” and the third character “Garden” of the word “Fujien” included in the word database 201b cannot be obtained. However, the character string “蔦 contend country” combining the candidate characters positioned at the top of the candidate character group is described in the error character string database 201d. Therefore, the word recognition unit 203 refers to the error character string database 201d, and uses the word “Fujien” corresponding to the error character string “invention country” as the word recognition result. By this method, in the example of FIG. 10, “Fujien” is correctly detected as the word recognition result.

(Discussion # 1: When an error character string is included in the photo PT)
Here, let us consider a case where the photo PT contains the same word as the error character string described in the error character string database 201d. For example, in the case where a photograph PT obtained by photographing a signboard including the word “a project country” not described in the word database 201b is input, the word recognition unit 203 stores the error character string database 201d in the same manner as in the example of FIG. Using the candidate character group, the error character string “Tentative Country” is detected. In this example, since the correct word is “a plan country”, an error occurs if the word “Fujien” corresponding to this detection result is used as the word recognition result.

  Therefore, the word recognition unit 203 verifies whether or not the word “Fujizono” corresponding to the error character string “Foster Country” is a correct word recognition result. If the correct word recognition result is recognized, the word recognizing unit 203 sets the word “Fujien” corresponding to the error character string “Zhigokoku” as the word recognition result. On the other hand, when the result is not recognized as a correct word recognition result, the word recognition unit 203 does not use the word “Fujien” corresponding to the error character string “Foster Country” as the word recognition result. In this case, the word recognition unit 203 determines that word recognition has failed.

(Discussion # 2: Candidate character group includes multiple words)
Here, let us consider a case where a plurality of words are detected from the candidate character group. For example, when the character recognition result (candidate character group) shown in FIG. 9 is obtained, two words “Fujien” and “Snow Country” are detected from the candidate character group.

  Therefore, the word recognition unit 203 verifies which of the two words “Fujien” and “Snow Country” is the correct word recognition result. As a method of selecting a word, for example, a method of selecting a word having the smallest sum or average of recognition costs can be considered. However, in the second embodiment, a verification method using a candidate character group is proposed. The word recognition unit 203 sets a word recognized as a correct word recognition result as a word recognition result. If none of the words is recognized as a correct word recognition result, the word recognition unit 203 determines that word recognition has failed.

(Discussion # 3: When the same registered word as the error character string exists)
Here, let us consider a case where the same word as the error character string described in the error character string database 201d is described in the word database 201b.

  In the above description, the method of detecting an error character string from the candidate character group when the word described in the word database 201b is not included in the candidate character group is illustrated. However, the same character string as the word described in the word database 201b may be included in the candidate character group, but the word may not be correct. For this reason, a method is considered in which a word described in the word database 201b and an error character string described in the error character string database 201d are detected from the candidate character group, and a correct character recognition result is selected from each detection result.

  For example, the word “snow country” is described in the word database 201b illustrated in FIG. 7, and the error character string “snow country” is described in the error character string database 201d illustrated in FIG. In this case, as shown in FIG. 11, the word recognition unit 203 recognizes which of the word recognition result # 1 based on the word described in the word database 201b and the word recognition result # 2 based on the error character string is correct. Verify whether it is a result. FIG. 11 is a third diagram for explaining word recognition processing by the server device according to the second embodiment. In the example of FIG. 11, as a result of the verification, the word recognition result # 2 based on the error character string is obtained as a correct word recognition result.

(About verification method)
As described in the above considerations # 1 to # 3, the word recognizing unit 203 performs the verification process without directly using the word detected based on the word database 201b and the error character string database 201d as a character recognition result. Here, the verification process by the word recognition unit 203 will be described with reference to FIG. FIG. 12 is a fourth diagram for explaining word recognition processing by the server device according to the second embodiment.

(Verification of word recognition results based on registered words)
When a word described in the word database 201b is detected from a character recognition result (hereinafter referred to as a candidate character group X) for the image area WA, the word recognition unit 203 selects a candidate character described in the word database 201b corresponding to the word. A group (hereinafter, candidate character group Y) is extracted. Next, the word recognition unit 203 compares the two candidate character groups X and Y, and counts the number of candidate characters (hereinafter, common candidate characters) that exist in common in both.

  Next, the word recognition unit 203 determines whether or not the number of common candidate characters is larger than a set threshold value (for example, 7). The number of common candidate characters is an example of an evaluation value. When the number of common candidate characters is larger than the set threshold value, the word recognition unit 203 uses a word detected from the candidate character group X as a character recognition result. On the other hand, when the number of common candidate characters is smaller than the set threshold, the word recognition unit 203 does not use the word detected from the candidate character group X as the character recognition result. The threshold is set based on the number of candidate characters included in one or both of candidate character groups X and Y, for example.

  In the example of FIG. 12, the result of comparing the candidate character group X (“character recognition result” in the figure) with the candidate character group Y (from “word DB” in the figure) corresponding to the word “snow country”. Five common candidate characters are obtained. When the threshold is set to 7, the number of common candidate characters is smaller than the threshold, so the word “snow country” is not regarded as a word recognition result. When a plurality of words are detected from the candidate character group X, a word whose number of common candidate characters is larger than the threshold is determined as a character recognition result.

(Verification of word recognition results based on error strings)
When an error character string described in the error character string database 201d is detected from a character recognition result (candidate character group X) for the image area WA, the word recognition unit 203 selects a candidate character group ( Hereinafter, the candidate character group Z) is extracted from the error character string database 201d. Next, the word recognition unit 203 compares the two candidate character groups X and Z, and counts the number of candidate characters (common candidate characters) that exist in common in both.

  Next, the word recognition unit 203 determines whether or not the number of common candidate characters is larger than a set threshold value (for example, 7). When the number of common candidate characters is larger than the set threshold value, the word recognition unit 203 sets a word corresponding to the error character string detected from the candidate character group X as a character recognition result. On the other hand, when the number of common candidate characters is smaller than the set threshold value, the word recognition unit 203 does not use the word corresponding to the error character string detected from the candidate character group X as the character recognition result. The threshold is set based on the number of candidate characters included in one or both of candidate character groups X and Z, for example.

  In the example of FIG. 12, a candidate character group X (“character recognition result” in the figure), a candidate character group Z (from “error character string DB” in the figure) corresponding to the error character string “snow country”, and As a result of comparison, nine common candidate characters are obtained. When the threshold value is set to 7, the number of common candidate characters is larger than the threshold value, so the word “Fujien” corresponding to the error character string “Snow Country” is set as the word recognition result. In the example of FIG. 12, a candidate character group Y (from “word DB” in the figure) and a candidate character group Z (from “error character string DB” in the figure) are obtained. The word “Fujien” corresponding to “country” is taken as the word recognition result.

(About implementation of verification process)
Here, an implementation form of the above verification processing will be described. In the above description, the application method of the verification process has been discussed in various cases. However, when actually implementing the verification process, a part of the verification process can be omitted. For example, when a word described in the word database 201b is detected from the candidate character group X, the detection result may not be verified. In this case, when an error character string is detected from the candidate character group, the above verification is performed on the word corresponding to the error character string. In this way, the balance between recognition accuracy and processing load can be adjusted by selecting an application method in consideration of the degree of contribution to recognition accuracy and the like.

  Refer to FIG. 5 again. When the word recognition is successful, the word recognition unit 203 inputs a word as a word recognition result to the information transmission unit 204. On the other hand, when the word recognition fails, the word recognition unit 203 inputs information indicating that the word recognition has failed to the information transmission unit 204. In this case, the word recognizing unit 203 inputs information indicating that the word recognition has failed and information on candidate character groups detected from the image area WA to the database updating unit 205.

(Information transmission unit 204)
When a word recognition result (word) is input from the word recognition unit 203, the information transmission unit 204 extracts registration information A corresponding to the input word from the information database 201a. Then, the information transmission unit 204 transmits the registration information A extracted from the information database 201a to the terminal device 100. On the other hand, when information indicating that word recognition has failed is input from the word recognition unit 203, the information transmission unit 204 notifies the terminal device 100 that word recognition has failed.

(Database update unit 205)
The database update unit 205 updates the error character string database 201d. For example, when information indicating that word recognition has failed and information on candidate character groups detected from the image area WA are input from the word recognition unit 203, the database update unit 205 uses the input information to generate an error character string. The database 201d is updated.

  As an update method, for example, a method of generating an error character string combining the highest candidate character from the input candidate character group, and storing the candidate character group and the error character string in association with each other in the error character string database 201d Can be considered. This method is applied when the database update unit 205 can acquire the correct word information included in the image area WA. For example, when the correct word information is notified from the terminal device 100 to the server device 200, the database update unit 205 stores the generated error character string and the candidate character group in association with the notified word. The same applies when correct word information is directly input to the server apparatus 200.

  The error character string database 201d may be updated when the word recognition is successful. For example, when the error character string generated from the candidate character group already exists in the error character string database 201d, the candidate character group in the error character string database 201d is updated using the cost of the candidate character obtained from the image area WA. Can be considered. For example, an average value of the cost of the candidate character obtained from the image area WA and the cost described in the error character string database 201d corresponding to the same candidate character as the candidate character is calculated, and the error character string database 201d is calculated by the average value. The method of updating the description of can be considered. When this method is applied, the candidate rank of each candidate character is also updated based on the updated cost.

The function of the server device 200 has been described above.
[2-5. About recognition processing]
Next, the flow of word recognition processing executed by the server device 200 will be described with reference to FIGS. FIG. 13 is a first diagram illustrating a flow of word recognition processing executed by the server device according to the second embodiment. FIG. 14 is a second diagram illustrating a flow of word recognition processing executed by the server device according to the second embodiment. FIG. 15 is a third diagram illustrating the flow of word recognition processing executed by the server device according to the second embodiment. FIG. 16 is a fourth diagram illustrating a flow of word recognition processing executed by the server device according to the second embodiment.

(Overall processing flow)
First, the overall processing flow will be described with reference to FIG.
(S101) The image receiving unit 202 receives a photo PT that is a word recognition target from the terminal device 100. The photo PT received by the image receiving unit 202 is input to the word recognition unit 203. Note that the photo PT received by the image receiving unit 202 may be temporarily stored in the storage unit 201.

  (S102) The word recognition unit 203 cuts out an image area WA including a character string from the photograph PT. The word recognition unit 203 that cuts out the image area WA cuts out a portion (character area) including each character from the image area WA.

  (S103) The word recognition unit 203 that cuts out the character area performs matching between each registered character and the character shape of the character area using the registered character template stored in the registered character database 201c, and recognizes each registered character. Calculate the cost. Further, the word recognition unit 203 extracts a preset number of registered characters as candidate characters in order from the lowest recognition cost. That is, a candidate character group including a plurality of candidate characters for each character region is generated.

  (S104) The word recognizing unit 203 that has generated the candidate character group detects a word described in the word database 201b based on the candidate character included in the candidate character group. However, if the word described in the word database 201b is not detected even if the candidate characters included in the candidate character group are combined, the word recognition unit 203 uses the error character string database 201d to detect the word from the candidate character group. . Further, the word recognition unit 203 verifies whether or not the detected word is a correct word recognition result. The flow of processing related to word detection will be described in detail later.

  (S105) The word recognition unit 203 determines whether or not the word detection is successful. If the word detection is successful, the process proceeds to S107. On the other hand, if the word detection fails, the process proceeds to S106. In this case, information indicating that word recognition has failed from the word recognition unit 203 is input from the word recognition unit 203 to the information transmission unit 204.

(S106) The information transmission unit 204 notifies the terminal device 100 that word recognition has failed. When the process of S106 is completed, the series of processes illustrated in FIG.
(S107) When a word recognition result (word) is input from the word recognition unit 203, the information transmission unit 204 extracts registration information A corresponding to the input word from the information database 201a. Then, the information transmission unit 204 transmits the registration information A extracted from the information database 201a to the terminal device 100. When the process of S107 is completed, the series of processes shown in FIG.

(Flow of processing related to word detection)
Here, the flow of processing relating to word detection will be further described with reference to FIGS. 14 and 15. Note that the processing illustrated in FIGS. 14 and 15 corresponds to the processing of S104.

  (S111) The word recognition unit 203 selects a word described in the word database 201b. For example, when M character areas (M is a natural number of 2 or more) are cut out from the image area WA, the word recognition unit 203 selects one word of M letters. At this time, the word recognition unit 203 selects an unselected word in the series of processes shown in FIGS.

  (S112) The word recognition unit 203 searches for the word selected in S111 from the combination of characters (candidate character group) recognized from the image area WA. For example, the word recognition unit 203 selects the first character from the word, and extracts the selected first character from the first candidate character. Similarly, the word recognition unit 203 selects the kth character (k = 2,..., M), and extracts the selected kth character from the kth candidate character.

  When all characters from the first character to the M-th character can be extracted from the candidate character group, the word recognition unit 203 determines that the word selected in S111 has been detected. On the other hand, if even one character of the word cannot be extracted from the candidate character group among the first to Mth characters, the word recognition unit 203 determines that the word selected in S111 could not be detected. .

  (S113) If the word recognition unit 203 detects a word in the process of S112, the process proceeds to S120 (see FIG. 15). On the other hand, if the word recognition unit 203 does not detect a word in the process of S112, the process proceeds to S114.

  (S114) The word recognition unit 203 determines whether or not the processing has been completed for all the words described in the word database 201b. If the processing is completed for all the words described in the word database 201b, the processing proceeds to S115. On the other hand, when the processing has not been completed for all the words described in the word database 201b, the process proceeds to S111. For example, when M character areas are cut out from the image area WA, if there are M character words not selected in the process of S111, the process proceeds to S111.

  (S115) The word recognition unit 203 selects an error character string described in the error character string database 201d. For example, when M character areas are cut out from the image area WA, the word recognition unit 203 selects one error character string of M characters. At this time, the word recognition unit 203 selects an unselected error character string in the series of processes shown in FIGS.

  (S116) The word recognizing unit 203 searches for the error character string selected in S115 from the combination of characters (candidate character group) recognized from the image area WA. For example, the word recognition unit 203 selects the first character from the erroneous character string, and extracts the selected first character from the first candidate character. Similarly, the word recognition unit 203 selects the kth character (k = 2,..., M), and extracts the selected kth character from the kth candidate character.

  If all characters from the first character to the Mth character can be extracted from the candidate character group, the word recognition unit 203 determines that the error character string selected in S115 has been detected. On the other hand, if even one character in the error character string cannot be extracted from the candidate character group among the first to Mth characters, the word recognition unit 203 can detect the error character string selected in S115. Judge that there was no.

  (S117) If an error character string is detected by the word recognition unit 203 in the process of S116, the process proceeds to S123 (see FIG. 15). On the other hand, if no error character string is detected in the process of S116 by the word recognition unit 203, the process proceeds to S118.

  (S118) The word recognition unit 203 determines whether or not the processing has been completed for all error character strings described in the error character string database 201d. When the process is completed for all error character strings described in the error character string database 201d, the process proceeds to S119. On the other hand, if the processing has not been completed for all error character strings described in the error character string database 201d, the process proceeds to S115. For example, when M character areas are cut out from the image area WA, if there are M character error character strings not selected in the process of S115, the process proceeds to S115.

  (S119) The word recognition unit 203 determines that word detection has failed. In this case, the word recognition unit 203 inputs information indicating that the word recognition has failed to the information transmission unit 204. In addition, the word recognition unit 203 inputs information indicating that word recognition has failed and information on candidate character groups detected from the image area WA to the database update unit 205. If correct word information is obtained, the database update unit 205 updates the error character string database 201d. When the process of S119 is completed, the series of processes shown in FIGS. 14 and 15 ends.

  (S120) The word recognizing unit 203 executes detection result verification processing. For example, the word recognition unit 203 extracts the candidate character group Y corresponding to the detected word from the word database 201b and compares it with the character recognition result (candidate character group X) for the image area WA. The word recognition unit 203 counts the number of common candidate characters common to the two candidate character groups X and Y. If the number of common candidate characters is greater than the threshold, the word recognition unit 203 determines that the detection result is correct. On the other hand, when the number of common candidate characters is smaller than the threshold, the word recognition unit 203 determines that the detection result is not correct.

  (S121) If it is determined in step S120 that the detection result is correct, the process proceeds to step S122. On the other hand, if it is determined in S120 that the detection result is not correct, the process proceeds to S115 (see FIG. 14).

  (S122) The word recognition unit 203 determines that the word detection has succeeded. When the process of S122 is completed, the series of processes shown in FIGS. 14 and 15 ends. Note that when a word described in the word database 201b is detected and the detection result is trusted and the verification process is omitted, the processes of S120 and S121 may be omitted. This omission reduces the processing load associated with word recognition.

  (S123) The word recognizing unit 203 executes detection result verification processing. For example, the word recognition unit 203 extracts the candidate character group Z corresponding to the detected error character string from the error character string database 201d, and compares it with the character recognition result (candidate character group X) for the image area WA. Then, the word recognition unit 203 counts the number of common candidate characters common to the two candidate character groups X and Z. If the number of common candidate characters is greater than the threshold, the word recognition unit 203 determines that the detection result is correct. On the other hand, when the number of common candidate characters is smaller than the threshold, the word recognition unit 203 determines that the detection result is not correct.

  (S124) If it is determined in the process of S123 that the detection result is correct, the process proceeds to S122. On the other hand, if it is determined in S123 that the detection result is not correct, the process proceeds to S119 (see FIG. 14).

(Verification process flow)
Here, with reference to FIG. 16, the flow of processing relating to verification of detection results will be further described. The process illustrated in FIG. 16 corresponds to the process of S123.

  (S131) The word recognition unit 203 acquires a candidate character group corresponding to the detection result. For example, when a word described in the word database 201b is detected from the candidate character group X generated at the time of character recognition, the word recognition unit 203 acquires a candidate character group Y corresponding to the detected word from the word database 201b. When the error character string described in the error character string database 201d is detected from the candidate character group X, the word recognition unit 203 extracts the candidate character group Z corresponding to the detected error character string from the error character string database 201d. get.

  (S132) The word recognition unit 203 compares the candidate character group acquired in S131 with the candidate character group generated at the time of character recognition. Then, the word recognizing unit 203 counts the number of common candidate characters common to the two candidate character groups to be compared.

  (S133) The word recognition unit 203 determines whether or not the counted number of common candidate characters is greater than a threshold value. If the number of common candidate characters is greater than the threshold, the process proceeds to S134. On the other hand, if the number of common candidate characters is not greater than the threshold, the process proceeds to S135. The threshold value is a preset value, and is set according to, for example, the number of candidate characters to be considered.

(S134) The word recognition unit 203 determines that the detection result is correct. When the process of S134 is completed, the series of processes illustrated in FIG.
(S135) The word recognition unit 203 determines that the detection result is not correct. When the process of S135 is completed, the series of processes illustrated in FIG.

(Modification: Verification method considering cost)
Here, the flow of processing according to the verification method in consideration of the cost will be described with reference to FIGS. FIG. 17 is a first diagram for describing a verification method according to a modification of the second embodiment. FIG. 18 is a second diagram for explaining a verification method according to a modification of the second embodiment. FIG. 19 is a third diagram for describing a verification method according to a modification of the second embodiment.

(About calculation of evaluation value P)
So far, a method has been described in which the number of common candidate characters is compared with a threshold value, and the correctness of the detection result is determined according to the comparison result. Here, a verification method considering cost is proposed. As shown in FIG. 17, this modification introduces a method of paying attention to the cost of a common candidate character and performing verification using an evaluation value P based on the cost of the common candidate character.

  The evaluation value P is defined by the following formula (1). The evaluation value P is a value in which a weight is considered in the average value of the distances per common candidate character calculated for each word (or error character string). In this, M is the number of characters included in the word (or error character string). N is the number of candidate characters corresponding to one character of a word (or error character string). C is a weight value. The value of C is a value determined in advance based on, for example, experimental results.

D _ij and d _ij are costs corresponding to the j-th common candidate character among the common candidate characters corresponding to the i-th character included in the word (or error character string). However, D _ij is the cost described in the word database 201b (or error character string database 201d). D _ij is a cost obtained as a result of character recognition. n _i is the number of common candidate characters corresponding to the i-th character included in the word (or error character string).

図１７の例では、誤り文字列データベース２０１ｄから得た候補文字毎のコストと、文字認識結果のコストとに基づいて計算した評価値Ｐは１１１となる。一方、単語データベース２０１ｂから得た候補文字毎のコストと、文字認識結果のコストとに基づいて計算した評価値Ｐは７７８となる。

In the example of FIG. 17, the evaluation value P calculated based on the cost for each candidate character obtained from the error character string database 201d and the cost of the character recognition result is 111. On the other hand, the evaluation value P calculated based on the cost for each candidate character obtained from the word database 201b and the cost of the character recognition result is 778.

  The word recognition unit 203 determines whether or not the evaluation value P is smaller than a threshold value T (T = 500 in the example of FIG. 17). In the example of FIG. 17, the evaluation value P of the detection result based on the error character string is smaller than the threshold T, and the evaluation value P of the detection result based on the word is larger than the threshold T. In this case, the word recognizing unit 203 sets the word “Fujien” corresponding to the error character string “snow country” as the word recognition result.

(About the flow of verification processing)
Here, with reference to FIGS. 18 and 19, the flow of processing related to verification of detection results will be further described. Note that the processing illustrated in FIGS. 18 and 19 corresponds to the processing of S123.

  For convenience of explanation, an explanation will be given by taking as an example a case where an error character string described in the error character string database 201d is detected from the candidate character group X generated at the time of character recognition and the detection result is verified. However, the same applies to the case where a word described in the word database 201b is detected from the candidate character group X generated at the time of character recognition and the detection result is verified. In this case, the error character string database 201d may be replaced with the word database 201b.

(S141) The word recognition unit 203 acquires a candidate character group (candidate character group Z) corresponding to the detection result from the error character string database 201d.
(S142) The word recognition unit 203 initializes the index K to 1.

(S143) The word recognition unit 203 extracts, from the candidate character group X, a character group (hereinafter, candidate A) of common candidate characters corresponding to the K character of the word (or error character string).
(S144) The word recognition unit 203 selects a character group (hereinafter, candidate B) of common candidate characters corresponding to the K character of the word (or error character string) from the candidate character group Z acquired in the process of S141. .

  (S145) The word recognition unit 203 selects one candidate character from the candidate A. At this time, the word recognizing unit 203 selects an unselected candidate character in the series of processes shown in FIGS.

  (S146) The word recognition unit 203 determines whether the candidate character selected in the process of S145 exists in the candidate B. If the candidate character selected in the process of S145 exists in candidate B, the process proceeds to S147. On the other hand, if the candidate character selected in the process of S145 does not exist in candidate B, the process proceeds to S149.

  (S147) The word recognizing unit 203 calculates a cost difference for candidate characters that exist in common with the candidates A and B. First, the word recognition unit 203 acquires the cost corresponding to the candidate character of candidate A selected in the process of S145 from the error character string database 201d. Next, the word recognizing unit 203 calculates the absolute value of the difference between the cost calculated at the time of character recognition for the candidate character of candidate B corresponding to the candidate character of candidate A and the cost acquired from the error character string database 201d.

  (S148) The word recognition unit 203 adds the absolute value of the cost difference calculated in the process of S147 to the value Q (the initial value of Q is 0). When the process of S148 is completed, the process proceeds to S150.

(S149) The word recognition unit 203 adds the constant C to the value Q. The constant C is the same as the value C included in the above equation (1). When the process of S149 is completed, the process proceeds to S150.
(S150) The word recognition unit 203 determines whether or not all candidate characters included in the candidate A are selected in the process of S145. If all candidate characters included in candidate A have been selected, the process proceeds to S151 (see FIG. 19). On the other hand, if there is an unselected candidate character included in candidate A, the process proceeds to S145.

  (S151) The word recognition unit 203 determines whether or not the index K is smaller than the number M of characters in the error character string. When the index K is smaller than the number M of characters in the error character string (when K <M), the process proceeds to S152. On the other hand, when the index K is not smaller than the number M of characters in the error character string (when K = M), the process proceeds to S153.

(S152) The word recognition unit 203 increments the index K by one. When the process of S152 is completed, the process proceeds to S143 (see FIG. 18).
(S153) The word recognition unit 203 calculates a value Q (evaluation value P) per character. That is, the word recognition unit 203 sets the average value obtained by dividing the value Q by the number of characters as the evaluation value P.

  (S154) The word recognition unit 203 determines whether or not the evaluation value P is smaller than the threshold value T. When the evaluation value P is smaller than the threshold value T (P <T), the process proceeds to S155. On the other hand, when the evaluation value P is not smaller than the threshold value T (P ≧ T), the process proceeds to S156.

(S155) The word recognition unit 203 determines that the detection result is correct. When the process of S155 is completed, the series of processes shown in FIGS. 18 and 19 ends.
(S156) The word recognition unit 203 determines that the detection result is not correct. When the process of S156 is completed, the series of processes shown in FIGS. 18 and 19 ends.

The flow of word recognition processing executed by the server device 200 has been described above.
[2-6. About update processing]
Next, the update process of the error character string database 201d will be further described with reference to FIGS. FIG. 20 is a first diagram showing a flow of update processing of the error character database according to the second embodiment. FIG. 21 is a second diagram showing a flow of update processing of the error character database according to the second embodiment. FIG. 22 is a third diagram showing a flow of update processing of the error character database according to the second embodiment.

(Overall processing flow)
First, the overall processing flow will be described with reference to FIG. Note that the processing shown below is mainly executed by the database update unit 205.

  (S161) The database update unit 205 determines whether or not the word recognition unit 203 has detected a word described in the word database 201b from the candidate character group generated during character recognition. When a word is detected, a series of processes shown in FIG. On the other hand, if no word is detected, the process proceeds to S162. For example, when an error character string is detected from the candidate character group and a word corresponding to the error character string is set as the word recognition result, the process proceeds to S162.

  (S162) The database update unit 205 determines whether or not the word recognition unit 203 has detected an error character string described in the error character string database 201d from the candidate character group generated during character recognition. If an error character string is detected, the process proceeds to S163. On the other hand, if an error character string has not been detected, the process proceeds to S164.

  (S163) The database update unit 205 updates the error character string database 201d based on the candidate character group generated during character recognition. In this case, the error character string generated from the candidate character group already exists in the error character string database 201d. Therefore, for example, the database update unit 205 updates the cost described in the error character string database 201d regarding the common candidate character by using the cost of the candidate character calculated at the time of character recognition. When the process of S163 is completed, the series of processes illustrated in FIG.

  (S164) The database update unit 205 determines whether or not a correct word has been notified for the photo PT that is the target of word recognition. If the correct word is notified, the process proceeds to S165. For example, when a correct word is notified from the terminal device 100 or when a correct word is input to the server device 200, the process proceeds to S165. On the other hand, when the correct word is not notified, the series of processes shown in FIG. 20 ends.

  (S165) The database update unit 205 generates an error character string based on the candidate character group generated at the time of character recognition, associates the generated error character string with the candidate character group, and adds them to the error character string database 201d. . When the process of S165 is completed, the series of processes illustrated in FIG.

(About update processing)
Here, the update process will be further described with reference to FIG. The process illustrated in FIG. 21 corresponds to the process of S163.

  (S171) The database updating unit 205 calculates a weighted average value of the cost between corresponding candidate characters for the candidate character group X generated at the time of character recognition and the candidate character group Z described in the error character string database 201d. To do. However, the candidate character group Z is a candidate character group corresponding to an error character string detected from the candidate character group X. A candidate character common to the two candidate character groups X and Z is called a common candidate character.

Of the candidate characters in the candidate character group X corresponding to the i-th character included in the error character string, the cost corresponding to the j-th common candidate character is denoted as c _ij . Also, the cost corresponding to the j-th common candidate character among the candidate characters of the candidate character group Z corresponding to the i-th character included in the error character string is denoted as C _ij . Also, weights w ₁ and w ₂ are defined. The weights w ₁ and w ₂ are set, for example, as “w ₁ = 0.75”, “w ₂ = 0.25”, and the like. However, “w ₁ + w ₂ = 1”. The database update unit 205 calculates the updated cost Cr _ij for all possible i and j by the following equation (2).

Cr _ij = w ₁ · C _ij + w ₂ · c _ij
... (2)
(S172) a database updating unit 205 replaces the costs according to the error string database 201d, the weighted average value calculated in the processing of S171 (the cost of the updated) Cr _ij. That is, the database update unit 205 sets the cost C _ij corresponding to the j-th common candidate character among the candidate characters of the candidate character group Z corresponding to the i-th character included in the error character string as the weighted average value Cr _ij . replace. Note that the database update unit 205 executes cost C _ij replacement processing for all possible i and j.

  (S173) The database update unit 205 re-sorts the records in the error character string database 201d based on the cost after replacement. That is, the candidate character candidate rank is updated. When the highest candidate character is changed, the database update unit 205 generates an error character string in which the highest candidate character is combined. Then, the database update unit 205 updates the error character string described in the error character string database 201d with the generated error character string. When the process of S173 is completed, the series of processes illustrated in FIG.

(About additional processing)
Here, the additional processing will be further described with reference to FIG. The process illustrated in FIG. 22 corresponds to the process of S165.

(S181) The database update unit 205 generates an error character string by combining candidate characters positioned at the top of the candidate character group X generated during character recognition.
(S182) The database update unit 205 records a record in which the error character string and the candidate character group are associated with the correct word in the error character string database 201d.

  (S183) The database update unit 205 increments the number of error character strings corresponding to the correct word by one. For example, in the case of the error character string database 201d illustrated in FIG. It has become. When a new error character string “Wage” is added to the word “Fujien”, “Wage” is added to the error character string column, the corresponding candidate character group is added, and the number of error characters is reduced. Updated to 3. When the process of S183 is completed, the series of processes illustrated in FIG.

The update process of the error character string database 201d has been described above.
The second embodiment has been described above.
As described above, the word recognition method according to the second embodiment includes the verification process using the candidate character group. This verification process uses the property that even if the correct word is not detected from any image as a result of word recognition, a set of candidate characters having the same tendency can be obtained if the image includes the same word. Yes. In the case of a system that performs word recognition using index values such as a recognition score and a recognition cost, a candidate character group is obtained in the process of word recognition. For this reason, the already obtained candidate character group may be used for the verification, and the increase in the load accompanying the verification process is small.

  In the second embodiment, a method for verifying a detected word with a candidate character group and a method for verifying a word detected based on an error character string with a candidate character group have been proposed. By using an error character string, it is possible to detect words that are likely to fail in recognition, such as words written in design characters. On the other hand, when applying word recognition using an error character string, there is a high risk that an incorrect word will be detected. However, if the method of the second embodiment is applied, the recognition accuracy can be improved by reducing erroneous detection of words, and the practicality of word recognition using an error character string is improved.

  Thus, according to the second embodiment, it is possible to improve word recognition accuracy while suppressing an increase in load. As a result, when the user takes a picture of a signboard or poster on which a word including special font characters or design characters is written, registration information A based on the word automatically recognized from the photograph is provided to the user. Can be provided. Of course, the technology according to the second embodiment can also be applied to a system that recognizes words from handwritten documents or printed materials, and even when applied to such a system, an effect of improving the recognition rate can be expected.

  As mentioned above, although preferred embodiment was described referring an accompanying drawing, this invention is not limited to the example which concerns. It is obvious for those skilled in the art that various variations and modifications can be conceived within the scope of the claims, and such variations and modifications are naturally also included in the technical scope of the present disclosure. It goes without saying that it belongs to a range.

DESCRIPTION OF SYMBOLS 10 Information processing apparatus 11 Memory | storage part 12 Calculation part CG Character group G1 1st set G2 2nd set L1 1st character string P1 1st image P2 2nd image

Claims (8)

The plurality of first character strings expressed by the characters of the set character group and the shape of the character part cut out from the first image including the first character string for each of the plurality of first character strings A storage unit that stores a first set of characters of a character group in association with each other;
Generating a second set of characters of the character group similar to the shape of the character portion cut out from the second image, selecting the first character string based on the second set, and selecting the selected first character An arithmetic unit that calculates an evaluation value representing a difference between the first set and the second set corresponding to a column, and uses the selected first character string as a recognition result when the evaluation value is smaller than a set threshold value When,
An information processing apparatus. The information processing apparatus according to claim 1, wherein the arithmetic unit uses an index value that increases as the number of the characters included in common in the first set and the second set decreases as the evaluation value. . The storage unit stores the character included in the first set, and a first index value representing the similarity between the character and the shape of the character part cut out from the first image,
The computing unit calculates a second index value representing the similarity between the character included in the second set and the shape of the character portion cut out from the second image, and the first set, the second set, The information processing apparatus according to claim 1, wherein a difference between the first index value and the second index value is calculated for the characters included in common, and the magnitude of the difference is used as the evaluation value. The first set includes a predetermined number of the characters extracted from the character group in descending order of similarity with the shape of the character portion cut out from the first image,
The second set includes the predetermined number of characters extracted from the character group in descending order of similarity with the shape of a character portion cut out from the second image. The information processing apparatus according to any one of the above. The storage unit includes a second combination of the characters having the maximum similarity among the characters included in the first set with respect to the shape of the character portion corresponding to each character of the first character string. A character string and the first character string corresponding to the first set are stored in association with each other,
The computing unit is
When selecting the first character string based on the second set, if the first character string is obtained by combining the characters included in the second set, select the first character string,
The first character string corresponding to the second character string is selected when the first character string is not obtained by combining the characters included in the second set and the second character string is obtained. Item 5. The information processing apparatus according to Item 4. The arithmetic unit, for the second image including the first character string, when neither the first character string nor the second character string is obtained by combining the characters included in the second set, The information processing apparatus according to claim 5, wherein the character of the second set is added to the first set corresponding to the first character string. A computer that can access the storage unit
For each of the plurality of character strings expressed by the characters of the set character group and the plurality of character strings, the characters of the character group similar to the shape of the character part cut out from the first image including the character string are collected. With reference to the information in the storage unit stored in association with the first set,
Generate a second set of characters of the character group similar to the shape of the character portion cut out from the second image, select the character string based on the second set, and correspond to the selected character string A character recognition method for calculating an evaluation value representing a difference between the first set and the second set, and using the selected character string as a recognition result when the evaluation value is smaller than a set threshold value. To a computer that can access the storage unit,
For each of the plurality of character strings expressed by the characters of the set character group and the plurality of character strings, the characters of the character group similar to the shape of the character part cut out from the first image including the character string are collected. With reference to the information in the storage unit stored in association with the first set,
Generate a second set of characters of the character group similar to the shape of the character portion cut out from the second image, select the character string based on the second set, and correspond to the selected character string A program for calculating an evaluation value representing a difference between the first set and the second set, and executing a process of using the selected character string as a recognition result when the evaluation value is smaller than a set threshold value.

Images