JP4802502B2 - Word recognition device and word recognition method - Google Patents

Description

  The present invention belongs to a technical field related to word input means using character string recognition.

  Conventionally, an apparatus that reads characters printed or handwritten on paper is known as OCR. The main application fields are form processing, mail classification, and text conversion of documents. In typical OCR, characters are read in the following procedure. First, the paper is photoelectrically converted into a computer using a scanner (image input), the area to be read is estimated, individual characters are cut out (character cutout), and the individual characters are identified (characters). The character group read using (identification) and language information is interpreted as a character string (post-processing). When recognizing Japanese by such OCR, a large amount of storage capacity is required especially for the means (language dictionary) for storing language information. In addition, there are a plurality of character lines on the page, and a part to be read may be a part thereof. In such a case, the apparatus automatically determines a character line to be read in accordance with a rule predetermined according to the application field.

In general, it is difficult to specify which partial image corresponds to a correct character at the stage of character extraction. For this reason, a method of cutting out characters based on various hypotheses and specifying how to cut out characters by post-processing is widely used.
In addition, when there are characters with similar shapes, it may be difficult to specify the character type by the character identification process alone. In such a case, character identification outputs a plurality of candidate characters.

  In recent years, attempts have been made to read characters such as documents, signboards, signs, and the like using a camera mounted on a portable device such as a mobile phone or a PDA (personal digital assistant) as an image input means. The recognition targets on these devices are phone numbers, e-mail addresses, URLs, English words, and the like. The recognition results are used for services such as telephone and e-mail transmission, web access, and word translation. In such an application, it is assumed that the user can freely read documents, signboards, signs, etc. around him / her with a portable device and receive a service. For this reason, it is necessary to realize ease of operation, short waiting time, and the like.

  In the case of the recognition target of the conventional technology, it is relatively easy to specify the character string to be read from the image. For example, in the case of a telephone number, a character string such as “Tel.” Is usually written immediately before, and the number of digits, parentheses, and hyphens are regular. Also, email addresses and URLs have regularity such as “http:” at the beginning, “@” in the middle, and “.com”, “.jp”, etc. The conventional technique can automatically detect the character string to be recognized using such regularity. In the case of English words, there are spaces before and after the word. For this reason, it is easy to specify a word to be recognized based on a rough position specification. For example, in Masahiro Yamazaki et al., "Development of an electronic dictionary function for mobile phones using the OCR function" (Proceedings of the 2004 IEICE General Conference Proceedings D-12-35) (Non-patent Document 1) An application example is described in which an English word to be read is matched with a mark at the center of the screen, and an English word in the vicinity is read and a translation result of the word is displayed.

  However, when recognizing a word from a character string written in a language such as Japanese or Chinese without a space between words, it is difficult to specify the character string to be read. This is because Japanese prints and writes without spaces between words. For example, even if the operator aligns the mark with the center of “repair” to read the word “repair” in the text “temporary repair cost claim”, it automatically determines which range is expected by the operator. It is difficult to specify it. As an alternative, there is a method of designating a reading area with a rectangle, but this significantly increases the amount of operation and reduces the convenience of the device.

  Further, in a portable device having such a character recognition function, a problem of dictionary storage capacity in post-processing occurs. In the conventional method, it is common to detect a probable word while there is ambiguity in the result of character segmentation or character identification while applying restrictions using information in the word dictionary. When reading general Japanese words, current affairs words, etc., the number of words becomes enormous and it is difficult to store them in a portable device. As a solution to this problem, it is conceivable to store the dictionary in an external computer such as a server and connect it to a portable device with a communication function. However, in such post-processing, it is necessary to frequently access the word dictionary, and if the dictionary is placed outside, there is a problem that the processing time becomes long.

Japanese Patent Laid-Open No. 11-085909

Masahiro Yamazaki et al., "Development of an electronic dictionary function for mobile phones using the OCR function" Proceedings of the 2004 IEICE General Conference D-12-35 H. Bunke, P.S.P.Wang, “Handbook of Character Recognition and Document Image Analysis,” World Scientific, 1997 Katsumi Marukawa et al. "Error Correction Algorithm for Handwritten Kanji Address Recognition", IPSJ Transactions, Vol. 35, No. 6, 1994-6, pp. 1101-1110

The first problem to be solved by the present invention is to make it possible to specify a word to be read by a simple operation from a document in a language such as Japanese or Chinese that is described without a space between words. That is. As described above, since there is no space between words in Japanese, it is difficult to automatically specify a word range even if only one position is specified. In the present invention, this problem is solved and a word to be read can be designated by an operation equivalent to recognizing an English word or the like.
The second problem to be solved by the present invention is to reduce the frequency of access to the word dictionary in post-processing, and to enable word reading in a practical processing time even if the word dictionary is on the server. is there.

  As a first means for solving the above-described problem, the present invention provides means for selecting the closest one to the position information designated by the operator from a set of candidate words obtained as a result of word matching. Here, word collation is a process of detecting an arrangement of partial images that are likely to be words stored in a dictionary in advance based on a character identification result. The dictionary stores one or more words in advance. If a plurality of partial image sequences that are likely to be words are found, these are output as candidate words. As a measure of the degree of proximity between the designated position and the candidate word, for example, the distance between the center of gravity of the circumscribed rectangle of the candidate word and the designated position is used. As a result, even when there is no space between words, it is possible to read a word near the position designated by the operator.

  As a second means for solving the above problem, the present invention provides a character string output means for outputting a plausible character string candidate without using word information after character identification. The character string output means outputs a plausible character string based on information such as the certainty factor obtained as a result of character identification and the uniformity of the interval between partial images. When there are a plurality of likely character strings, the plurality of character strings are output as candidate character strings.

  The character string output means adopts a method for optimizing likelihood by iterative processing. Conventionally, a method is widely used in which the positional relationship of partial images is represented by a network (character segmentation network), the confidence level of each partial image as a character is obtained, and the route that maximizes the sum of confidence levels on the network is obtained. It has been broken. However, this method cannot optimize the uniformity of the interval between the partial images. Therefore, the character cutout method is repeatedly changed little by little to optimize the likelihood as a character string.

  By using the character segmentation method, word collation method, and data format as described above, the operator can specify from character strings in languages such as Japanese and Chinese that are described without spaces between words. It is possible to automatically cut out a word close to the position to be performed. Thereby, the operation amount of the operator for recognizing the word is greatly reduced, and the convenience of the device is improved.

  Even if the word dictionary is in a remote server, it is not necessary to perform frequent network access, and the processing speed is improved. In the present invention, character strings can be collectively transferred to a server, and the transfer time is shortened. The character string to be transferred is carefully selected according to the certainty of character identification, the dispersion of character intervals, and the like, and the transfer time is shortened. Furthermore, the ambiguity of character segmentation has been resolved at this point, and the word matching process at the server can be simplified.

  FIG. 1 shows an embodiment of the present invention. This embodiment is realized by two computers 100 and 101. The image input means 102 photoelectrically converts a character image into a computer. The position designation unit 103 identifies the designation of the position of the word to be read input by the operator. Here, the position is designated by an X coordinate value and a Y coordinate value on the image. The character cutout unit 104 cuts out partial images that are considered to correspond to individual characters. The character identifying means 105 identifies what character each cut out partial image is and outputs it together with the certainty factor. At this time, a means (character identification dictionary 109) for storing the shape of each character is referred to. The character string output means 106 outputs a plausible character string based on information such as the certainty factor obtained as a result of character identification and the uniformity of the interval between partial images. When there are a plurality of likely character strings, the plurality of character strings are output as candidate character strings. The word collating means 107 collates the candidate character string with words stored in advance in the word dictionary 110, and detects matching words. The word selection means 108 selects a word close to the designated position based on the output of the word collating means 107 and the output of the position designation means 103, and outputs it as a word recognition result. Finally, the recognition result display unit 111 displays the word recognition result.

  The computer 1 is a portable information terminal such as a camera-equipped mobile phone or a camera-equipped PDA. The computer 2 is a computer that can communicate directly or indirectly with the computer 1 wirelessly or by wire, and is, for example, a server connected to a cellular communication network. The recognition result display unit 112 is a display unit included in the computer 1. The image input means 111 of the computer 1 is realized by an image input device such as a camera. The position specifying unit 103, the character cutout unit 104, the character identification unit 105, and the character string output unit 106 are realized by executing a program stored in the storage unit of the computer 1 by the calculation unit. The character identification dictionary 109 is stored in the storage unit of the computer 1. The word collating unit 107 and the word selecting unit 108 are realized by executing a program stored in the storage unit of the computer 2 by the calculation unit. The word dictionary 110 is stored in the storage unit of the computer 2. The computers 1 and 2 have a communication function, and use this communication function to transmit and receive word position designation, character string output means output, word recognition results, and the like.

  An example of the external appearance (front side and back side) of the computer 1 (100) is shown in FIG. If the camera of the image input means 102 is installed on the side opposite to the display unit 111, it is convenient when the user inputs an image while viewing the image. On the display unit 111 side, an input button 112 used to operate display contents on the display unit and specify image input is provided.

  FIG. 2 schematically shows an input image and position specifying operation. A window 201 displays an input image on the display unit 111. It is assumed that words that the operator wants to read are taken in the input image. Reference numeral 202 denotes a mark for specifying a position. When the operator inputs an image in accordance with a word for which the mark is to be recognized, the position of the input image corresponding to the position of the mark is designated as a position to be specified by the position specifying means 103. In this example, in order to read the character string “Economy”, an image is input with a mark in the vicinity thereof.

  As shown in 202, when there are many characters that are divided in the input image, it is difficult to uniquely determine the boundary between the characters. In such a case, characters are cut out at this stage based on various hypotheses. FIG. 3 schematically illustrates an example of the output of the character cutout unit 103. Here, the character cutout result is in a network format as disclosed in Japanese Patent Laid-Open No. 11-085909 (Patent Document 1). The vertices of the network represented by circles in the figure represent candidates for boundaries between characters. The number in the circle represents the identifier of each boundary candidate. A broken line represents a cut out partial image. With such network representation, the character cut-out method is represented by a route in the network.

  As the character identification means 105, for example, a technique as described in H. Bunke, P.S.P. Wang, “Handbook of Character Recognition and Document Image Analysis,” (World Scientific, 1997) (Non-patent Document 2) is used. If there are characters with similar shapes, it may be difficult to specify the character type by the character identification process alone. In such a case, the character identification means 105 outputs a plurality of candidate characters together with the certainty factor.

  FIG. 4 schematically shows an output example of the character string output means 106. Since how to cut out characters cannot be determined at this stage, character strings are output assuming various ways of cutting out characters. Although six character strings are shown in the figure, this indicates a case where six candidate character strings are output. In addition, the candidate character strings are arranged in descending order of the character string certainty values shown below.

(Character string certainty factor) = a x (Average value of the first certainty factor of the character identification result)-b x (Distribution value of the center coordinate interval of the character)
(A and b are positive constants)
This is intended to make a character candidate that is as plausible as possible and has a good character pitch as a character string as a top candidate.

  FIG. 5 shows an example of a processing procedure in the character string output means 107. First, in step 501, a route on which the total sum of character identification certainty levels is maximized is searched on the network. This can be realized by a normal route search algorithm such as the Dijkstra algorithm. Next, in step 502, the character string certainty factor is calculated according to the route obtained in step 501 and substituted into variables a and b.

  Next, in the loop 503, the following processing is repeated. First, in a loop 504, for all boundary candidates, the character string certainty factor obtained by reversing the boundary candidates is calculated, and the value is substituted into the variable c. If the value of variable c is larger than the value of variable b, the value of c is substituted for b.

  In the processing of the loop 503, the reversal of the boundary candidate indicates the following processing. If the boundary candidate i is included in the route, a method of cutting out characters having the boundary candidates immediately before and after i as both ends is selected, and the route is corrected so that i is not included. If the boundary candidate i is not included in the route, the route is corrected to include i. In the example of FIG. 6B, the boundary candidate No. 3 in FIG. 6A is reversed, and in FIG. 6C, the boundary candidate No. 5 is reversed.

  In step 505, it is determined whether the value of the variable a is less than the value of b. If the determination result is true, the value of b is substituted into a. If false, the loop 503 is terminated, and the arrangement of partial images corresponding to the character string is determined according to the path at that time, and is output as a character string. The above processing is an example of outputting only one optimum character cutout method. Similarly to the above processing, it is possible to always output the top n character paths, repeat them, and modify them little by little to output the top n optimal character cutout methods.

As the output of the character string output means, a combination of the first candidate characters of the character identification result for each partial image of the obtained partial image array is used. As another example, a plurality of candidate characters (lattices) stored for each partial image as described later may be used.
The word matching means 107 uses a normal character string comparison method. Also, when using lattice as input, Katsumi Marukawa et al. “Error Correction Algorithm for Handwritten Kanji Address Recognition” (IPSJ Journal, Vol. 35, No. 6, 1994-6, pp. 1101-1110) ) (Non-Patent Document 3) is used.

  FIG. 7 schematically shows the result of displaying the output of the word selection means 108 by the recognition result display means 111. Reference numeral 201 denotes a screen used for position designation. A window 701 displays the word recognition result. The upper part of the window displays more likely words. For the word likelihood, the distance between the center of gravity of the recognized word on the circumscribed rectangle image and the reading position designated by the operator is used. Moreover, you may make it display the word candidate which includes the reading position which the circumscribed rectangle designated. Further, the recognition result display unit 111 displays a cursor 702 so that the operator can specify a desired word candidate. The operator operates buttons and moves the cursor up and down to select a desired word from the listed candidate words. The X coordinate of the character in the window 701 is displayed in accordance with the X coordinate of the corresponding character in the input image shown in the window 201.

  FIG. 8 shows a data format when the output format of the character string output means 106 is a lattice. Each row of the table, that is, one record corresponds to one character in the character string. The first two variables BL and BR store the identifiers of the left and right boundaries on the network. The following four variables L, T, R, and B store the coordinates of the left end, right end, upper end, and lower end of the cut out partial image. The next variable N stores the number of candidate characters to be output. In the arrays C [1] to C [N], character codes of candidate characters obtained as a result of character identification are stored. In arrays Lk [1] to Lk [N], the certainty factor of each candidate character is stored. As described above, by storing the character identification result together with the coordinates of the partial image, the word selection unit 108 can select a candidate word corresponding to the position designation result.

  FIG. 9 shows an output data format of the word collation 107. The first variable LEN stores the number of characters of the word. In the next four variables L, T, R, and B, the coordinates of the left end, right end, upper end, and lower end of the word are stored. The variable C [i] stores the character code of the i-th character of the word. A variable P [i] indicates a pointer to the record in the table of FIG. 8 corresponding to the i-th character of the word. By using such a storage format, a display as shown in FIG. 7 becomes possible.

1 shows a configuration of an embodiment of the present invention. Position specification screen. A network of character segmentation results. String output result. Processing procedure for character string output. An example of boundary reversal. Display example of word selection results. Data format for character string output. Data format for word matching results. 2 shows a configuration example of the computer 1.

Explanation of symbols

  100: first computer, 101: second computer, 102: image input means, 103: position designation, 104: character extraction, 105: character identification, 106: character string output, 107: word matching, 108: word selection 109: character identification dictionary, 110: word dictionary, 111: recognition result display, 201: input image display window, 202: cursor for position designation, 701: word recognition result display window, 702: word selection cursor.

Claims (2)

Image input means for photoelectrically converting an image into a digital image, character cutout means for cutting out a sequence of character candidate partial images based on boundary candidates between characters in the digital image, and each partial image obtained by the character cutout means Character identification means for identifying each character as a character and outputting a pair of the identification result and each character identification certainty factor, and a route search algorithm, and using the path search algorithm, the sum of the certainty factors for each character identification A character string including a variance value of the center interval of the character string obtained as a result of selecting a route with the largest value , further correcting a route with one boundary candidate i as a boundary, and reversing or not Repeatedly search for a set of boundaries to obtain an optimal partial image arrangement by increasing or decreasing the certainty of the A word storage means for storing a match, a character string word matching means for detecting a partial character string matching the word stored in the word storage means from the character string generation result of the character string generation means, and a word to be read in the image A position specifying means for specifying the position by the position of the mark input in the image, and a word selecting means for selecting a character string word matching result that is close to the position specified by the position specifying means. Feature word recognition device. The character string generation means repeats searching for a set of boundaries for obtaining an optimum partial image arrangement by correcting a path for reversing or not as the boundary for all boundary candidates obtained from the digital image. The word recognition apparatus according to claim 1.

Images