JP2006235817A - Character-recognizing device, character-recognizing method and recording medium for character recognition program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve recognition precision in character recognition, concerning an image identification technology for identifying a character string by photographing the character string as an image. <P>SOLUTION: A character-recognizing device 12 which recognize characters in an image is provided with a first stage character-recognizing means 21 for segmenting a local image, and for extracting features, while scanning an inputted image, and for deciding character candidates, based on the similarity of the features and the features of characters registered in a dictionary; a deformation estimating means 22 for estimating the deformation degree of the character candidates, based on the character candidates and the image; a deformation correcting means 23 for segmenting the corrected local image from the image, based on the degree of deformation; and a character rerecognizing means 24 for segmenting the local image, and for extracting the features, while scanning the corrected local image, and for deciding new character candidates, based on the similarity of the features and the features of the characters registered in the dictionary and the character candidates. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image identification technique for capturing a character string as an image and identifying the character string. As a specific industrial application system, for example, a Japanese translation system for a signboard can be cited.

  Character string identification existing in the landscape generally passes through four steps of character string position specification, character region specification, binarization, and character identification. However, such a technique has a problem in that character string position specification, character region specification, and binarization are not successful due to the influence of illumination fluctuations, complicated background, and the like, and character string identification accuracy is low.

In order to solve such a problem, as a character string identification technique existing in a landscape, there is a method that passes through three steps of full screen search, character candidate narrowing down, and character string estimation using a language model (for example, Non-Patent Document 1). reference.).
Yoshinori Kusachi, Naomi Ito, Akira Suzuki, Kenichi Arakawa, “Recognition of Characters in a Landscape without Image Area for Image Indexing”, IEICE Technical Report PRMU 2004-89, (2004-10), p.37 -42

  However, in the above method, the definition of features and the identification algorithm are limited, and the identification accuracy is not sufficient. In particular, it has been found that the deformation of characters depending on the imaging direction greatly affects the identification accuracy. In addition, many character candidates remain in the background portion, and there is a problem that the character string estimation by the language model does not work well.

  This invention is made | formed in view of this situation, The objective is to provide the character recognition technique which solved the said subject.

  Accordingly, in order to solve the above-described problem, the invention according to claim 1 is a character recognition device that recognizes characters in an image, and extracts features by cutting out a local image while scanning the input image. First-stage character recognition means for determining a character candidate based on the similarity between the feature and the character feature registered in the dictionary, and deformation for estimating the deformation degree of the character candidate based on the character candidate and the image An estimation unit; a deformation correction unit that extracts a corrected local image from the image based on the degree of deformation; and a feature that is extracted by extracting the local image while scanning the corrected local image, and is registered in the feature and dictionary And a character re-recognizing means for determining a new character candidate based on the character candidate.

  The invention according to claim 2 is characterized in that the degree of deformation is a parameter of affine transformation.

  According to a third aspect of the present invention, in the character re-recognition means, a character candidate determined as having a high similarity by the first-stage character recognition means and the character re-recognition means is used as a recognition result. To do.

  According to a fourth aspect of the present invention, there is provided a character recognition method in a character recognition device for recognizing characters in an image, wherein the first stage character recognition means cuts out a local image while scanning the input image. A first-stage character recognition step for determining a character candidate based on the similarity between the feature and the feature of the character registered in the dictionary, and a deformation estimation unit, based on the character candidate and the image, A deformation estimation step for estimating the deformation degree of the character candidate, a deformation correction unit for cutting out the corrected local image from the image based on the deformation degree, and a character re-recognition unit for extracting the corrected local image A character re-recognition system that extracts a feature by cutting out a local image while scanning, and determines a new character candidate based on the similarity between the feature and the character feature registered in the dictionary, and the character candidate. Characterized in that it has Tsu and up, the.

  The invention according to claim 5 is characterized in that the degree of deformation is a parameter of affine transformation.

  Further, the invention according to claim 6 is characterized in that, in the character re-recognition step, character candidates determined to have high similarity in the first-stage character recognition step and the character re-recognition step are used as recognition results. To do.

  The invention described in claim 7 is characterized in that a program that records the character recognition device or the character recognition method described in any of claims 1 to 6 so as to be executable by a computer is recorded.

  In the first to seventh aspects of the invention, the accuracy of identification is improved by estimating the degree of deformation of the character candidate, correcting the deformation, and recalculating the similarity.

  In the inventions according to claims 3 and 6, the character candidates can be further narrowed down.

  According to the first to seventh aspects of the present invention, it is possible to improve recognition accuracy in character recognition.

  According to claims 3 and 6, it is possible to reduce the number of character candidates in the background portion by further narrowing down the character candidates.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, an example in which the character recognition device is applied to a character string translation system will be described with reference to FIG. The character string translation system includes a mobile terminal 11 such as a PDA with a camera, a character recognition device 12, a character string estimation device 13, and a translation device 14.

  In this character string translation system, the user takes an image with the mobile terminal 11 and sends it to the character recognition device 12. The character recognition device 12 extracts character candidates from the image and sends them to the character string estimation device 13. The character string estimation device 13 estimates a character string from the character candidates and sends it to the translation device 14. The translation device 14 translates the character string and sends the translation result to the portable terminal 11 so that the user can view the translation result. In this manner, the present character string translation system allows the user to see the translation result of the character string based on the captured character string image.

  Here, the configuration of the character recognition device 12 will be described with reference to FIG. As shown in FIG. 2, the character recognition device 12 includes a first stage character recognition unit 21, a deformation estimation unit 22, a deformation correction unit 23, and a character re-recognition unit.

  The first-stage character recognition means 21 is a means for extracting character candidates from an image. For example, “character recognition in a landscape that does not require a text area for the purpose of image indexing”, IEICE Technical Report PRMU 2004-89. (2004-10), p. 37-42.

  Here, the first stage character recognition means 21 will be described with an example. The first-stage character recognition means 21 generates a multi-resolution image in order to correspond to characters of different sizes, cuts out an image of a predetermined size while shifting the position, and performs a coarse / fine search. This result is used as an index. In the image search, when a keyword is input, only the corresponding character is extracted from the index to determine regularity, and an image determined to be regular is output as a result.

  Here, the coarse / fine search includes pattern learning and pattern identification. Hereinafter, pattern learning and pattern identification will be described.

(Pattern learning)
Pattern learning is composed of four stages: feature extraction, creation of a category hierarchical structure, pattern generation by geometric deformation, and dictionary generation.

In the feature extraction, an original pattern (w × w) obtained by photographing a character from the front is prepared, and the feature is extracted. The feature utilizes a weighted direction index histogram feature (WDCH). WDCH is used for OCR and is intended for binary images, but can be easily extended to gray images. The outline of the algorithm is shown below. However, M and N are positive constants.
1: A differential value and direction are obtained from the original pattern using a Sobel operator.
2: The direction of differentiation is quantized in the M direction.
3: The original pattern is divided into N × N grids.
4: The magnitude of the differential value is added in each M direction of each grid.
5: Consider a feature vector of N × N × M, and normalize the norm.

  Since WDCH is based on differential values, it is not easily affected by brightness fluctuations. Further, by using the sum of the differential values in the grid, it is possible to absorb minute variations in shape such as deformation due to fonts.

  In creating a category hierarchical structure, each category is clustered based on the similarity of feature vectors to create a hierarchical structure. Each node includes a plurality of categories. In the lowest layer node, only a single category is included.

  In the pattern generation by geometric deformation, a character deformation pattern accompanying a change in viewpoint is generated for each category. The original pattern is geometrically deformed by five-parameter affine transformation of rotation, vertical skew, horizontal skew, aspect ratio, and enlargement / reduction. Although the size of the generated pattern may be larger than the original pattern, features are extracted using only partial patterns within the window size of the current pattern, and this feature vector is used for dictionary generation.

  In dictionary generation, a dictionary for each node is created in the following procedure.

  As a first stage, feature compression is performed. First, the features are compressed in each layer. All feature vectors including the geometric deformation pattern are subjected to principal component analysis and compressed using eigenvectors having higher eigenvalues. This compressed feature vector is represented as f (c, r, p). However, c is a category, r is a compression rate, and p is a deformation parameter.

  As a second stage, dictionary generation at each node is performed. If the category set of each node is C, the principal component analysis is performed on the vector of f (C, r, p), and the subspace Ed (C, r) is obtained. Here, d is the number of dimensions of the subspace, and is an integer determined by the system, which is determined by the contribution rate.

  The compression rate of each layer is set to be lower toward the lower layer, thereby realizing the coarse / fine search. In the upper layer, although the accuracy is low, identification is performed at high speed, and in the lower layer, identification is performed at low speed but with high accuracy.

(Pattern identification)
While moving the position over the entire surface of the multi-resolution image, a small area of size W × W is cut out and pattern identification is performed. In pattern identification, a coarse / fine search is performed while following a plurality of routes in a hierarchical structure. The outline of the algorithm is shown below.
1. Feature extraction: Extracting features by cutting out regions while changing the position of each resolution image. Features of all cutout areas are calculated in advance.
2. Initialization: Start from the root node of the tree structure.
3. Candidate node setting: Nodes in the first layer are set as candidate nodes in all cutout areas. Repeat 4-6 for each cutout area.
4). Compression: compresses the features of the clip region using the compression ratio of the lower layer. This is represented as I ′ (r).
5. Projection distance calculation: The projection distance L (C) is obtained using the partial space of the candidate node C according to the following equation.

Where D is the subspace dimension.
6). Screening: The rank of each candidate node is calculated from the above distance values. The candidate nodes are updated from the distance and rank thresholds.
7). Peak detection: Three-dimensional (vertical, horizontal, resolution) spatial connectivity is calculated for each candidate node in all cutout regions, and a segment is obtained. Only the candidate nodes with the smallest peak of distance value within each segment are left. Other candidate nodes are deleted.
8). Screening in the local region: The following processing is performed for each set of peaks having the same resolution. First, the space is divided into blocks, and the peaks included in each block are sorted according to distance values, and only a certain number from the top is adopted. Thereafter, the same processing is performed by shifting the block division position by a half block horizontally / vertically.
9. Peak integration of the same candidate characters: If two peaks having the same candidate character are taken out and integrated with a smaller distance value if the center coordinate and the resolution are close to each other, the processing is repeated until there is no pair of peaks to be integrated.
10. Candidate node update: A node connected to a layer below the candidate node is registered as a new candidate node.
11. End determination: If the candidate node has reached the lowest layer, the remaining candidate node is output as an index and the process ends. Otherwise, the process returns to 4 above. The format of the index is (category name, position, size, similarity).

  The spatial connectivity of 7 is not limited to three dimensions, but two dimensions (vertical and horizontal) can be considered. In addition, the above 7 to 9 are processes for reducing the processing amount, which is a tradeoff between accuracy and processing amount. It is not necessary to carry out at all levels, and it is sufficient to carry out only at a predetermined level.

(Image search)
In an image search, when a keyword is input as a character string, the index obtained by pattern identification is searched for a place where the pattern is spatially regularly arranged, and an image having an index in which such a place exists is searched. Output as search results. As a rule of spatial arrangement of patterns, here
(1) The pattern size is almost the same
(2) The pitch is almost constant
(3) The pitch size is within a certain range relative to the size of each individual pattern.
(4) The pattern arrangement order matches the input character string order, and the difference in angle between the pattern arrangement direction and the horizontal or vertical direction is within a certain range.

  In this case, the search algorithm searches for all the locations in the index where any combination of two characters in the forward direction that make up the input character string exists, and at these points the start position of the virtual input character string is determined. A two-dimensional coordinate and a value of a two-dimensional vector representing character advance are calculated, and voting is performed in a voting space constituted by these parameters. However, at the time of voting, it is checked whether or not the combination does not violate the rules (1), (3), and (4) above. Finally, the voting space is searched for the presence or absence of a part whose score is equal to or greater than a threshold value.

  In this algorithm, the candidate characters to be processed at the time of voting are limited to only two categories, so it is possible to perform high-speed processing even for indexes that contain many false candidate characters, and partial due to the characteristics of voting processing. It is robust against lack of correct answers.

  In this way, the first stage character recognition means 21 determines character candidates (candidate category, position, size, similarity) from the image.

  The deformation estimation means 22 estimates the deformation degree of the character at each position / size from the character candidates (candidate category, position, size, similarity) and the image. This can be realized, for example, by a parameter estimation method described in “Posture estimation of a three-dimensional object by support vector regression”, IEICE Technical Report PRMU 2004-91 (2004-10).

  Specifically, the parameters are the camera yaw angle, pitch angle, and roll angle, but the character deformation parameters are affine transformation parameters (vertical expansion / contraction, horizontal expansion / contraction, vertical skew, horizontal skew, rotation). ) Etc. are considered good targets. In order to learn the parameter estimation function, several types of character fonts may be artificially deformed and used.

  The deformation parameter estimation function thus obtained is derived for each character category. Therefore, in parameter estimation, a deformation parameter value may be calculated by calling a deformation parameter estimation function corresponding to a “candidate category” that is a property of a character candidate (candidate category, position, size, similarity). At this time, the feature vector to be input to the deformation parameter estimation function is extracted from the local image specified by “position, size” that is a property of the character candidate (candidate category, position, size, similarity). . The feature amount at this time may use the same feature as that of the first-stage character recognition means 21, or may use a different feature, for example, a feature amount having each pixel constituting the image as an element.

  The deformation parameter can be estimated from each character candidate (candidate category, position, size, similarity) by the above method. Among these, those having the same position and size are calculated as an average value of these parameter values, and are combined into a single deformation parameter for the position and size. This result is output as the degree of deformation (position, size).

  The deformation correction unit 23 cuts out the corresponding local image after deformation correction from the input image based on the degree of deformation (position, size).

  Here, FIG. 3 shows how the local image after deformation correction is cut out from the input image. The local image after deformation correction is cut out so that the deformation of the character is corrected. For example, if the deformation parameter is an affine transformation, a local image after deformation correction can be obtained by cutting out the inverse transformation of the affine transformation on the image. The result is output as a corrected local image (position, size).

  The character re-recognition means 24 performs the same processing as the first-stage character recognition means 21 on the corrected local image (position and size), and calculates the similarity. This similarity is output as a new character candidate as a new similarity of the corresponding character candidate (candidate category, position, size, similarity) received from the first-stage character recognition means 21.

  At this time, in each position / size, the similarity rank of the new character candidate is included within the Nth rank, and the similarity rank of the recognition result in the first stage character recognition means 21 is included within the Mth rank. It is possible to output only as a recognition result. As a result, both the first-stage character recognition means 21 and the character re-recognition means 24 leave the character candidates having a high similarity ranking, and further narrowing down the character candidates can be performed. N and M are predetermined parameters, and the value of N and the value of M may be the same. May be different.

  In the above-described embodiment, the character recognition device 21 is realized by, for example, a CPU included in a computer device that constitutes the character recognition device 21, and the required first-stage character recognition processing, deformation estimation processing, deformation correction processing, and Character re-recognition processing can be installed as an application program.

  In addition, data such as processing results and calculation results obtained in the required first-stage character recognition processing, deformation estimation processing, deformation correction processing, character re-recognition processing, etc. can be written to and read from the internal memory or external storage device. You may do it.

  In addition, a recording medium recording software program codes for realizing the functions of the present embodiment is supplied to a system or apparatus, and a CPU (MPU) of the system or apparatus reads and executes the program code stored in the storage medium. It is also possible. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and examples of the storage medium storing the program code include CD-ROM, DVD-ROM, and CD-R. , CD-RW, MO, HDD and the like.

A character string translation system using a character recognition device. The block diagram of a character recognition apparatus. Explanatory drawing of a local image cutout after deformation | transformation correction.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Portable terminal 12 ... Character recognition apparatus 13 ... Character string estimation apparatus 14 ... Translation apparatus 21 ... First stage character recognition apparatus 22 ... Deformation estimation means 23 ... Deformation correction means 24 ... Character re-recognition means

Claims (7)

A character recognition device for recognizing characters in an image,
First stage character recognition means for extracting a feature by cutting out a local image while scanning an input image, and determining a character candidate based on the similarity between the feature and the feature of a character registered in the dictionary;
Deformation estimating means for estimating a deformation degree of the character candidate based on the character candidate and the image;
Deformation correction means for cutting out a corrected local image from the image based on the degree of deformation;
Character scanning is performed by extracting a feature by cutting out the local image while scanning the corrected local image, and determining a new character candidate based on the similarity between the feature and the character feature registered in the dictionary, and the character candidate. And a character recognition device.   The character recognition device according to claim 1, wherein the degree of deformation is a parameter of affine transformation.   3. The character recognition according to claim 1, wherein the character re-recognition means uses a character candidate determined as having a high degree of similarity in the first-stage character recognition means and the character re-recognition means as a recognition result. apparatus. A character recognition method in a character recognition device for recognizing characters in an image,
First stage character recognition means extracts a feature by cutting out a local image while scanning the input image, and determines a character candidate based on the similarity between the feature and the feature of the character registered in the dictionary A step recognition step;
A deformation estimating step for estimating a deformation degree of the character candidate based on the character candidate and the image;
A deformation correction step in which a deformation correction unit cuts out a corrected local image from the image based on the degree of deformation;
The character re-recognition means extracts a feature by cutting out the local image while scanning the corrected local image, and based on the similarity between the feature and the character feature registered in the dictionary, and the character candidate And a character re-recognition step for determining a candidate.   The character recognition method according to claim 4, wherein the degree of deformation is an affine transformation parameter.   The character recognition according to claim 4 or 5, wherein, in the character re-recognition step, a character candidate determined as having high similarity in the first-stage character recognition step and the character re-recognition step is used as a recognition result. Method.   A recording medium having recorded thereon a program that can execute the character recognition device or the character recognition method according to any one of claims 1 to 6 by a computer.