KR100874747B1 - Camera Character Recognition Apparatus and Method Using Pixel Shift Document Image Recognition - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for recognizing a camera character using a pixel shift document image combination recognition method, wherein a camera document image is divided into a plurality of sub-document images in which pixels are shifted, and each document image is input. After applying the image enhancement algorithm and the binarization to, extract the feature of the character by using the combined character feature extractor such as the pixel shift network feature, and then recognize it through the weighted heterogeneous recognizer to obtain the final recognition result. The results can be used in the form of document editing, database construction, and voice, so that various services can be provided in response to user needs.

Description

Apparatus and method for recognizing camera character using pixel shift document image recognition method TECHNICAL FIELD

The present invention relates to a technique of recognizing a document image acquired using a camera among image processing techniques. More particularly, the present invention relates to a method of acquiring sub-document images having pixels moved using a camera, and to applying image enhancement and local binarization. The present invention relates to a camera character recognition apparatus and method using a pixel shift document image combination recognition method which extracts a feature of an individual character, extracts a feature of a pixel shifted individual character, and recognizes and combines the result using a weighted heterogeneous recognizer.

The present invention is derived from a study conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication and the Ministry of Information and Telecommunication Research and Development. [Task Management Number: 2005-S-003-02, Title: Diligence-level Digital Video Contents Authoring software development].

Camera character recognition has the advantage of easily acquiring any type of character existing in the real world, unlike conventional paper documents inputted through a scanner. In other words, in addition to inputting a paper document into a scanner, camera character recognition may acquire and recognize characters to be recognized very easily by an input means such as a camera, and then use the recognition result. Camera character recognition can be recognized by inputting not only the indoor environment but also characters existing in the outdoor environment. For example, a guide letter of a tourist destination or an explanation of various data, a monument, a signboard, a business card, etc. may be mentioned. As described above, camera character recognition is a field of character recognition that is being actively researched since it is possible to easily acquire and recognize characters with a portable camera and utilize the recognition results regardless of the medium in which the input target characters are recorded. Such camera character recognition is also utilized for robots that read books and mobile phone camera characters.

As such, camera-based text recognition has an advantage of easily inputting real-world document information into the camera regardless of the medium on which the text is written. Through this, the acquired camera document image is processed by the user's computer and then recognized and used for document editing, database construction, or the like. After receiving the recognized result, it can be converted into a form suitable for user purpose such as foreign language translation and voice real-time service. Recently, the real-world camera character recognition is utilized by the mobile phone camera character recognition and a robot for reading books as described above, and the user's need for camera character recognition is increasing.

However, the camera text image, unlike the scanner text image, has a disadvantage that the conditions for inputting the text image are not limited and are affected by the ambient light and the input camera. . To compensate for this drawback, a technique for camera character recognition has been announced.

The first of these methods is image preprocessing to improve the camera document image. The camera document image is improved using various filters, and then the camera document image is binarized using global binarization and local binarization. Character recognition. Most of the camera document image recognition methods relate to a method of improving a camera document image. This method can be utilized to database paper documents using cameras.

The second method of the present invention relates to acquiring and recognizing a character existing in a real environment with a camera, and to allowing a user to designate and recognize a character region to acquire and recognize a document image with a camera. This method is used to acquire characters written on street signs in real environment with PDA cameras and then recognize them. Only the text area to be recognized in the background of the image acquired by the camera is directly selected by the user and transmitted to the server for recognition, and the recognition result is transmitted back to the user. The character recognition result is used for the purpose of providing the user's convenience, such as searching for information required by the user or translating into a foreign language.

Mainly dealt with in the top two methods is simply a method of image enhancement and binarization for camera character images, and no camera character recognition method for distorted situations caused by camera distortion and ambient lighting effects is presented. As such, a problem in recognizing document images acquired by cameras is to recognize texts by minimizing the effects of vignetting and ambient lighting of cameras. Can not guarantee the recognition performance. In addition, when recognizing a character acquired in a real environment, a feature extraction method or a recognition method capable of recognizing various character forms should be sought. Therefore, the camera character recognition is performed using the image feature or recognition method regardless of the various character images inputted by the camera and the method of properly extracting and recognizing the character region from the camera document image to be robust to the surrounding environment (minimizing distortion and environmental influence). Should be performed.

As such, camera document images are blurring or noisy due to camera distortion and ambient light effects, and image enhancement algorithms or new binarization techniques are proposed to solve the problem that character segmentation and recognition are difficult. However, even if the characters to be recognized are extracted using these methods, it is difficult to extract the individual characters. Therefore, in addition to the application of image enhancement algorithms that have been mainly studied for camera document recognition, it is necessary to search for feature extraction and recognition methods that can properly recognize the deformed characters generated while acquiring the document image using the camera input means.

Accordingly, the present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to divide a camera document image into a plurality of sub-document images in which pixels are moved, and input each document. After applying the image enhancement algorithm and the binarization to the image, we extract the feature of the character using the combined character feature extractor such as the pixel moving network feature, and then recognize the character through the weighted heterogeneous recognizer to obtain the final recognition result. Disclosed is a camera character recognition apparatus and method using a pixel shift document image combination recognition method capable of improving capability.

Specifically, the camera document image is obtained by dividing the sub-document image into which pixels are moved, and then the image quality is improved by image preprocessing for each sub-document image, and the local binarization threshold is investigated by examining the intensity distribution of each pixel. Obtain the binarized character image by Here, the proposed local binarization is not a problem for processing multiple sub-document images since the processing time is up to 10 times faster than the conventional local binarization. By moving the pixels and recognizing the sub-document images, it is possible to solve the problem that characters that are not recognized are often generated because an unrecognizable character region exists in the edge region of the camera document image. In addition, the individual characters are extracted by applying a combination and separation algorithm to each sub-document image, and then the vertical and horizontal pixel shifting network features are extracted and recognized by a heterogeneous recognizer. In this case, the heterogeneous recognizer can ensure a more improved recognition performance by playing a role of complementing the disadvantages of each recognizer. The recognition result multiplied by the weight of each recognizer is used as the final recognition result. Through this, the present invention is not only to apply the image enhancement algorithm and the binarization method that has been intensively studied for the existing low resolution camera document image recognition, but also to recognize the various types of camera characters and unrecognized regions of the camera document image. By suggesting the character feature extraction and the character recognition method together, it is not only to improve the recognition performance of the camera character but also to ensure stable recognition performance.

Camera character recognition apparatus using the pixel moving document image combination recognition method of the present invention for achieving the above object, by processing a plurality of sub-document images of the pixel is moved to the input image An output sub-document image processing unit; A binarization unit configured to locally binarize an image that has undergone an image enhancement algorithm on the sub-document image; A character processing unit for extracting structural characteristic information of individual characters and pixel moving network characteristics of a character region to which pixels are moved with respect to the binarized image; And a recognition processing unit for outputting a recognition result by combining a classification of a character type and a heterogeneous recognizer for recognizing the individual character.

On the other hand, the camera character recognition method using the pixel shift document image combination recognition method of the present invention, (a) processing and outputting a plurality of sub-document images (pixels) with respect to the input image shifted; ; (b) applying an image enhancement algorithm to the sub-document image; (c) extracting a binary document image by applying binarization to an image that has undergone the image enhancement algorithm; (d) extracting structural feature information of individual characters and pixel moving network characteristics of a character region to which pixels are moved to the binarized image; And (e) combining the classification of character types and a heterogeneous recognizer for recognizing the individual characters, and outputting a recognition result.

As described above, the present invention obtains a pixel shifted sub-document image using a camera, applies an image enhancement algorithm, performs local binarization on the sub-document image, and then combines character extraction, character feature extraction, and heterogeneous recognizer. A method of character recognition is proposed. In addition, the present invention is to apply the sub-document image input, image enhancement algorithm and local binarization method for each sub-document image in order to reduce the effect of ambient lighting and distortion of the camera. In addition, the individual characters are extracted using a combination and separation algorithm to extract the pixel moving network feature to classify the characters by type and to recognize them by combining heterogeneous recognizers.

As described above, the apparatus and method for recognizing a camera character using the pixel shift document image combination recognition method according to the present invention combines a weighted heterogeneous recognizer by extracting and weighting individual character features such as pixel shift sub-document image input and pixel shift network feature. By recognizing by using the method, it is possible to improve the recognition performance by compensating for the problem that the existing camera character image was considered to be difficult to recognize due to the influence of the ambient light or the camera lens.

Recently, camera character recognition is used in robot visions such as robots reading books, or as a means of acquiring information by embedding camera functions and recognition functions in mobile devices such as mobile phones. While ensuring that the recognition results can be combined with other application services, it can provide convenience to the user.

Hereinafter, an apparatus and method for recognizing a camera character using the pixel shift document image combination recognition method of the present invention will be described in detail with reference to the accompanying drawings.

1 is a control circuit block diagram of a character recognition apparatus using a pixel shift document image combination recognition method according to an embodiment of the present invention.

Referring to FIG. 1, the character recognition apparatus using the pixel shift document image combination recognition method of the present invention is largely divided into a sub-document image processing unit 1, a binarization unit 2, a character extraction unit 3, and a character feature extraction unit. (4), the recognizer 5, and the coupler 6 are comprised. Here, the character extractor 3 and the character feature extractor 4 may be defined as a character processor, and the recognizer 5 and the combiner 6 may be defined as a recognition processor. In addition, the character recognition apparatus using the pixel moving document image combination recognition method of the present invention may be embedded in the camera itself, and may be mounted in a server to receive a service through a network from the camera.

The sub-document image processing unit 1 has a function of outputting a plurality of sub-document images in which pixels are moved when inputting a document image in order to eliminate an impossible recognition area in the camera document image. Perform.

In the binarization unit 2, distortion and focus problems according to the characteristics of the camera lens are generated, so that a character segmentation or recognition is often difficult in a particular portion of the camera document image, particularly characters located at the edge of the document. In addition, in the case of a low resolution camera, a text image may be blurred or overlap with a neighboring text. Therefore, the image enhancement algorithm is applied to the pixel shifted document image, and the local binarization is performed by examining the pixel contrast value of each sub-document image.

The character extracting unit 3 performs a function of dividing the binarized document image into individual characters from structural feature information of the individual characters. Here, the character extracting unit 3 includes a function of extracting a word from a binarized document image using a projection technique, and a function of extracting individual characters from a word region by applying a combination and separation algorithm.

The character feature extracting unit 4 extracts a feature for the extracted individual character. In order to extract a character feature irrespective of the deformation of the camera character, a pixel is extracted for each of the extracted individual characters. And extracting pixel moving network features from the moved text area, and extracting structural features such as contours, chain codes, distances, and aspect ratios of the text images.

The recognizer 5 performs recognition on the extracted characters, and in order to reduce the burden of the recognizer, an MLP type classifier 51 capable of classifying characters and a character having various shape variations such as a camera character are recognized. It consists of a heterogeneous recognizer for individual characters, that is, a Hidden Markov Model (HMM) and a Multi-Layer Perceptron (MLP) 52.

The combiner 6 performs a function of deriving a final recognition result by assigning weights to the recognition results outputted from the heterogeneous recognizer and weighting and combining the recognition results outputted from the MLP type classifier.

2 is a flowchart of a character recognition method using a pixel shift document image combination recognition method according to an embodiment of the present invention.

Referring to FIG. 2, a pixel moving document image combination recognition method according to an embodiment of the present invention is performed by using a camera, a mobile phone camera, a web camera, or the like for a character to be recognized (indoor / outdoor document, sign / card, guide / explanation). In operation S1, a sub-document image obtained by shifting a pixel to the right is obtained according to the input to the character recognition apparatus using the SLR. An image enhancement algorithm is applied to properly extract the text part of the acquired text image. The intensity distribution of the pixels is calculated to perform local binarization on the sub-document image (S2). The words and the individual letters are extracted using the combining and separating algorithm (S3). Up, down, left, and right pixel shifting mesh features and structural feature vectors are extracted for the extracted individual characters (S4). The extracted individual characters are recognized by the heterogeneous recognizer (S5). The recognition result obtained from each recognizer is weighted and then multiplied by the results of the two recognizers to obtain a final recognition result (S6). After that, the recognition result and application software are combined to suit the user's convenience. Hereinafter, specific embodiments will be described in detail for each step.

Example

First, a recognition target camera document image is obtained. That is, camera character image samples are obtained by varying the distance between the camera and the document in order to obtain various types of recognition target documents. Also, samples may be input to a document image in which various character fonts are recorded. Character images used in the simulation are images obtained in an unconstrained lighting condition environment. In order to experiment with the recognition performance of various camera character images, a character image written in a printed form is acquired regardless of a medium in which characters are recorded in indoor and outdoor environments.

S1: Unlike a character image input by a scanner, a camera character image generates a vignette phenomenon and a blurring phenomenon in which a character image is blurred at the edge of the acquired character image due to ambient lighting effects. Since these factors act as factors causing mis-division and misrecognition of characters, a method of properly extracting and recognizing character parts is required. Therefore, the sub-document image in which the pixel is moved is acquired by the camera to reduce the misrecognized character area. 3 is a diagram illustrating document image samples input to a camera.

S2: The following [Formula 1] is applied to convert the color image to the brightness image.

--- [수식 1]

--- [Formula 1]

는 각각 0.11, 0.59 및 0.30으로 계산하였다. The natural color image is converted into a 256 color image and then converted into a brightness level image using Equation 1 above. At this time,

Were calculated to be 0.11, 0.59 and 0.30, respectively.

In addition, binarization is performed by calculating a threshold for local binarization for each sub-document image. By applying an image enhancement algorithm to partially localize the text image in the local region of the input image, the text region can be extracted insensitive to the ambient lighting effects. Local binarization results for sub-document images are well illustrated in FIG. 4. In the target region rxr where the binarization threshold is to be calculated, the difference between the pixel value having the highest brightness level and the lowest pixel value is obtained, and then calculated by the following [Equation 2] to perform binarization with a value corresponding to t of this difference value. do.

--- [수식 2]

--- [Equation 2]

와

는 국소 영역 r x r에서 명도 레벨이 최대 및 최소 화소 값을 각각 나타낸다.

는 실험 결과 얻어진 값이다. At this time,

Wow

In the local region rxr, brightness levels represent maximum and minimum pixel values, respectively.

Is the value obtained from the experiment.

S3: A word is extracted using vertical projection and margin information for character division, and then individual characters are extracted using a combination and separation algorithm. Hangul, unlike other characters, combines vowels and consonants to form letters, so that individual letters are extracted by applying a combination and separation algorithm for the connected pixels. Strings, words and individual character extraction processes and results are well illustrated in FIGS. 5A-5C.

S4: Extract the feature of the individual letter using the structural feature information of the character to be recognized. Features of individual characters are extracted by using mesh features, distance information features, and contour information. Among them, the mesh feature information extracts up, down, left, and right pixel moving mesh features. 6 illustrates a concept of pixel moving network feature extraction.

S5: Classify and recognize the types of the extracted individual characters. In order to recognize characters mixed with Korean, English, symbols, and numbers, Korean characters are classified into six types of characters, and English, symbols, and numbers are classified into non-Hangul types. In this embodiment, a type classifier using MLP and an individual character recognizer using HMM and MLP are implemented.

S6: In order to recognize characters that are difficult to recognize, such as camera character recognition, weights are combined by using a heterogeneous recognizer, and then the characters are recognized using a multiplication method. 7 is a diagram illustrating a weighted heterogeneous recognizer combination recognition method. As shown in FIG. 7, weights are respectively assigned to the recognition result 1 that combines the result of performing the recognition using a plurality of HMMs for the individual characters and the recognition result 2 that combines the result of performing the recognition using the MLP. To combine.

Afterwards, the text recognition result is used for document editing, database construction, or combined with application software such as foreign language conversion and voice service.

As described above, in the present invention, the pixel-shifted sub-document image is acquired by using a camera, and then the image quality is improved by using an image enhancement algorithm, and local binarization is performed to eliminate the document region that is not recognized. In order to extract and recognize individual characters, various structural character feature information is extracted and combined. Especially, the pixel shifted network feature is extracted and applied for character recognition with various forms such as camera characters. In addition, the heterogeneous recognizer was designed to obtain recognition results of each sub-document image, and the recognition results were weighted by using the advantages of the complementary functions of the heterogeneous recognizers, and the final recognition results were obtained by multiplying the two recognition results. .

As described above, since the camera character recognition is very difficult to recognize characters due to the influence of the ambient light and the camera lens, unlike the conventional scanner character recognition, the present invention improves the image by inputting a plurality of pixel shifted sub-document images. Local binarization was allowed. When extracting the features of individual characters, we also extract features that take into account the shape change of the camera characters, and improve the recognition performance of camera character recognition by combining them with heterogeneous recognizers.

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

1 is a control circuit block diagram of a character recognition apparatus using a pixel shift document image combination recognition method according to an embodiment of the present invention;

2 is a flowchart of a character recognition method using a pixel shift document image combination recognition method according to an embodiment of the present invention;

3 is a diagram illustrating document image samples input to a camera;

4 is a diagram illustrating local binarization results for sub-document images;

5A to 5C are diagrams illustrating a process of extracting a string, a word, and an individual character, and results;

6 is a diagram illustrating pixel moving network feature extraction;

7 is a diagram illustrating a weighted heterogeneous recognizer combination recognition method.

<Explanation of symbols for main parts of the drawings>

1: sub-document image processing unit 2: binarization unit

3: character extracting unit 4: character feature extracting unit

5: recognizer 51: MLP type classifier

52: Hidden Markov Model (HMM) and Multi-Layer Perceptron (MLP)

6: combiner

Claims (12)

A sub-document image processing unit which processes and outputs a plurality of sub-document images in which pixels are moved with respect to the input image; A binarization unit configured to locally binarize an image that has undergone an image enhancement algorithm on the sub-document image; A character processing unit for extracting structural characteristic information of individual characters and pixel moving network characteristics of a character region to which pixels are moved with respect to the binarized image; Recognition processing unit for outputting the recognition result by combining the classification of the character type and the heterogeneous recognizer for recognizing the individual characters Camera character recognition apparatus using a pixel moving document image combination recognition method comprising a. The method of claim 1, wherein the character processing unit, A character extraction unit for dividing and extracting the individualized characters from the structural feature information of the individual characters in the binarized image; And Extracting the pixel moving network feature for the character region in which the pixels are moved in the up, down, left, and right directions with respect to the extracted individual characters, as well as the aspect ratio of the contour, chain code, distance, and character image Character feature extraction unit for extracting structural features including Camera character recognition apparatus using a pixel moving document image combination recognition method comprising a. The method of claim 2, wherein the character extracting unit, And extracting a word from the binarized image by using a projection technique, and extracting individual characters by applying a combination and separation algorithm to the word. The method according to any one of claims 1 to 3, wherein the recognition processing unit, An MLP type classifier for classifying types of characters including Korean, English, symbols, and numbers; A heterogeneous recognizer configured to perform character recognition in response to a character already learned for the individual character; And A combiner that derives the final recognition result by giving weight to the recognition result outputted from the heterogeneous recognizer and weighting and combining the recognition result outputted from the MLP type classifier. Camera character recognition apparatus using a pixel moving document image combination recognition method comprising a. The apparatus of claim 4, wherein the heterogeneous recognizer comprises a plurality of Hidden Markov Models (HMMs) and a Multi-Layer Perceptron (MLP). 6. (a) processing and outputting a plurality of sub-document images in which pixels are moved with respect to an input image; (b) applying an image enhancement algorithm to the sub-document image; (c) extracting a binary document image by applying binarization to an image that has undergone the image enhancement algorithm; (d) extracting structural feature information of individual characters and pixel moving network characteristics of a character region to which pixels are moved to the binarized image; And (e) combining the classification of character types and a heterogeneous recognizer that recognizes the individual characters and outputting a recognition result; Camera character recognition method using a pixel shift document image combination recognition method comprising a. 7. The method of claim 6, wherein the image enhancement algorithm of step (b) is a brightness level normalization method of extracting only a portion of a character by emphasizing a character from a background. 7. The camera according to claim 6, wherein the binarization of the step (c) is performed by localizing the binarization threshold by investigating the intensity distribution of the pixels with respect to the sub-document image. Character recognition method. 7. The pixel moving document image combination recognition according to claim 6, wherein the step (c) further includes a noise removing step of analyzing and removing the size of the connection pixel including a table and a picture other than the characters resulting from the binarization. Camera Character Recognition Method The method of any one of claims 6 to 9, wherein the extracting of the pixel shift network feature for the character region to which the pixel of step (d) is moved comprises: (d-1) dividing and extracting the individualized characters from the structural feature information of the individual characters in the binary image; And (d-2) Extract the pixel movement network feature for the character area in which the pixels are moved in the up, down, left, and right directions with respect to the extracted individual characters, as well as the horizontal lines of the contour, chain code, distance, and character image. Extracting structural features including aspect ratios Camera character recognition method using a pixel shift document image combination recognition method comprising a. 11. The method of claim 10, wherein the step of dividing into individual letters of the step (d-1) comprises extracting a word from the binarized image using a projection technique, and applying a combining and separating algorithm to the word. Camera character recognition method using a pixel shift document image combination recognition method characterized in that the extraction of characters. The method of claim 11, wherein combining the classification of the character type and a heterogeneous recognizer for recognizing the individual character and outputting a recognition result includes: (e-1) classifying a type of characters including Korean, English, symbols, and numbers in a multi-layer perceptron (MLP) type classifier; (e-2) performing character recognition using a heterogeneous recognizer corresponding to a character already learned for the individual character; And (e-3) deriving a final recognition result by assigning weights to the recognition results outputted from the heterogeneous recognizer and weighting and combining the recognition results outputted from the MLP type classifier; Camera character recognition method using a pixel shift document image combination recognition method comprising a.

Images