KR20040023498A - Device and method for recognizing character image in picture screen - Google Patents

Abstract

PURPOSE: An apparatus and method for recognizing a letter image in an image screen is provided to recognize a letter image, contained in an image screen, through a device having an image processing function. CONSTITUTION: A control part(101) controls the overall operations of a portable terminal that recognizes a document. A memory(103) stores a program to control the operations of the portable terminal, and executes a function to temporarily store the data generated in the execution of the program. A camera(107) executes a function to photograph the image of a document. An image processing part(109) converts an image screen, photographed from the camera(107), into digital data. An audio processing part(111) processes a voice for a modification of a text having an error generated in the execution of the program. An input part(113), a touch screen module, can be embodied with a display part(115) in a single unit. The input part(113) generates desired text and function key input using a stylus pen. A key input part(105) comprises function keys to set up various function of the portable terminal.

Description

DEVICE AND METHOD FOR RECOGNIZING CHARACTER IMAGE IN PICTURE SCREEN}

The present invention relates to an apparatus and method for character recognition, and more particularly, to an apparatus and method for recognizing a character image included in an image screen.

Currently, portable terminal devices are converting to a structure capable of transmitting high speed data. In particular, if a mobile communication network of the IMT 2000 standard is implemented, high-speed data communication may be implemented using the small portable terminal device. Data that can be processed by the portable terminal device performing the data communication may be packet data and image data.

However, since the portable terminal device uses a limited keypad when inputting information, a method of inputting a text is complicated. That is, since the portable terminal device uses a soft keyboard input device, character input speed is slow and very cumbersome. Therefore, a text and / or voice recognition device may be used to solve the disadvantage of the soft keyboard. However, even when the handwritten character recognition apparatus is used, the sentence recognition and input speed are slow. And even when using a speech recognition device has a problem that only limited word recognition. Therefore, a method of using a separate hardware keyboard input device for character input can be used. However, the above method has a problem in that the portable terminal requires an additional device for text input.

In addition, an image processing function is added to the portable terminal device to have a complex function. The portable image processing apparatus as described above includes a camera for photographing an image and a display unit for displaying an image signal photographed from the camera. The camera may use a CCD or a CMOS sensor, and the display unit may use an LCD. In addition, with the miniaturization of the camera device, the device for capturing the image is becoming more and more miniaturized. In this case, the portable terminal device can capture a video screen to display a moving picture and a still picture, and can also transmit the captured screen. However, the portable terminal device with the camera performs only a simple function of capturing, storing, managing, and transmitting a video screen.

The portable terminal device may be a portable telephone and / or a PDA. The conventional character input method of the PDA terminal is a pen (stylus pen) input using a soft keypad, or through a handwriting recognition method. However, the conventional character input methods as described above are causing great inconvenience to users because of the slow speed and cumbersome work when a large amount of characters need to be input. In particular, if you want to input the contents of the business card containing a lot of information to the PDA requires a lot of time and effort. Therefore, it is essential to develop a new input method or a method that can emphasize user convenience.

Accordingly, an object of the present invention is to provide an apparatus and method for recognizing a character image included in an image screen in an apparatus having an image processing function.

Another object of the present invention is to provide an apparatus and method for recognizing a text image included in an image screen and storing the text image included in an image screen in a device having an image processing function.

Another object of the present invention is to provide an apparatus and method for recognizing a character image included in an image screen and correcting an error of the recognized character information in an apparatus having an image processing function.

Another object of the present invention is to provide an apparatus and method for preprocessing a text image area included in an image screen in a device having an image processing function and recognizing it.

Still another object of the present invention is a device capable of capturing a document using a camera in a terminal device having a camera, recognizing an image of the photographed document as a character, and using a candidate character table when correcting an error of a recognized character. And providing a method.

Still another object of the present invention is to shoot a document using a camera in the terminal device having a camera and a voice recognizer, recognize the image of the photographed document as a character, an error occurs through the voice recognizer when correcting the error of the recognition character It is an object of the present invention to provide an apparatus and a method for modifying a given character.

Still another object of the present invention is to photograph a document using a camera in a terminal device having a camera, recognize the image of the photographed document as a character, and recognize a handwritten character input by a user when correcting an error of a recognized character. It is to provide an apparatus and method for modifying the generated character.

Still another object of the present invention is to shoot a document using a camera in the terminal device having a camera, recognize the image of the photographed document as a character, when the error of the recognition character by using a soft keypad character It is to provide an apparatus and method for modifying the.

Accordingly, an object of the present invention is to provide a device and method for photographing a document including phonebook information using a camera in a portable communication device having a camera, and recognizing and storing phonebook information included in an image of the photographed document. have.

1 is a diagram illustrating a configuration of an apparatus for document recognition according to an embodiment of the present invention.

2 is a diagram illustrating a procedure of recognizing a document according to the first embodiment of the present invention.

3 is a diagram illustrating an operation procedure of a process of photographing a document of FIG.

4 is a diagram illustrating a configuration of a preprocessor 121 in an image document processing apparatus according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a configuration of an image blurring determiner for determining whether an image screen is blurred in FIG. 4. FIG.

FIG. 6 is a diagram illustrating a block classification unit configuration of FIG. 5. FIG.

FIG. 7 is a diagram illustrating a configuration of a letter block energy calculation unit of FIG. 5. FIG.

8 is a diagram illustrating a procedure of determining whether an image blur is blurred by an image blurring decision unit;

FIG. 9 is a diagram illustrating a configuration of a subject tilt correction unit for correcting a tilt of a subject in an image screen in FIG. 4; FIG.

FIG. 10 is a diagram illustrating a configuration of the binarization unit in FIG. 9. FIG.

FIG. 11 is a diagram showing the detailed configuration of the block classification unit in FIG. 10; FIG.

FIG. 12 is a diagram for describing a procedure of calculating a rotation angle of a stripe in the rotation angle determination unit of FIG. 9. FIG.

FIG. 13 is a diagram illustrating a procedure of correcting a tilt of a subject in an image screen in a subject tilt correction unit; FIG.

FIG. 14 is a diagram illustrating a configuration of an image region expansion unit which extends a character region in an image screen in FIG. 4; FIG.

15 is a diagram illustrating the configuration of a block classification unit of FIG. 14.

FIG. 16 is a diagram for explaining a text area expansion procedure in an image area expansion unit; FIG.

FIG. 17A is a diagram illustrating adjacent pixels in the noise canceling unit of FIG. 4, and FIG. 17B is a diagram illustrating four directions of center pixels in the noise removing unit.

18A to 18D are diagrams illustrating pixel pairs of respective directional regions in the noise canceling unit of FIG. 4.

19 is a diagram illustrating a configuration of an image binarization unit in FIG. 4.

20 is a diagram illustrating a detailed configuration of a block classification unit in FIG. 19.

FIG. 21 is a diagram showing a detailed configuration of the edge enhancement unit in FIG. 19; FIG.

FIG. 22 is a diagram for explaining an operation of performing edge enhancement of a letter block in an edge enhancement unit; FIG.

FIG. 23 is a diagram for describing a procedure of binarizing an image screen in an image binarization unit using a quadratic filter; FIG.

24A and 24B illustrate an operation procedure of the character recognition and storage item selection process of FIG. 2.

25A and 25B illustrate an operation procedure of the error correction process of FIG. 2.

26A to 26E are views showing the display unit state in the process of photographing a document;

27A to 27B are diagrams showing a display unit state of a character recognition and storage item selection process;

28A-28D show a display state of an error correction process;

29A to 29B show the display unit state after error correction

30 is a diagram illustrating a procedure of recognizing a document according to the second embodiment of the present invention.

FIG. 31 is a view illustrating an operation procedure of a process of photographing a document of FIG. 30;

32 is a flowchart illustrating an operation procedure of a character recognition, storage item selection, and storage process of FIG. 30;

33 is a diagram illustrating a detailed operation procedure of a storage item selection process of FIG. 32;

34A to 34D illustrate an operation procedure of the error correction process of FIG. 30.

DETAILED DESCRIPTION A detailed description of preferred embodiments of the present invention will now be described with reference to the accompanying drawings. It should be noted that the same components in the figures represent the same numerals wherever possible.

In the following description, specific details such as business card, PDA, size of video screen, etc. are shown to provide a more general understanding of the present invention. It will be apparent to one of ordinary skill in the art that the present invention may be readily practiced without these specific details and also by their modifications.

According to an embodiment of the present invention, a terminal device having an image processing function recognizes a text image included in an image screen and stores the text image as a document. That is, the embodiment of the present invention improves the user's character input method (usability), minimizes the operation of the user's input device, and recognizes a character that is misrecognized when the character is recognized when the character image of the video screen is recognized and documented. Easy to modify and enable a large amount of sentence input.

To this end, the terminal device according to an embodiment of the present invention, the image preprocessing function for preprocessing the character image in the image screen before recognizing the character image included in the image screen, the recognition function for recognizing the character image in the pre-processed image screen, It has a function of correcting misrecognized character information among the recognized character information. And in order to correct the misrecognized character information, the terminal device according to an embodiment of the present invention is a voice recognition function for correcting a misrecognized character by voice, handwriting recognition for correcting a misrecognized character by the user's handwriting input It may have a modified user interface having a function, a function for displaying and selecting candidate letters having letters similar to a misrecognized character, and / or a function for inputting a misrecognized character with a softkey pad.

The terminal device according to the embodiment of the present invention as described above has the above-described configuration, recognizes the text image included in the video screen, and edits and stores it as a document. In this case, the document of the video screen may be a document having a certain form, or may not be. In addition, the terminal device may be a device having a camera, photographing a document to be recognized through the camera, and recognizing a character image included in the captured video screen. In addition, the terminal device is a device having a communication function, and may recognize a text image included in a received video screen and store it as a document. The terminal device may include an external input device, store an image screen input by the external input device, and recognize a text image included in the stored image screen and store the image as a document.

In order to implement the above features, the camera of the terminal device preferably uses a camera capable of fine focus adjustment. This is to improve the resolution of the document image for recognition.

As described above, the image preprocessing function for character recognition requires support of hardware specifications and software specifications. First of all, the hardware spec should support the focus fine adjustment function of the captured video image, obtain the display speed (minimum 12 fps) to check the optimal focus state when adjusting the focus, and secure the maximum screen size to check the optimal focus state when adjusting the focus. In addition, an excellent lens for obtaining the best image quality for character recognition should be secured. The software preprocessing should restore the circular distortion of the camera image by the pinhole lens, the distortion of the focus mismatch of the target image due to the close-up photography can be restored, and determine the suitability of the character recognition of the character size and focus adjustment. It should be possible to be able to recover the distortion of the image due to the non-vertical projection of the object, and to be able to binarize the object character under complex lighting conditions.

In addition, a text recognition function for recognizing an image of a document photographed from a camera as a text should be added as described above. In order to recognize the character, an engine for recognizing light characters should be developed, and the engine size should be smaller than a predetermined amount of data (engine size <5 Mbytes), and the characters to be recognized should be printed fonts of various fonts in English, Korean, and numbers. The minimum recognition rate (at least 80% recognition rate per character) should be secured. In addition, it is preferable to include a voice recognition module to correct the error text by the voice at the time of error correction, it is necessary to implement a sentence input user interface by the text recognition and voice recognition.

As described above, it is assumed that the terminal device according to the embodiment of the present invention is a PDA, and that the document to be photographed is a business card. And photographing the image of the business card, preprocessing the photographed image to extract a text image, recognizing the extracted text image, converting it into text data, correcting the error of the recognized text data, and storing it in the phone book. An example is demonstrated.

In the background of the present invention in the embodiment of the present invention using a variety of input devices (character recognizer, voice recognizer, pen, keyboard), such as to easily enter a document containing a lot of information, such as a business card to the PDA such as Suggest a way.

First, a business card or document is photographed using a camera embedded in the PDA, and then the text image in the image screen is sharply processed by the preprocessor, and the text image is recognized by the text recognizer and converted into text data. In the converted contents, an error part is corrected by various means such as stylus pen, voice recognition, handwriting recognition, soft keypad, and stored in a database of a desired area.

1 is a diagram illustrating a configuration of a portable terminal device recognizing a character image included in an image screen according to an embodiment of the present invention.

Referring to FIG. 1, the controller 101 controls the overall operation of a portable terminal device recognizing a document. The memory 103 stores a program for controlling the operation of the portable terminal device, and also temporarily stores data generated during program execution.

The camera 107 performs a function of photographing an image of a document. The document may be a business card. The camera may be a camera capable of performing a preprocessing function. That is, the camera is a camera that can adjust the focus and distance, and can improve the quality of the captured video screen. The image processor 109 may perform a function of converting and compressing and encoding an image screen captured by the camera 109 into digital data. The image processor 109 may use the image processor of Korean Patent Application No. 2002-22844 filed by the applicant of the present application.

The audio processor 111 processes a voice for correcting a character in which an error has occurred during program execution, and also processes a voice for displaying a guide and a result of execution during program execution. The input unit 113 may be implemented integrally with the display unit 115 as a touch screen module.

The input unit 113 may generate a desired character and function key input using a stylus pen. The input unit 113 includes a photographing key, a document recognition key, a confirmation key, a correction key, a completion key, an insertion key, a deletion key, and the like for recognizing a document taken according to an embodiment of the present invention. The photographing key is a key for storing a displayed image image. The document recognition key is a key for recognizing a text image of a video screen currently being displayed. In this case, when each of the recognized documents has a unique form, a corresponding document recognition key may be provided. For example, in the case of a document in which specific information is recorded, such as a business card, a phone book of the portable terminal device can be created using the document. In this case, if a business card recognition key is provided and items for selecting common information recorded on the business card are stored in a table, the phone book of the portable terminal device can be easily created. The confirmation key is a key for registering character data of the selected item. The modify key is a key for correcting the character data of the selected item. The insertion key is a key for adding a character at a position when a cursor exists at a specific position of the selected sentence. That is, when one or more letters are missing from the text recognition result sentence, a new character can be inserted before the cursor. The delete key is a key for deleting the character data of the selected item. The completion key is a key for ending the current operation.

The key input unit 105 includes function keys for setting various functions of the portable terminal device. Here, the function keys that may be located in the key input unit 105 may store a voice recognition key for driving the voice recognition unit 123, a focus and distance adjustment key for controlling a preprocessing operation of the camera 107, and a preview screen output from the camera 107. It can be composed of photo taking keys to Of course, the keys of the key input unit 105 may also have the input unit 113. In the embodiment of the present invention, it is assumed that all the function keys are arranged in the input unit 113 for convenience of description. The camera 107, the input unit 113, the audio processor 111, and the key input unit 105 may all operate as input devices.

The display unit 115 displays a state of the character recognition process performed by the embodiment of the present invention. That is, the display unit 115 includes an area for displaying a video image captured by the camera 107 on a preview screen, displaying a result of character recognition when performing a character recognition mode, and displaying an error correction result. The display unit 115 includes a first display area 71, a third display area 73, and a second display area 75. The first display area 71 is an area for displaying recognized character data, the second area 73 is an area for displaying text data of a selected item or candidate character data for error correction, and the second display area 75 is for item information or It may be an area for displaying handwritten characters inputted for error correction, and / or a soft keypad for inputting characters. In addition, specific areas of the first display area to the second display area 75 may be appropriately positioned to display menu information for inputting various commands during character recognition according to an embodiment of the present invention.

The controller 101 drives the preprocessing unit 121 and the character recognition unit 123 when a character recognition key is input from the input unit 113.

First, the preprocessor 121 performs a preprocessing operation by inputting an image screen displayed on the display 115. The preprocessing unit 121 determines whether the input video screen has a recognizable resolution or a blurred picture. The controller 101 notifies the controller 101 of the determined result, and if it is determined that the blood video screen is determined, the controller 101 displays the recognition impossible on the display unit 115. However, if the video screen is not a blood video screen, the preprocessing unit 121 checks whether the subject is inclined in the video screen to correct the inclination of the subject, and removes an area where the image does not exist in the video screen, and then displays the image. And removes noises in the video screen, and outputs the binarized pixels in the video screen. The preprocessing unit 121 may include all of the above-described image blurring determination, subject tilt correction, image region expansion, noise reduction, and image binarization, and may include only some of the above functions.

Second, the character recognition unit 123 performs a function of recognizing character images of the image screen preprocessed by the preprocessor 121 and converting the character images into character data. The recognized character data are displayed in the first display area 71 of the display unit 115 under the control of the controller 101. The character recognition unit 123 may be composed of a print recognition module and a handwriting recognition module. The print recognition module may be used as a module for recognizing a character image in an image screen preprocessed by the preprocessing unit 121, and the handwriting recognition module may be used as a module for recognizing a handwritten character image input when an error is corrected. In addition, the character recognition unit 123 may include a module capable of converting soft key data input by a soft keypad into a character.

The controller 101 drives the recognition error processor 125 when an error correction key is input from the input unit 113. The recognition error processor 125 corrects a character selected from the text data displayed on the first display area 71 of the display unit 115 with a character modified by the voice recognition unit 129 or the character recognition unit 123 and recognizes a character generated when the character is recognized. Modify

The controller 101 drives the voice recognition unit 129 when the voice recognition key is received while the error correction key is input. The voice recognition unit 129 recognizes the voice received from the audio processor 111. In this case, the voice recognition may be a voice for selecting an item for which error correction is desired and a voice for correcting an error text generated from the selected item. The voice recognition unit 129 converts the input voice signal into text data to correct the error text. The voice synthesizer 127 performs a function of synthesizing the text data of the recognized result into voice under the control of the controller 101 and outputting the voice. That is, the voice recognition unit 129 performs a function of correcting an error by converting a voice signal for correcting a text data which is misrecognized in the recognition process under the control of the control unit 101 into text data, and the voice synthesis unit 127 controls the control unit 101. After the recognition is completed under the control of the text data to be stored by synthesizing the voice function.

The database 131 stores the recognized text data corresponding to each item under the control of the controller 101. When the recognized document is a business card, the database 131 may be a phone book or an address book memory. The user interface unit 133 functions to interface user data connected to the terminal device with the portable terminal device.

As described above, the portable terminal device according to the embodiment of the present invention includes a camera module, an input (including a touch screen) module, an audio module, a preprocessing module, a character recognition module, a recognition error correction module, a voice recognition and synthesis module, and a user interface module. And the like. It consists of six parts of the terminal device having the above configuration, which may be a video image input process, preprocessing process, character recognition process, storage item selection process, error correction process and storage process. Each of the above processes are organically connected to each other, and can be implemented in a variety of ways internally. Looking briefly at the main modules used in each process, the image image process is performed by the camera module, the pre-processing process of the image screen is performed by the pre-processing module, the character recognition process is performed to the character recognition module and the voice recognition module , The storage item selection process is performed by the voice recognition module and the stylus pen module, and the error correction process is performed by the voice recognition module, the stylus pen module, the handwriting recognition module, and the softkey recognition module. Is performed by the database module.

The document recognition procedure may be implemented in various ways. According to the first embodiment of the present invention, as shown in FIG. 2, after photographing a document on an image screen, preprocessing character images in the captured image screen, and recognizing character images in the preprocessed image screen, A method of sequentially implementing an operation of correcting and storing an error by selecting an item of characters is illustrated. In the second embodiment of the present invention, after photographing a document on an image screen, preprocessing the text images of the photographed video screen, recognizing the preprocessed text images as text data, and selecting an item to be corrected for error are misrecognized. A method of modifying and storing a character and then repeatedly implementing an operation of selecting an item to correct an error is illustrated. Here, in the first and second embodiments, the document to be recognized on the image screen has been described as an example, but the operation of capturing the document on the image screen to be recognized may be omitted. That is, the terminal device may recognize the document image while performing the above procedure even when the character screen function is selected after selecting the video screen stored or the video screen input from the external device.

In addition, in the following description, the error correction of the first embodiment of the present invention will be described as a method of implementing an operation of selecting an item to be corrected and correcting an error through a document recognizer, and the error correction of the second embodiment corrects the error. The selection and error correction operations of the item will be described as a method of implementing the document recognizer and the speech recognizer. However, in the first embodiment, the document may be recognized and modified using the document recognizer and the voice recognizer, and in the second embodiment, the document may be recognized and modified using only the document recognizer.

First, a document recognition procedure according to the first embodiment of the present invention will be described.

2 is a flowchart showing a document recognition procedure according to the first embodiment of the present invention.

Referring to FIG. 2, the controller 101 generates a video screen of a document to be photographed and recognized in step 200. In this case, the video image photographed by the camera 107 is converted into digital data by the image processor 109 and displayed on the display unit 115. In this case, the photographing of the subject may be taken as a moving picture or a still image. In this case, when shooting with a video, the controller 101 displays a video signal photographed in the form of a preview screen on the display unit 105, and when a picture taking command is generated while the video signal is displayed on the display unit 115, the controller 101 displays the display unit. The video image displayed on the screen is displayed as a still image, and the video image displayed on the display section 115 is stored in the image memory area of the memory 103. In this case, the video image displayed on the display unit 115 may be a general video screen, or may be a video screen having a character image such as a business card. In the embodiment of the present invention, it is assumed that the captured video screen is a video screen having a text image.

In addition, the 200 process may be omitted. In this case, the user may display the stored video screen or the input video screen on the display unit 115. That is, when a user wants to recognize a document, the user may select and display a stored video screen or an input video screen, and perform a character recognition process while the video screen is displayed.

In the above state, while the user of the terminal device generates a key command for recognizing the text images included in the image screen currently displayed through the input unit 113, a document recognition procedure according to an embodiment of the present invention is performed. It is assumed here that the key for recognition is a document recognition key. Then, the controller 101 drives the preprocessor 121 in step 210 by the input of the document input key. The preprocessor 121 may include an image blur determination unit, a subject tilt correction unit, an image region expansion unit, a noise remover, an image binarization unit, and the like. Detailed operation thereof will be described in FIG. 4.

When the preprocessing of the video screen is finished, the preprocessed video screen is input to the character recognition unit 123. Then, the text recognition unit 123 recognizes text images in the preprocessed video screen and converts text data. Herein, the character recognition unit 123 may use a recognizer corresponding to each language. In an embodiment of the present invention, FineReader 5.0 office trial version (company: ABBYY, mainly recognizes English language) may be used for English characters, and GN2000 version (company: HIART corporation, recognizes Korean and English) may be used for Korean characters. . Thereafter, the controller 101 displays the character data recognized by the character recognition unit 123 in the first display area 71 of the display unit 115, and displays item information according to the type of the document input key in the second display area 75 of the display unit 115. do.

Then, when the user selects the recognized text data displayed in the first display area 71 of the display unit 115 and the item displayed in the second display area 75, the controller 101 selects the selected text data and the item in step 230. Is displayed on the third display area 73 of the display unit 115. The storage item is for selecting and storing only desired items from among the items of the recognized document. That is, in the case of a business card, a name, a mobile phone number, an e-mail address, a company address, a company phone number, a facsimile number, and the like have many items. In this case, the user may select and store desired items among the above items.

Thereafter, if a correcting key is input, the controller 101 proceeds to step 240 and corrects an error character among the recognized character data. In this case, the correction method displays candidate character groups for the misrecognized character, and if selected from the candidate characters, the controller 101 replaces the misrecognized character with the selected candidate character. However, if the misrecognized character cannot be corrected among the candidate characters, the user inputs a character to be corrected through the input unit 113 as a cursive handwriting, and the controller 101 drives the character recognition unit 157 to recognize and correct the handwritten character. In addition, a soft keypad is provided in addition to the handwriting recognition module, and a method of correcting an error character by analyzing soft key data generated from the soft keypad is also possible.

Here, the storage item selection process of step 230 and the error correction process of step 240 may have the same effect even if the order is changed.

When the modification is completed as described above, the controller 101 stores the modified character data in the database 131 as the character data of the corresponding item.

FIG. 3 is a diagram illustrating a document photographing procedure performed in step 210 of FIG. 2.

Referring to FIG. 3, the user places a document to be recognized at an appropriate position and starts photographing using the camera 107 of the terminal device. Then, the video screen shot by the camera 107 is processed by the image processor 109 and displayed on the display 115. In this case, when the user of the terminal device inputs a camera adjustment key of the key input unit 105 (also possible in the input unit 113), the control unit 101 detects this in step 313 and controls the camera 107 in step 315. In this case, adjustment of the camera 107 may be distance and exposure adjustment. The distance adjustment may be a method of adjusting a distance between a subject and a terminal device by performing a zoom function, or a method of moving a terminal device by a user. In addition, the exposure adjustment may use a method of controlling exposure of an image sensor in the camera 107. The adjustment operation as described above may be omitted or only one adjustment method may be used. The document capturing method may be a method of photographing the entire document or a method of photographing a desired part of the document. 26A and 26B show an example in which a document to be photographed is a business card and a part of the business card is photographed.

As described above, the document image photographed according to the distance and exposure adjustment of the camera 107 is displayed on the display unit 115 as shown in FIG. 26A. When the user presses the photographing key of the input unit 113 using the stylus pen (or inputs the photographing key of the key input unit 105) in the above state, the control unit 101 detects this by photographing in step 317 and the photographing key. In step 319, the document image at the time of inputting is displayed on the display unit 115 as a still image as shown in FIG. 26C. When the document image displayed on the display unit 115 is satisfactory as shown in FIG. 26C, the user presses a storage key displayed on the input unit 113 using the stylus pen. When the storage key is generated, the controller 101 detects it in step 321 and stores the displayed document image in the image memory area of the memory 103 with a name. At this time, the display unit 115 performs the display operation as shown in FIGS. 26C to 26E during steps 321 and 323. However, if the user generates a cancel key, the controller 101 detects this in step 325 and stops and ends the display operation of the displayed document image.

As described above, in step 210 of capturing the document, a user inputs a desired image through a camera, obtains a clear image screen by increasing the resolution of the input image through camera fine adjustment, and stores it for character recognition. After that, the character image is extracted from the input image through text recognition to check whether the text is to be stored as text data or just as a picture.

Although a procedure of capturing and acquiring an image of a document to be recognized is described herein, a document image being stored or a document image input from the outside may be acquired and performed. In this case, when the user of the portable terminal device requires recognition, the preprocessing operation is performed in step 220 of FIG.

FIG. 4 is a diagram illustrating a configuration of the preprocessor 121 in FIG. 1.

Referring to FIG. 4, the input signal may be a signal of an image screen, and may be an image signal generated from a communication interface unit including a camera, a scanner, a modem, a network, and the like. In addition, the input video screen may be a video signal stored in the memory 103.

A decision on blurring of image part 910 classifies the input image screen into a letter block and a background screen. The average energy ratio of the classified letter blocks is calculated, and the average energy ratio of the letter blocks is compared with a preset reference value to determine whether the image screen is blurred. At this time, if the video screen is determined to be a blurred image screen, the controller 101 notifies this to request re-entry of the video screen, and if the video screen is not a non-blurred image, the input video screen is displayed. It is transmitted to the subject tilt correction unit 920. Therefore, the controller 101 controls to generate the video screen again or processes the video screen in the preprocessor 121 according to the blurring determination output from the image blurring determiner 910.

The subject skew correction part 920 first divides the input image screen into blocks having a predetermined size, and then classifies the divided blocks into letter blocks and background blocks, and divides the pixels of the classified blocks. Binarize. Second, by performing a dilation on the area of the binarized letter block, a candidate stripe in which neighboring letters are in contact with each other is generated. Third, candidate stripes having a length greater than or equal to a predetermined size among the candidate stripes are classified into stripes. Third, the direction angles of the classified stripes are respectively calculated and the number is accumulated, and the direction angle with the largest accumulated number is selected to determine the rotation angle of the subject in the tilted image screen. Fourth, the image signal output from the input unit 10 is input, and the image signal is rotated by the determined rotation angle to correct the inclination of the subject in the image screen. Fifth, the image screen having the same size as the input image screen is generated by filling certain pixels in the blank area of the image screen where the pixels are absent due to the tilt correction.

The ROC (Region of contents) extension part (930) first divides the image screen generated by the subject tilt correction unit 920 into the block, examines the pixels included in the divided blocks, and checks the character block and After classifying the background blocks, the pixels of the classified letter blocks are binarized. Second, the median filtering of the binarized video screen is performed to remove a text area that is incorrectly classified by a border or noise in the video screen. Third, the position of the text area is searched by scanning the median filtered video screen in horizontal and vertical directions. Fourthly, the image screen of the text area where the position is found is extracted. Fifth, the image screen of the extracted text area is expanded to the size of the input image screen.

The noise reduction part 30 performs a function of removing noise included in an image screen output from the image area expansion unit 930.

In general, noise is generated when an image is obtained from a camera. Among the noise components, a representative noise component is Gaussian noise. In order to remove the Gaussian noise, various kinds of noise removing filters may be used. However, in the case of an image screen photographing a business card or the like, a lot of information is provided at the edge of the text area. Therefore, in the case of an image screen such as a business card, only a simple noise reduction filter may cause serious damage to character information. Therefore, the noise removing unit 940 preferably uses a filter for removing noise in the image while preserving edge information of the letter well. Here, it is assumed that the noise canceling unit 940 uses a special noise canceling filter such as a directional lee filter.

The noise canceller 940 may be positioned between the image blurring determiner 910 and the subject tilt correction unit 920, or may be positioned between the subject tilt correction unit 920 and the image area extension unit 930. May be omitted.

The image binarizer 950 first divides the image screen output from the image region expansion unit 930 or the image screen output from the noise canceller 940 into blocks having a predetermined size, and examines pixels included in the divided blocks. Classify into blocks and background blocks. By using the relationship between the text pixels of the second classified letter blocks and the neighboring pixels, pixels with improved edges and reduced noise of the letter blocks are generated, and a reference value for binarizing the pixels is calculated. Edge enhancement and noise reduction of the video signal may use a quadratic filter or an improved quadratic filter. Third, pixels of the letter block and the background blocks from which edge enhancement and noise are removed are compared with the reference value and binarized to the first and second brightness values.

The binarized image screen information output from the image binarization unit 950 is applied to a character recognition unit 123 to recognize characters in the image screen.

The image blurring determiner 910, the subject tilt correction unit 920, the image region expansion unit 930, the noise removing unit 940 and the image binarization unit 950 of the preprocessor 121 according to the embodiment of the present invention can be implemented as follows. .

5 is a view for explaining the configuration of the image blurring determiner 910, FIG. 9 is a view for explaining the configuration of the subject tilt correction unit 920, and FIG. 14 is the image area expansion unit 930. 17A to 18D are views for explaining the configuration of the noise canceling unit 940, and FIG. 12 is a view for explaining the configuration of the image binarization unit 950. FIG.

FIG. 5 is a diagram illustrating a configuration of the image blurring determiner 910 of FIG. 4.

Referring to FIG. 5, a block classification part 1110 divides the input image screen into the block, examines pixels included in the divided blocks, and then selects a character block (CB); Performs a classification of background blocks (BBs). The reason why the block classification unit 1110 classifies the blocks into the letter blocks and the background blocks as described above is to determine whether to blur using only the area in which the letters are included. It is assumed here that the block has a size of 8x8 pixels.

The letter block average energy calculator 1120 calculates an average energy ratio of the letter blocks output from the block classification unit 1110. The reason for calculating the average energy ratio of the letter blocks as described above is to determine whether blurring is performed using only the areas containing the letters by calculating the average energy ratio of the letter blocks constituting the image screen.

The blurring determination unit 1130 determines whether the image screen is blurred by comparing the average energy ratio of the letter block output from the letter block average energy calculator 1120 with a preset reference value. When it is determined that the video screen is a blurred image screen, the blurring unit 1130 notifies the input unit 10 to request re-entry of the video screen.

6 is a diagram illustrating a configuration of the block classification unit 1110. The block classification unit 1110 divides the image screen into blocks having a predetermined size, and classifies each block into a letter block and a background block, respectively. In this case, the purpose of classifying the blocks by the block classifier 1110 is to perform only the region including the text when determining whether the image screen is blurred.

Referring to FIG. 6, the block dividing unit 1111 divides the video screen into blocks having a predetermined size. In this case, if the video screen is 640 × 480 pixels and the block is 8 × 8 pixels, the block divider 1111 divides the video screen into 4800 blocks.

The blocks output from the block division unit 1111 are applied to the DCT transform unit 1113 to perform a discrete cosine transform (DCT) transformation. The energy calculator 1115 calculates a sum of absolute values of dominant DCT coefficients in the DCT transformed block. At this time, the energy distribution of the DCT coefficients of the letter block has a larger value than that of the background block. As described above, the DCT coefficients of the letter block have a larger value than the DCT coefficients of the background block, and it can be seen that some DCT coefficients have a larger average of the sum of absolute values. Therefore, in the embodiment of the present invention, the superior DCT coefficients used in the block classification are D ₁ to D ₉ . Therefore, the sum of absolute values of excellent DCT coefficients in the k-th block may be calculated as in Equation 1 below.

In Equation 1 Denotes the i-th excellent DCT coefficient of the k-th block, and S ^k denotes the sum of absolute values of the DCT coefficients of the k-th block. Therefore, the embodiment of the present invention calculates the sum of absolute values of DCT coefficients up to D ₁ -D ₉ which are excellent DCT coefficients.

The energy calculator 1115 performs the same calculation as for Equation 1 on all blocks k = 0,1,2, ... 4799. The energy values S ^k (k = 0,1, ..., 4799) for each block are applied to the block reference value calculator 1117.

The block reference value calculation unit 1117 adds the energy values S ^k (k = 0,1, ..., 4799) calculated for each block, and then adds the energy values of all the added blocks to the total number of blocks (TBN). Divide by) to get the average value <S ^k >. In this case, the value of <S ^k > is obtained as shown in Equation 2 below, and the value of <S ^k > is a block reference value Cth for determining the block as a letter block or a background block.

In Equation 2, TBN represents the total number of blocks.

The block determiner 1119 sequentially inputs energy values for each block (sum of absolute values of excellent DCT coefficients) output from the energy calculator 1115, and compares the input block energy value with the reference value Cth to form a letter block or a background. Determine with block. In this case, as shown in Equation 3, the block decision unit 1119 determines that the k-th block is a letter block if the S ^k value is greater than or equal to the block reference value Cth, and if the block value is less than the reference value Cth, The k-th block is determined as the background block.

As described above, the pixels of the blocks classified by the block classification unit 1110 may have gray levels of 0-255. The image of the letter block output from the block classifier 1110 is input to the letter block average energy calculator 1120. The letter block average energy calculator 1120 calculates an energy ratio of each classified letter block, and then calculates an average energy ratio of the letter block on the entire image screen using the same. 7 is a diagram illustrating a configuration of the letter block energy calculation unit 1120.

Referring to FIG. 7, the energy ratio calculation unit 1121 calculates the energy ratio of the DCT coefficient in each letter block classified by each block classification unit 1110. In this case, the ratio of the letter block DCT coefficient in the block of M × M pixel size may be obtained as in Equation 4 below.

In Equation 4 above

DCT coefficient of low frequency component at (m, n) of kth block

DCT coefficient of high frequency component at (m, n) of kth block

In the embodiment of the present invention, as described above, the block is assumed to be 8x8 pixels (M = 8). Here, experiments are conducted to verify the validity of the position selection of the DCT coefficients of the low frequency and high frequency components used to calculate the energy ratio of the letter block, and the positions of the DCT coefficients for calculating the DCT energy ratio in each letter block are obtained. . At this time, the experiment confirms the change of the average energy ratio value of the letter block while increasing the degree of image blurring. Among the DCT coefficients for calculating the energy ratio of the DCT coefficients of each letter block, L _{m, n} becomes the DCT coefficient of the low frequency component at the position of m + n = 1,2 according to the experimental results as described above. _{m, n} becomes DCT coefficient of the high frequency component in the position of m + n = 3,4,5,6.

As described above, the energy ratio calculating unit 1121 obtains the energy ratio R ^k of the DCT coefficients for the respective letter blocks in Equation 4. The average energy ratio calculator 1123 calculates the average energy ratio <R ^k > of the DCT coefficients in the entire image. That is, the average energy ratio calculator 1123 calculates an average R ^k of the entire image using the R ^{k values} obtained by the energy ratio calculator 1121 as shown in Equation 5 below.

In Equation 5, TCN means the total number of letter blocks.

When the average energy ratio <R ^k > of the entire image is calculated as described above, the blurring determination unit 1130 inputs the average energy ratio <R ^k > by comparing with the reference value Bth experimentally obtained as shown in Equation 6 below. It is determined whether the captured video screen is blurred. That is, the blurring determination unit 1130 determines that the input image screen is blurred when the average energy ratio <R ^k > is greater than or equal to the reference value Bth, and requests the input unit 10 to re-enter the image screen. However, when the average energy ratio <R ^k > is smaller than the reference value Bth, the average energy ratio <R ^k > may be applied to the noise removing unit 940 or the image binarization unit 950 to recognize the input image screen.

In this case, the reference value Bth is experimentally selected based on whether visual recognition of character information of an image screen is possible and the performance of the binarization output result of the image screen.

8 is a diagram illustrating a procedure of determining whether an input video screen is blurred according to an embodiment of the present invention.

Referring to FIG. 8, first, an image screen is input in step 1151. In this case, it is assumed that the input video screen has a size of 640 × 480 pixels. In operation 1153, the image screen is divided into a set block size. The block has an 8x8 pixel size and 4800 blocks are generated. After that, in step 1155, the respective blocks are DCT transformed, and in step 1157, the sum of absolute values of excellent DCT coefficients of each of the DCT-converted blocks, as shown in Equation 1, S ^k (k = BN = 0,. Calculate .47,4799 and output the energy of each block. Then, in step 1159, as shown in Equation 2, the sum of the absolute values of the excellent DCT coefficients of each block is added and averaged to calculate a block reference value Cth (= <S ^k >). The block reference value Cth is an average of sums of absolute values of excellent DCT coefficients of each block of the entire video screen, and is a block reference value for classifying each block into a letter block and a background block. Thereafter, in step 1161, while sequentially accessing the sum (S ^k ) of the absolute values of the excellent DCT coefficients of the blocks, the value (S ^k ) is compared with the block reference value as shown in Equation (3), and the comparison result If the value is greater than or equal to the block reference value, it is classified as a letter block. If the value is smaller, it is classified as a background block in step 1163. In operation 1165, the energy ratio R ^k of the DCT coefficient is calculated for the blocks classified into the letter blocks as shown in Equation 4, and in step 1167, the DCT coefficients of the calculated letter blocks are calculated. Calculate the average energy ratio <R ^k > of the letter block in the entire image by adding and averaging the energy ratios of. In step 1169, as shown in Equation 6, the average energy ratio <R ^k > of the letter block is compared with the reference value Bth for the blurring determination to determine whether the blurring occurs. At this time, if the average energy ratio <R ^k > is greater than or equal to the reference value Bth, the letter block determines the input image screen as a blurred image and returns to step 510. However, if the letter block has an average energy ratio <R ^k > less than the reference value Bth, the input image screen is determined as a non-blurred image screen, and the character recognition unit 123 is notified. Then, the character recognition unit 123 recognizes the characters included in the pre-processed image screen output from the preprocessor 121.

FIG. 9 is a diagram illustrating a configuration of the subject tilt correction unit 920 of FIG. 4.

Referring to FIG. 9, a binarization part 1210 divides the input image screen into the blocks, examines the pixels included in the divided blocks, and classifies them into letter blocks and background blocks. The function of binarizing the pixels of each block is performed. The reason why the binarization unit 1210 classifies each block into a letter block and a background block is to binarize the character string by using the binarized area after the character is included.

The horizontal pixel reduction unit 1220 reduces the horizontal pixels of the video screen by performing subsampling in the horizontal direction on the binarized video screen. The reason for reducing the horizontal pixels in the horizontal pixel reduction unit 1220 is that the character strings can be formed as a stripe that is well grouped in the horizontal direction when generating the candidate stripe described later.

The candidate stripe generation unit 1230 generates a candidate stripe in which neighboring letters come into contact with each other by performing a dilation on the area of the letter block. The candidate stripe generation unit 1230 generates candidate stripe that the neighboring letters encounter by performing a dilation on the regions of the binarized letter blocks, and the candidate stripe adjacent to each other in the expansion step. Perform an erosion operation to prevent them from sticking together.

The vertical pixel reduction unit 1240 reduces the vertical pixels by performing subsampling in the vertical direction on the video screen converted into the candidate stripe at the reduction ratio of the horizontal pixels. The vertical pixel reduction unit 1240 restores the ratio of the image screen changed according to the horizontal pixel reduction of the horizontal pixel reduction unit 1220 to the ratio of the original image screen. The vertical pixel reduction unit 1240 may perform the same function even if the horizontal pixel is increased.

The stripe classification unit 1250 classifies stripes having a length greater than or equal to a predetermined size among the candidate stripes whose vertical pixels are reduced. The stripe classification unit 1250 classifies stripes having a predetermined size or more by calculating a blob size and / or eccentricity using the moments of the binarized candidate stripes. The stripes are used as a target signal for calculating a direction angle of a subject in the image screen inclined with respect to the horizontal axis of the image. That is, the stripe classification unit 1250 performs a function of classifying stripes to obtain a steering angle by using stripes of the binary letters.

The rotation angle determiner 1260 calculates the direction angles of the classified stripes and accumulates the number of direction angles, and selects the direction angle with the largest cumulative number to determine the rotation angle of the subject in the tilted image screen. The rotation angle determiner 1260 calculates the direction angles of the stripes, and determines the direction angle having the largest number as the rotation angle by accumulating the number of the calculated results.

The tilt correction unit 1270 inputs an image signal output from the input unit 10, and rotates the image signal by the rotation angle of the rotation angle determiner 1260 to correct the tilt of the subject in the image screen.

The image compensator 1280 inserts a video signal into a corner of the video screen in which the inclination of the subject in the video screen is corrected. That is, when the tilt correction unit 1270 corrects the tilt of the subject in the video screen, an area without pixels is generated according to the rotation of the video screen. The image correction unit 1280 fills specific pixels in a blank area of the image screen in which no pixels are caused by the tilt correction. In this case, the pixels to be filled are data irrelevant to the text, and thus outputting the tilt correction 1270 as it is does not affect the recognition of the text on the image screen.

The operation of the subject tilt correction unit 920 having the configuration as shown in FIG. 9 will be described in detail.

First, the input video screen has a size of N × M. In addition, the input image may be a color image or a gray image without color information. In the embodiment of the present invention, it is assumed that the video screen is a black and white video.

The image screen is inputted to the binarization unit 1210, divided into blocks, classified into a letter block and a background block, and binarized.

10 is a diagram illustrating a configuration of the binarization unit 1210. The binarization unit 1210 divides the input image screen into blocks having a predetermined size, classifies each block into a letter block and a background block, and binarizes the pixels of the classified block images into letter pixels and background pixels. In this case, after the binarization unit 1210 classifies the letter block and the background block, the purpose of binarizing the block image pixels is to obtain the rotation angle of the subject in the image screen by obtaining the direction angle of the character strings when correcting the tilt of the subject in the image screen. For sake. Referring to FIG. 10, the block classification unit 1211 divides the input image screen into a set block size, and classifies the divided blocks into letter blocks and background blocks. Then, the block grouping unit 1213 groups the classified letter blocks with eight adjacent blocks, and the reference value calculator generates a reference value from the grouped blocks. Then, the pixel determiner 1217 converts the pixels of the background block output from the block classification unit 1211 into background pixels having a second brightness value by using the reference value output from the reference value calculator, and the pixels of the letter block Based on the reference value, the output signal is binarized to a character pixel having a first brightness value and a background pixel having a second brightness value.

FIG. 11 is a diagram illustrating a detailed configuration of the block classification unit 1211 in FIG. 10. The block classification unit 1211 may be configured in the same manner as the block classification unit 1110 of the image blurring determiner 910. Therefore, the same block classification unit 1211 as shown in FIG. 11 has the same configuration as the block classification unit 1110 as shown in FIG. 6, and the operation of classifying blocks in the image screen is the same as the operation of the block classification unit 1110.

As described above, the pixels of the letter blocks classified in the block classification unit 1211 may have a gray level of 0-255. The image of the letter block output from the block classification unit 1211 is input to the block grouping unit 1213 and the pixel determiner 1217.

The sorted blocks output from the block classification unit 1211 are applied to the block grouping unit 1213. In this case, since the binarization unit 1210 classifies the character strings of the image screen, the background blocks are collectively converted into the background pixels having a predetermined brightness value. Therefore, it is assumed that the block grouping and the reference value calculation operation are not performed on the background block.

The block grouping unit 1213 creates a grouped block by grouping eight adjacent blocks around the letter block output from the block classification unit 1211. The letter block has a size of 8 × 8 pixels, and when the binarization process is performed by setting a reference value for distinguishing a background pixel and a text pixel using only one letter block of this size, the block size is too small to determine the size of the adjacent letter block. Since the difference between the reference value and the value is large, discontinuity between blocks may occur in the binarized image. Therefore, the reliability of binarization can be improved by extending the area for performing binarization by generating the grouped blocks as described above.

The pixel reference value calculator 1215 calculates a pixel reference value Pth for classifying each pixel of the letter block into a letter pixel and a background pixel. The pixel reference value calculator 1215 generates the pixel reference value Pth, and the pixel reference value Pth is used as a pixel reference value during binarization by dividing a character pixel and a background pixel. In this case, the pixel reference value Pth is another method such as an Otsu method or a Kapur method that selects a gray value in which the difference between two classes of pixels is maximum. Can be selected using. It is assumed that the pixel reference value Pth is calculated using the Otsu method. The method of calculating the pixel reference value Pth by the Otsu method can be obtained by Equation 7 below, which is a paper published by N. Otsu "A Threshold Selection Method from Gray-Level Histogram" [IEEE Trans. on Systems Man and Cybernetics, Vol. SMC-9, no. 1, pp. 62-66, Jan. 1979.].

Then, the pixel panel unit 1217 binarizes each pixel of the letter block output from the block classification unit 1211 into a background pixel and a letter pixel using the pixel reference value, and binarizes each pixel of the background block into a background pixel. . That is, when the letter block image is input, the pixel panel unit 1217 compares the corresponding pixel reference value Pth with the pixels of the letter block, and if the image pixel value is greater than or equal to the pixel reference value Pth, it is classified as a character pixel. If it is small, it is classified as a background pixel. According to the classified result, the pixel panel unit 1217 converts the font pixel into an alpha brightness value and converts the background pixel into a beta brightness value and binarizes it. A method of binarizing the pixels of the letter block in the pixel plate unit 1217 is shown in Equation 8 below.

In Equation (8), y (m, n) is the image pixels of the letter block output from the block classification unit 1211, Pth is the pixel reference value, and y _B (m, n) is a pixel of the binarized letter block. admit.

In addition, the pixel determiner 1217 receives the pixels of the background block output from the block classification unit 1211 and collectively converts them into β brightness values.

The image screen binarized by the binarization unit 1210 may be input to the candidate stripe generation unit 1230 or the horizontal pixel reduction unit 1220. In this case, it is assumed that the horizontal pixel reduction unit 1220 is input.

The horizontal pixel reduction unit 1220 performs subsampling with respect to the binarized image at a ratio set in the horizontal direction. In this case, when the subsampling ratio is 2: 1, the horizontal pixel reduction unit 1220 subsamples the binarized video signal in a horizontal direction of 2: 1 to reduce the number of pixels in the horizontal direction by 1/2. do. The purpose of reducing the horizontal pixels as described above is to allow the strings to be well formed in a stripe form in the candidate stripe generation unit 1230 at the rear end.

The candidate stripe generation unit 1230 inputs a binarized image screen output from the binarization unit 1210 or a binarized image screen from which the horizontal pixel output from the horizontal pixel reduction unit 1220 is reduced. The candidate stripe generating unit 1230 stripes each character string formed of letters on the received video screen. The candidate stripping unit 1230 may be implemented as a morphological filter (morphological filter) composed of a dilation part and an erosion part. The morphological filter expands the letter area and then contracts the letters to abut each other. That is, the expander expands the binarized character areas to make contact with neighboring letters, thereby generating character strings in which letters are in contact with each other. Herein, the generated character string will be referred to as a candidate stripe. And the systolic contracts the generated candidate stripe. This is to cause the candidate stripe to fall off when the candidate stripe sticks to adjacent upper and lower candidate stripe in the expansion process. Such morphological filters are described in the book "Digital Image Processing" [2nd ed., Prentice Hall, pp. 519-560, 2002.] published by RCGonzalez and R. Woods et al. .

The vertical pixel reduction unit 1240 performs subsampling at a ratio set in the vertical direction with respect to the image output from the candidate stripping unit 1230. In this case, it is assumed that the subsampling ratio is 2: 1 as in the horizontal pixel reduction unit 1220. Then, the vertical pixel reduction unit 1240 is used to reduce the aspect ratio of the video screen converted by the horizontal pixel reduction to the ratio of the video screen. Accordingly, the video screen output from the vertical pixel reduction unit 1240 outputs the video screens each reduced by 1/2 of the size of the video screen. In this case, a horizontal pixel extension unit may be used instead of the vertical pixel reduction unit 1240. Then it can be reduced to the size of the original image screen.

The stripe classification unit 1250 may input a binarized image screen output from the binarization unit 1210, an image screen generated from the candidate stripe generation unit 1230, or an image screen output from the vertical pixel reduction unit 1240. In this case, it is assumed that an image screen output from the vertical pixel reduction unit 1240 is input.

The stripe classification unit 1250 numbers a candidate stripe generated from the binarized image. In this case, the numbered candidate stripes are candidate stripes for calculating a direction angle. Thereafter, the stripe classification unit 1250 examines the stripe shape of the numbered candidate stripe and classifies the candidate stripe having a certain size or more and having an elongated shape. In this case, the candidate stripe classification method uses a blob size and an eccentricity using moments. Equation 9 below shows the definition of the center moment used to calculate the blob size and eccentricity, and the size of the blob is obtained when p = 0 and q = 0 in Equation 9 below. In addition, Equation 10 shows a method of calculating the eccentricity using the central moment. This is described in the book "Digital Image Processing Algorithms" published by Peters (Prentice Hall, pp. 326-331, 1993.).

In this case, the eccentricity e indicates how long the candidate stripe has.

The candidate stripe is selected as a stripe by comparing the blob sizes μ (= μ ₀₀ ) and the eccentricity e obtained in Equations 9 and 10 with the preset reference values μth and eth, respectively. Here, the reference values μth and eth are obtained experimentally, and candidate stripes corresponding to the case of μ≥μth and / or e≥eth are classified into stripes. In an embodiment of the present invention, however, candidate strips corresponding to the case where the blob size μ and the eccentricity e are μ≥μth and e≥eth are classified into stripes, and any one of the blob sizes μ and the eccentricity e is the reference value μth and If less than eth or less than both, the candidate stripe is assumed not to be selected as a stripe. That is, in this case, the candidate stripe is not selected as the stripe because it is determined that the stripe is not suitable for calculating the direction angle. In the exemplary embodiment of the present invention, the candidate stripe that satisfies both the blob size μ and the eccentricity e is selected as a stripe. However, only one of the two conditions may be examined to determine whether the candidate stripe is striped. have.

Stripes classified by the stripe classification unit 1250 are input to the rotation angle determination unit 1260, and the rotation angle determination unit 1260 calculates a direction angle θ for each of the classified stripes. Accumulate and store the number of direction angles. The direction angle having the largest number among the direction angles in which the number is stored is determined as the rotation angle. 12 is a diagram for describing a procedure of calculating a rotation angle of a stripe by the rotation angle determiner 1260. In FIG. 12, SP is a stripe classified by the stripe classification unit 1250, and x 'and y' axes are coordinate axes where the stripe is located. Therefore, for each of the stripes output from the stripe classification unit 1250, the direction angle θ between the x 'axis and the real X axis of the stripe is calculated, and the number of direction angles θ obtained for each stripe is accumulated and stored. do. At this time, the direction angle θ for the stripe SP can be obtained as shown in Equation 11 below.

After the calculation of the direction angle θ for all the stripes is completed, the rotation angle determiner 1260 determines the direction angle θ having the largest number as the rotation angle by checking the accumulated number of direction angles θ. That is, the rotation angle determination unit 1260 determines the direction angle θ having the largest number as the rotation angle. That is, the rotation angle determiner 1260 determines the direction angle θ of the most stripes as the rotation angle.

When the rotation angle is determined, the tilt correction unit 1270 rotates the image screen output to the input unit 10 at the rotation angle determined by the rotation angle determiner 1260 to correct the tilt of the image signal. That is, the tilt correction unit 1270 rotates the image screen by the rotation matrix when the rotation angle is determined. At this time, the rotation of the video screen may use a rotation method using an inverse mapping method. For a description of the reverse mapping and rotation method, see the book "Handbook of Computer Vision and Applications" published by B.Jahne et al. [Academic Press, vol 2, pp. 94-95, 1999 and "Computer Vision" [Prentice Hall, pp. 415-418, 2001.] published by L. G. Shapiro and G.C.Stockman, respectively.

As described above, when the tilt correction unit 1270 rotates the video screen, a blank without pixels appears in a corner of the video screen. This gap can affect subsequent recognition processes. The image correcting unit 1280 performs a function of filling specific pixels in the blank area generated at the corner of the image screen in which the tilt correction unit 1270 is corrected. In this case, when the pixel is filled in the blank at the corner of the corrected image screen, the image correction unit 1280 may fill the blank area with the value of the pixel closest to the blank area in the horizontal direction. In addition, the blank area may be collectively corrected with the brightness value of the background pixel used for binarization.

When the letters of the image screen are recognized as described above, after extracting stripes by the character string of the input image screen, the direction angles according to the slope of the stripes are calculated, and the most direction angles among the calculated direction angles are calculated. After determining the rotation angle, the image screen is rotated according to the determined rotation angle. Therefore, a video screen can be made by correcting an inclination of a subject of an input video screen. In addition, when the tilt of the subject of the image screen is corrected, corner blanks without pixels may be filled with specific pixel brightness values, thereby reducing errors in recognition.

The tilt correction procedure of the subject in the image screen according to the embodiment of the present invention as described above will be described with reference to FIG. 13.

First, the image screen is input in step 1310. In operation 1315, the input image screen is binarized. In this case, the binarization process first divides the received video screen into a predetermined block size, and classifies the divided blocks into letter blocks and background blocks, respectively. In addition, eight adjacent blocks are grouped around the letter block to form a grouped block, and a pixel reference value for classifying pixels of the block image into letter and background pixels is generated from the grouped block. Subsequently, pixels of the classified letter block are classified into a character pixel and a background pixel by comparing with the pixel reference value, and the pixels of the background block are collectively converted into a background pixel. Therefore, in operation 1315, the pixels of the input image screen are binarized and output to the character pixels and the background pixels.

The binarized video screen is subsampled in a horizontal direction in step 1320. The subsampling is performed in the horizontal direction. The reason for subsampling the pixels in the horizontal direction is to cause the character strings to be combined in a stripe shape in the process of generating a candidate stripe at the rear end. Thereafter, in steps 1325 and 1330, candidate stripes are generated by morphologically filtering the horizontally reduced video image. That is, in step 1325, the binary character areas of the video screen are expanded to make neighboring letters contact each other, and in step 1330, the candidate stripe is attached to another candidate stripe vertically adjacent to each other in the expansion process. Make it fall After performing the morphological filtering operation as described above, in step 1335, the vertical pixels of the video screen are subsampled to be reduced to the ratio of the original video screen. The purpose of subsampling the pixels in the vertical direction of the morphologically filtered binarization image is to obtain the inclination angle of the character string in the reduced image.

Thereafter, in step 1340, a number of candidate stripes is assigned to the image screen, and in step 1345, stripes for calculating a direction angle are calculated by calculating the eccentricity and the blob size of each candidate stripe. In operation 1350, the direction angles of the selected stripes are calculated and the numbers are accumulated. When the calculation of the direction angles with respect to the selected stripes is completed, in operation 1355, the direction angle at which the number is accumulated is the highest among the accumulated direction angles as the skew angle of the image screen.

When the rotation angle is determined, the tilt of the subject of the image screen is corrected by rotating the input image screen at the rotation angle in step 1360. The tilted image screen generates a blank portion without pixels at the corners by the rotation of the image screen. To correct this, in step 1365, the pixel value closest to the corner portion of the image screen is selected in the horizontal direction. Fill it up. In operation 1370, the screen on which the tilt and the image pixel are corrected is output to the image region expansion unit 930, the noise removing unit 940, or the image binarization unit 950.

FIG. 14 is a diagram illustrating a configuration of the image region expansion unit 930 of FIG. 4.

Referring to FIG. 14, the input video screen is an input video screen or a video screen output from the subject tilt correction unit 920.

The average filter 1410 causes the image screen to be blurred by mean filtering the input image screen. The reason for performing the average filtering is to blur the input video screen to reduce the influence of the background area outside the character area in the block classification at the rear end.

The block classification unit 1420 divides the image screen output from the average filter 1410 into the block, examines the pixels included in the divided blocks, classifies them into letter blocks and background blocks, and then classifies the classified letter blocks. It converts the pixels to a specific value. The reason why the block classification unit classifies each block into a letter block and a background block is to enable the text area to be extracted by converting a specific pixel value from the area containing the letter. It is assumed here that the block has a size of 8x8 pixels as described above.

The subsampling part 1430 reduces the number of pixels by subsampling an image screen output from the block classifier 1420. The reason for reducing the pixel is to increase the filtering speed by making the filter window smaller when median filtering is performed at the rear end. In the embodiment of the present invention, it is assumed that the pixel reduction ratio is (2: 1) ² . In this case, since the pixel reduction unit 1430 subsamples the horizontal pixels 2: 1 and subsamples the vertical pixels 2: 1, the number of pixels of the output image screen is reduced to 1/4.

The median filter 1440 removes wrongly classified letter blocks of the video screen by median filtering the video screen output from the pixel reduction unit 1430. The median filter 1440 removes isolated letter blocks that are incorrectly classified as letter blocks by noise in the block classification process.

The interpolation part 1450 extends by interpolating pixels of an image screen output from the median filter 1440. In an embodiment of the present invention, it is assumed that the pixel interpolation ratio is (2: 1) ² . In this case, the pixel restoring unit 1450 interpolates the horizontal pixels and the vertical pixels of the video screen output from the median filter 1440 at 2: 1, respectively, so that the size of the output video screen is expanded four times. The reason for restoring the pixel is to extend the size of the video screen reduced in the pixel reduction process to the size of the original video screen in order to search for the correct position of the text area.

The location search unit 1460 scans the median filtered video screen in horizontal and vertical directions to search for a location of a text area. The position search unit 1460 scans the median filtered image screen in a horizontal direction to search for the position (x1) of the leftmost letter block and the position (x2) of the letter block located at the rightmost, and in the vertical direction. After scanning, the position y1 of the uppermost letter block and the position y2 of the lowermost letter block are searched to determine the position of the character area on the image screen according to the search result. At this time, the positions of the upper left and the lower right of the character area are (x1, y1) and (x2, y2). At this time, the positions (x1, y1) and (x2, y2) of the upper left and lower right ends of the character area are determined to have the aspect ratio of the input image screen. This is to prevent distortion when the image is extended by the image expansion unit 170 at the rear end.

The text area extractor 1470 extracts an image screen of the text area searched by the location search unit 1460. That is, the character area extracting unit 1470 inputs the position (x1, y1) and (x2, y2) values of the upper left and lower right ends of the character area output from the position search unit 1460, The image screen existing in the positions (x1, y1) and (x2, y2) of the upper left and lower right is extracted. Accordingly, the video screen output from the text area extractor 1470 becomes a video screen of the text area from which the background area is removed from the input video screen.

The image expansion unit 1480 expands the extracted video screen of the text area to the size of the input video screen. In this case, the image extension may be implemented by interpolation, and in the embodiment of the present invention, it is assumed that the image extension is implemented by a bilinear interpolation method. At this time, the image expansion performs an interpolation operation so as to be the same size as the image screen of the input screen.

Hereinafter, the operation of the image region expansion unit 930 will be described in detail with reference to FIG. 14.

The video screen input first has a size of N × M. In addition, the input image may be a color image or a gray image without color information. In the embodiment of the present invention, it is assumed that the video screen is a black and white video.

The average filter 1410 inputting the video screen causes the video screen to be blurred by average filtering the video screen. This is to reduce the influence of the background area outside the text area of the image screen when classifying the text area in the block classification unit 1420 at the rear end. Such average filters are described in the book "Digital Image Processing" [2nd ed., Prentice Hall, pp. 119-123, 2002.] published by R. C. Gonzalez and R. Woods et al.

The average filtered video screen is applied to the block classifier 1420. The block classification unit 1420 divides the image screen output from the average filter 1410 into the block, examines the pixels included in the divided blocks, classifies them into letter blocks and background blocks, and then classifies the classified letter blocks. Converts the pixels of the pixel to a specific value.

FIG. 15 is a diagram illustrating a configuration of the block classification unit 1420 of FIG. 14. The block classification unit 1420 may be configured in the same manner as the block classification unit 1110 of the image blurring determiner 910. Accordingly, the block classification unit 1420 of FIG. 15 has the same configuration as that of the block classification unit 1110 of FIG. 6, and the operation of classifying blocks on an image screen is the same as that of the block classification unit 1110.

As described above, the pixels of the letter blocks classified in the block decision unit 1119 of FIG. 15 may have a gray level of 0-255. Then, the block filling part 1421 converts the pixels of the letter block classified in the block determining unit 1119 into pixels having a first brightness value, and converts pixels of the background block into pixels having a second brightness value. do. In the embodiment of the present invention, it is assumed that the block pixel correction unit 1421 converts the pixels of the letter block into white pixels and the pixels of the background block into black pixels. Accordingly, the block pixel correction unit 1421 fills blocks classified as letter blocks on the image screen with white pixels and blocks classified as background blocks with black pixels. As described above, the block classification unit 1420 classifies the blocks into letter blocks and background blocks, and then fills them with pixel values having different brightness values to display letter areas of the image screen.

Thereafter, the pixel reduction unit 1430 reduces the number of horizontal and vertical pixels by subsampling the image screen output from the block classification unit 1420. The reason for reducing the pixel is to increase the filtering speed by making the filter window smaller when median filtering is performed in the median filter 1440 at the rear stage. In the embodiment of the present invention, it is assumed that the pixel reduction ratio is (2: 1) ² . In this case, the number of pixels of the image screen output from the block classification unit 1420 is reduced to 1/4. In this case, the size of the reduced video screen has a size of 320 × 240 pixels.

The median filter 1440 removes the background block and the wrongly classified letter block of the video screen by median filtering the video screen output from the pixel reduction unit 1430. The median filter 1440 removes isolated letter blocks that are incorrectly classified as letter blocks by noise in the block classification process. Such median filters are described in the book "Fundamentals of Digital Image Processing" [Prentice Hall, pp. 246-249.] Published by A.K.Jain.

After the median filtering is performed on the video screen, the pixel restoration unit 480 interpolates horizontal and vertical pixels of the video screen output from the median filter 1440 to expand the video screen to the size of the input video screen. . In an embodiment of the present invention, it is assumed that the pixel interpolation ratio is (2: 1) ² . The reason for restoring the pixel is to extend the size of the video screen reduced in the pixel reduction process to the size of the original video screen in order to search for the correct position of the text area.

The location search unit 1460 scans the median filtered video screen in horizontal and vertical directions to search for a location of a text area. The position search unit 1460 scans the median filtered video screen in a horizontal direction, searches for the position of the leftmost letter block (x1) and the position of the rightmost letter block (x2), and retrieves the result value. Save it. Thereafter, the image screen is scanned again in the vertical direction to search for the position y1 of the uppermost letter block and the position y2 of the lowermost letter block, and then store the result value. Subsequently, the positions (x1, y1) and (x2, y2) of the upper left and lower right portions of the character area are determined according to the search result. At this time, the position (x1, y1) and (x2, y2) of the character area is determined to have the aspect ratio of the input image screen. This is to prevent distortion when expanding the image in the image expansion unit 170 at the rear end. In the exemplary embodiment of the present invention, the aspect ratio of the input image screen is 4: 3 (640 pixels: 480 pixels), so that the character area searched by the location searcher 1460 also has an aspect ratio of 4: 3. And the positions (x1, y1) and (x2, y2) of the lower right end.

The text area extractor 1470 extracts an image screen of the text area searched by the location search unit 1460. That is, the text area extractor 1470 inputs the position (x1, y1) and (x2, y2) values of the upper left and lower right ends of the text area output from the location search unit 1460, and the image screen output from the input unit 10. Extracts an image screen existing within the positions (x1, y1) and (x2, y2) of the upper left and lower right ends of the character area. At this time, the letter region extracting unit 450 is positioned between x1 position and x2 position in the horizontal direction and y1 position in the vertical direction by the positions (x1, y1) and (x2, y2) of the upper left and lower right ends of the character area. Pixels existing between y2 positions are extracted as the pixels of the text area. The video screen output from the text area extractor 1470 becomes a video screen of the text area from which the background area is removed from the input video screen.

The image expansion unit 1480 expands the extracted video screen of the text area to the size of the input video screen. The image expansion may be implemented by interpolation, and in the embodiment of the present invention, it is assumed that the image expansion is implemented by a bilinear interpolation method, which is represented by Equation 12 below.

At this time, the image expansion performs an interpolation operation so as to be equal to the size and aspect ratio of the input image screen. Such a bilinear interpolation method is described in the book "Numerical Recipies in C" published by WHPress and SATeukolsky et al. [2nd ed., Cambridge, pp. 123-125, 1988.]. have.

16 is a diagram for describing a procedure of expanding an image area according to an embodiment of the present invention.

Referring to the image expansion procedure based on the procedure of FIG. 16, the image screen is input in step 1510. In operation 1515, the average image is filtered to generate a blurred image. This is to reduce the influence of the background area outside the letter area in the block classification process as described above.

Thereafter, in step 1520, the average filtered video screen is divided into blocks having a predetermined size, the pixels included in the divided blocks are inspected, classified into a letter block and a background block, and then the pixels of the classified letter block. Converts them to specific values. When the block classification operation is performed, the video screen is classified into a letter block and a background block, the classified letter block is converted into a white pixel, and the background block is converted into a black pixel. Therefore, the image screen is filled with white or black pixels according to the classified blocks.

When the video screen is generated in step 1520, the video screen is subsampled in step 1525 to generate a video screen with reduced horizontal and vertical pixels. The reason for reducing the pixel is to increase the filtering speed by making the filter window of the median filtering process of the next process small. After performing the subsampling operation as described above, the median filter of the reduced image screen is performed in step 1530. When the median filtering is performed, isolated letter blocks that are incorrectly classified by the edge or noise of the video screen and remain on the video screen are removed. After the median filtering operation is performed to remove the wrongly classified letter block, the horizontal and vertical pixels of the median filtered video screen are interpolated and expanded to the size of the input video screen in step 1535.

Thereafter, in step 1540, the image screen restored to the original size is scanned in the horizontal and vertical directions to search for the position of the text area. The location search procedure scans the median filtered video screen in a horizontal direction and searches for the position x1 of the leftmost letter block and the position x2 of the rightmost letter block. The image screen is scanned again in the vertical direction to search for the position y1 of the uppermost letter block and the position y2 of the lowermost letter block. Thereafter, the positions (x1, y1) and (x2, y2) of the upper left and lower right portions of the character area are determined on the image screen according to the search result in step 1545, and the positions (x1, y1) and (x2, y2) determine to have the aspect ratio of the input video screen. This is to prevent distortion when expanding the image in the next step.

After the location of the text area is searched as described above, the image screen existing in the text area of the searched text area is extracted from the input image screen in step 1550. That is, the extraction of the text area extracts the video screen existing in the positions (x1, y1) and (x2, y2) of the upper left and lower right ends of the text area on the video screen. At this time, the extracted video screen of the text area is between x1 position and x2 position in the horizontal direction from the input image screen and between y1 position and y2 position in the vertical direction. The video screen of the text area becomes the video screen of the text area from which the background area is removed from the input video screen.

After extracting the video screen of the text area, in step 1555, the image expansion unit 1480 expands the video screen of the text area to the size of the input video screen. In this case, the image expansion may be implemented by interpolation, and in the embodiment of the present invention, it may be implemented by a bilinear interpolation method. The expanded video screen may be output or stored in the recognizer in step 1560 to be used for other purposes.

The operation of the noise canceling unit 940 of FIG. 4 will be described.

In general, when a video screen of a subject is acquired from a digital camera or the like, noise components are included in the video screen to be photographed. Among the noise components described above, a representative noise component may be a Gaussian noise. A method using a filter is generally used to remove the Gaussian noise, and there are various kinds of noise canceling filters for removing the Gaussian noise. However, when photographing a video screen such as a business card, etc., the video screen has a lot of information in the edge portion of the text area. Therefore, when the noise is removed using only a noise reduction filter on a video screen such as a business card, serious damage to character information may occur. Therefore, a special noise reduction filter is needed to remove the noise of the image while preserving the edge information of the character well. In an embodiment of the present invention, it is assumed that a directional Lee filter is used. Formal expression of the directional re-filter is as shown in Equation (13).

In Equation 13, the filter parameter is adaptively adjusted using the signal average and variance of the local region of the image screen. In Equation (13), in the background region of an image screen, as shown in Equation (14), the variance of noise is much larger than the variance of local signal, and thus the output image from which the noise is removed becomes a local mean value. The variance of the local signal is much larger than the variance of noise so that the noise-free output image is the average value of the weighted pixels of the edge directionality to remove edge noise while preserving the edge well.

In the edge region of the image, in order to preserve the components of the edge and remove noise at the same time, as shown in Equation 15 and FIG. 17B, the direction perpendicular to the main edge directions (90 °, 135 °, 0 °, 45 °) ( It is obtained by the sum of the product of the output (y _θ ) of the one-dimensional average filter at 0 °, 45 °, 90 °, 135 ° and the weight (w _θ ) of the directionality for each edge direction.

In Equation 16, referring to FIGS. 17A and 17B, directions perpendicular to each edge direction (0 °, 45 °, 90 °, 135 °) varying from n = 1 to 4 within a 3 × 3 filter window. We show that we perform a one-dimensional mean filter. This removes noise components in each edge region.

The formula for calculating the weight multiplied by the output of the one-dimensional average filter in the directions (0 °, 45 °, 90 °, 135 °) orthogonal to each edge direction (90 °, 135 °, 0 °, 45 °) is given by the following < Equation 18 > The weight w _θ is changed in each edge direction (90 °, 135 °, 0 °, 45 °) while changing from n = 1 to 4 in the 3 × 3 filter window as shown in FIGS. 18A to 18D. After calculating the strength of the edge (D _θ ), as shown in &gt; For a more detailed description, see the article "Adaptive Image Restoration Using Local Statistics and Directional Gradient Information" published by NC Kim [IEE Electronic Letters 4th, Vol. 23, no. 12, pp. 610-611, June. 1987.].

FIG. 19 is a diagram illustrating a configuration of the image binarization unit 950 of FIG. 4.

Referring to FIG. 19, a block classification part 1610 divides the input image screen into the block, classifies the image block into a letter block and a background block by inspecting pixels included in the divided blocks. Perform the function. The reason why the block classification unit 1610 classifies each block as a letter block and a background block is to perform binarization only on an area including a letter. It is assumed here that the block has a size of 8x8 pixels as described above.

The block growing part 1620 extends the letter blocks classified by the block classification part 1610. In the block classification, a block including a letter pixel may be classified as a background block due to a background effect between letters. The reason for growing the letter block is to extend pixels classified as a background block into a letter block.

The block grouping part 1630 inputs the letter blocks output from the block growth unit 1620, and generates eight grouped blocks by grouping adjacent eight blocks around the letter block. If the binarization process is performed by setting the reference value for distinguishing the background pixel from the text pixel using only one letter block (8 × 8), the block size is too small and the difference between the reference value of the adjacent letter block and the value is large. Discontinuities in the liver may occur. Therefore, the reason for grouping the blocks in the block grouping unit 1630 is to improve the binarization reliability of the letter block by extending the letter block area.

The edge enhancement part 1640 generates pixels that improve the edge of the letter block and reduce noise by using the relationship between the letter pixels of the grouped letter blocks and the peripheral pixels output from the block grouping part 1630. A pixel reference value for binarizing the pixels is generated. The edge enhancement unit 1640 may use the quadratic filter or the improved quadratic filter.

The block splitting part 1650 inputs the grouped block output from the edge enhancement unit 1640, and separates and outputs the letter block from the grouped block. That is, the block grouping unit 1630 performs a function of separating only the letter blocks for binarization from the grouped blocks.

The binarization part 1660 binarizes the pixels of the letter blocks separated by the block separator 1650 to the first and second brightness values of the letter pixel and the background pixel by comparing the pixels with the pixel reference value. The pixels of the output background block are binarized to the second brightness value. Prior to sending the binarized image to the character recognition unit 123 of FIG. 1, a compression unit may be added to compress the image to increase the efficiency of the storage space.

The video screen processed by the binarization unit 1660 is input to the character recognition unit 123 of FIG. 1 to recognize letters.

The input video screen is divided into blocks by the block classification unit 1610 and then classified into a letter block and a background block.

20 is a diagram illustrating the configuration of the block classification unit 1610. The block classification unit 1610 may be configured in the same manner as the block classification unit 1110 of the image blurring determiner 910. Therefore, the block classifier 1610 of FIG. 20 has the same configuration as that of the block classifier 1110 of FIG. 6, and the operation of classifying blocks on the image screen is the same as that of the block classifier 1110. As described above, the pixels of the letter blocks classified in the block classification unit 1610 may have gray levels of 0-255.

The block growth unit 1620 grows an area of the classified letter blocks. In this case, a block including a letter pixel is classified as a background block in the block classification unit 1610 under the influence of a background between letters. Cases may occur. The purpose of growing the letter block is to change the background block including the letter pixel to the letter block by expanding the letter block.

The block growth unit 1620 may be implemented using a morphological filter. The morphological filter grows the letter block by closing the letter block by dilation and then erosion. That is, the closing operation fills the hole inside the area. First, the letter block is expanded through expansion, so that isolated background blocks between the letter block and the letter block are converted to the letter block, and the original block size is reduced through the contraction of the closing operation. Is restored. Such morphological filters are described in the book "Digital Image Processing" [2nd ed., Prentice Hall, pp. 519-560, 2002.] published by RCGonzalez and R. Woods et al. . The block growth unit 1620 changes the background block including the letter pixel to the letter block when the block grows.

The block grouping unit 1630 groups eight adjacent blocks around the letter block output from the block growth unit 1620 to generate a grouped block having a size of 24 × 24 pixels. The letter block has a size of 8 × 8 pixels. When the binarization process is performed by setting a reference value for distinguishing a background pixel and a text pixel with only one letter block (8 × 8), the block size is too small to be adjacent. As the difference between the reference value of the letter block and the value is large, discontinuity between blocks may occur in the binarized image. Therefore, the reliability of binarization can be improved by extending the area for performing binarization by generating the grouped blocks as described above. The grouped blocks including the letter blocks output from the block grouping unit 1630 are applied to the edge enhancement unit 1640.

The edge enhancement unit 1640 may use a quadratic filter (QF) or an improved quadratic filter (IQF). Here, an operation of improving edge components using the improved quadratic filter will be described. The improved quadratic filter improves the edges of the normalized letter block after normalizing the letter block as shown in FIG. 21, and also normalizes the reference value calculated from the letter block to binarize pixels of the letter block. Generate the reference value BTH _N.

First, an operation of improving an edge of a letter block using an improved quadratic filter will be described with reference to FIG. 21.

Referring to FIG. 21, first, the first reference value calculator 1621 calculates a first reference value Th1 for classifying each pixel of the letter block into a letter pixel and a background pixel. The first reference value calculator 1621 calculates the first reference value Th1, and the first reference value Th1 is used to normalize two types of pixels distinguished in a next step by dividing a text pixel and a background pixel. In this case, the first reference value Th1 selects a gray value that maximizes the difference between the two classes of pixels (between-class variance). The first reference value Th1 may use an Otsu method or a Kapur method. The method of calculating the first reference value Th1 using the Otsu method can be obtained by Equation 19, which is a paper published by N. Otsu, "A Threshold Selection Method from Gray-Level Histogram." [IEEE Trans. on Systems. Man and Cybernetics, Vol. SMC-9, no. 1, pp. 62-66, Jan. 1979.].

The average value calculator 1623 classifies the pixels of the letter block into a letter pixel and a background pixel based on the first reference value Th1, and then calculates average brightness values of the letter pixel and the background pixel of the letter block. Looking at the mean computation for two classes, first, the pixels of the letter block x (m, n) are referred to as a character pixel (CP) as shown in Equation 20 based on the first reference value Th1. After dividing into and the background pixel (BP), the average brightness value μ ₀ and the average brightness value μ ₁ of the background pixel are calculated as shown in Equation 21.

In Equation 20, x (m, n) denotes a letter block, and Th1 is a reference value for classifying pixels of the letter block into letter pixels and background pixels.

In Equation 21, S _c is the sum of the brightness values of the text pixels, N _c is the number of the text pixels, S _b is the sum of the brightness values of the background pixels, and N _b is the number of the background pixels.

The normalizer 1625 then selects the pixels of the input text block x (m, n) by using the average brightness value μ ₀ of the text pixel output from the average value calculator 1623 and the average brightness value μ ₁ of the background pixel. Normalizes the background pixel to a value close to 1 and a value close to 0. The normalization unit 1625 performs a function of reducing the dynamic range of brightness values of the input letter block image by normalizing pixels of the input letter block x (m, n). The normalizer 1625 normalizes the pixels of the letter block x (m, n) input by Equation 22 below.

x _N (m, n) represents a normalized letter block, μ ₀ represents an average brightness value of a font pixel, and μ ₁ represents an average brightness value of a background pixel.

Thereafter, the normalized letter block x _N (m, n) is quadratically processed by the quadratic processor 1627 to improve edges of the letter block and reduce noise. The quadratic processor 1627 improves an edge and reduces noise by using a relationship with a peripheral pixel of the normalized pixel. FIG. 22 illustrates central and peripheral pixels processed by the quadratic processor 1627, and Equation 23 is a quadratic character block pixel in the quadratic processor 1627 to improve edges and reduce noise. Indicates. The quadratic processor 1627 performs a function of sharpening the edges of the text portion and removing noise by increasing the gray level difference so that the text pixels are darkened and the background pixels are bright.

Accordingly, in order to generate the reference value BTH _N for binarizing pixels of the letter block, the binarization unit 1660 normalizes the first reference value Th1 calculated by the first reference value calculator 1621 in the reference value normalizer 362 to generate a second reference value Th2. . In this case, the second reference value Th2 is used as the pixel reference value BTH _N for binarizing pixels of the letter block in the binarization unit 1660.

The reference value normalization unit 1631 normalizes the first reference value Th1 by the same method as that of the normalization unit 1625. The reference value normalizing unit 1631 generates a second reference value Th2 (reference value BTH _N ) by normalizing the first reference value as shown in Equation (24).

In Equation 24, Th2 is a normalized reference value BTH _N for distinguishing a text pixel and a background pixel in the binarization unit 1660, x _N (m, n) represents a normalized letter block, and μ ₀ represents a letter pixel. The average brightness value is shown, and μ ₁ represents the average brightness value of the background pixel.

As described above, the edge enhancement unit 1640 having the configuration as shown in FIG. 21 reduces the dynamic range by normalizing the text pixels and the background pixels in the received text block (or a grouped block including the text block), and the normalized pixel. The edges of the letter block (or grouping block including the letter block) are improved by quadratic processing of the blocks. In addition, since the letter block (or grouping block including the letter block) output from the quadratic processor 1627 is a normalized block, the first reference value is normalized to generate a reference value BTH _N for binarizing pixels of the letter block.

As described above, the edge enhancement unit 1640 in the image binarizer 950 may be implemented using the improved quadratic filter as shown in FIG. 21. The edge enhancer 1640 uses an improved quadratic filter to binarize a letter block (or a grouping block including the letter block) to solve the problem of black blocks around letters in a binarized image. To perform.

As a grouped letter block output from the edge enhancer 1640, the output of the edge enhancer 1640 is applied to the block separator 1650. The block separator 1650 for inputting a grouped block including the letter block separates and outputs an image of the letter block from the grouped block. This is to restore the grouping of the neighboring blocks of the letter block in the block grouping unit 1630.

The letter block output from the block separator 1650 is input to the binarizer 1660. The binarization unit 1660 receives the reference value output from the edge enhancement unit 1640 to binarize pixels of the letter block. In this case, the letter block inputted to the binarization unit 1660 may be y (m, n) (letter block output from the quadratic filter as shown in FIG. 21) or y _N (m, n) (in the improved quadratic filter as shown in FIG. 21). Output character block). Therefore, the pixel reference value is also BTH or BTH _N.

The binarization unit 1660 classifies each pixel of the received text block into a background pixel and a text pixel using the reference value, and converts the classified text pixel and the background pixels into two brightness values to perform a binarization operation. That is, when the letter block is input, the binarization unit 1660 compares the corresponding reference value with the pixels of the letter block, and classifies the image pixel value as a letter pixel that is greater than or equal to the reference value and as a background pixel when the letter block is smaller. . The binarization unit 1660 converts the font pixels into α brightness values and the background pixels into β brightness values according to the classified results. A method of binarizing pixels of a letter block in the binarization unit 1660 is shown in Equation 25 below.

In Equation 25, y (m, n) and BTH are letter blocks and reference values output from the quadratic filter, and y _N (m, n) and BTH _N are letter blocks output from the improved quadratic filter and The reference value, y _B (m, n) is a binary block.

In addition, the binarization unit 1660 receives a background block image output from the block classification unit 1610 or the block growth unit 1620. The binarization unit 1660 collectively converts the pixels of the background block into β brightness values.

FIG. 23 is a diagram for describing a binarization procedure when the edge enhancement unit 1640 is implemented using an improved quadratic filter.

Referring to FIG. 23, in operation 1711, an image screen is input. Then, in step 1713, the block classification unit 1610 divides the input image screen into the blocks, examines pixels included in the divided blocks, and classifies them into letter blocks and background blocks.

In operation 1715, the block growth unit 1620 expands the letter blocks classified by the block classification unit 1610. This is because a block including a letter pixel may be classified as a background block due to a background effect between letters in the block classification. The reason for growing the letter block is to extend pixels classified as a background block into a letter block. In operation 1717, the block growth unit 1620 sequentially outputs the letter blocks of the grown image screen to the block grouping unit 1630. In this case, the image output to the block grouping unit 1630 may be a letter block. In operation 1719, the block grouping unit 1630 inputs the letter blocks output from the block growth unit 1620, and groups eight adjacent blocks around the letter block to generate grouped blocks.

The grouped block images are input to the edge enhancer 1640. Here, the edge enhancement unit 1640 is an improved quadratic filter as described above. Referring to the operation procedure of the improved quadratic filter, in operation 1721, a first reference value Th1 for classifying each pixel of the letter block into a letter pixel and a background pixel is calculated. The first reference value Th1 may be obtained by the same method as in Equation 19. In operation 1723, the pixels of the letter block are classified into a letter pixel and a background pixel based on the first reference value Th1 while performing calculations as shown in Equation 20 and Equation 21. Calculate average brightness values of text pixel and background pixel. Subsequently, in step 1725, the text pixel is set to a value close to 1 by using the average brightness value μ ₀ of the text pixel to be output and the average brightness value μ ₁ of the background pixel to output the pixels of the letter block x (m, n). Normalize the pixel to a value close to zero. In the normalization process, the pixels of the letter block x (m, n) input by Equation 22 are normalized.

Then, in step 1727, the normalized letter block x _N (m, n) is improved by the improved quadratic processor 1627 to improve the edge of the letter block and reduce noise. The quadratic process performs a calculation process as shown in Equation 23. In operation 1751, the first reference value Th1 is normalized as shown in Equation 25 in the same manner as the normalization process to generate a second reference value Th2 (pixel reference value BTH _N ).

In operation 1733, the improved quadratic processed grouped block is input, and the letter block is separated from the grouped block. That is, the block separating process separates only the letter blocks located at the center of the grouped blocks. In operation 1735, the pixels of the separated letter blocks are compared with the pixel reference value BTH _N and binarized to the first and second brightness values of the letter pixel and the background pixel. The pixels of the background block generated after the block classification process or the block growth process are binarized to a second brightness value.

While repeating the above operation, the binarization operation of the letter block and the background block is performed, and when the binarization operation for all blocks of the image screen is completed, it is detected in step 1737 and the binarized image screen is output in step 1739.

24A and 24B illustrate a procedure of preprocessing, recognizing, and selecting an item of a document image performed in steps 210 to 230 of FIG. 2, and FIGS. 27A to 27B show results processed while performing the above steps. It is a figure which shows. In this case, it is assumed that the document is a business card, and the stored items are assumed to be items that can be stored in the phone book.

Here, the document recognition key for generating a document recognition command when recognizing a document is preferably configured for each type of document frequently used. For example, the text information recorded on the business card is information that can be stored in the phone book of the portable terminal device. The business name, company department, position, name, office telephone number, e-mail address, mobile phone number and the like are recorded on the business card. Therefore, when registering the phone book information in the mobile terminal device, it may be very convenient to create a phone book by text recognition of the information of the corresponding person using a business card. Therefore, when recognizing a text image of a document such as a business card, the controller 101 may preset a table for allocating items of a business card and an area for storing information of these items, and input the corresponding business card recognition key. It is convenient to detect that the document to be recognized is a business card and to automatically display each stored item of the business card so that the item can be selectively selected and registered. Therefore, according to the embodiment of the present invention, the document identification keys according to the types of documents are provided, the table is pre-assigned to the items according to the types of the documents, and when the corresponding document recognition keys are received, the items of the table of the corresponding document. Can be displayed. If you want to recognize a document that is not set in advance, select the document recognition key to manually set each item and process it. In the embodiment of the present invention described below, it is assumed that the document is a business card.

Referring to FIG. 24A, before the business card recognition key is generated in step 413, the controller 101 displays an image of the business card being stored on the display unit 115 as shown in FIG. 26E in step 411. When the user generates the business card recognition key of the input unit 113 in the above state, the control unit 101 detects this in step 413, and in step 415 drives the pre-processing unit 121 to perform the pre-processing operation of the displayed document image. In this case, the preprocessing operation may be performed by the configuration as shown in FIG. In this case, if it is determined that the blood image is in the process of performing the preprocessing, the controller 101 may stop the subsequent procedure and request the selection of a new document image.

If it is determined that the image is not a blood image, the character recognition unit 123 is driven in step 417 to recognize the character images in the preprocessed document image. Then, the character recognition unit 123 converts the image of the business card as shown in FIG. 26E into text data, and the controller 101 displays the converted text data on the display unit 115 as shown in FIG. 27A. The terminal apparatuses should be provided with a plurality of recognizers for character recognition. That is, documents such as business cards may include Korean characters, English letters, numbers, special symbols, Chinese characters, or characters of other languages. Therefore, in the character recognition process, a corresponding recognizer program should be selected according to the type of character to be recognized. In the embodiment of the present invention, it is assumed that the recognized character is an English letter, and it is assumed that the recognizer uses the FineReader 5.0 office trial version (company: ABBYY, mainly recognizes English language).

When the business card image is converted into text data, the controller 101 displays text data of the business card image on the first display area 71 of the display unit 115 and displays an item selection on the third display area 73 in step 419. , Items to be stored are displayed on the second display area 75. In this case, the items displayed on the second display area 75 may be a name, company, job title, company phone number, mobile phone number, home phone number, facsimile number, e-mail address, company address, and the like. As shown in FIG. 27A, the user selects the character data (sentence) of the first display area 71 using the stylus pen and selects a storage item of the second display area 75 as shown in FIG. 27B. In operation 421, the controller 101 detects this, and in operation 423, the controller 101 displays the selected item and the corresponding text data in the third display area 73 of the display unit 115 as illustrated in FIG. 27B. When the confirmation key is generated from the input unit 113, the control unit 101 detects this in step 425 and proceeds to step 427 to register the selected item and the text data of the selected item. When the correction key is generated from the input unit 113, the control unit 101 detects this in step 429 and proceeds to step 431 to perform an error correction process as shown in FIG. 25A. Thereafter, the error data of the modified item is further registered through the checking process of steps 425 and 427. When the completion key is input from the input unit 113, the controller 101 detects this in step 433 and displays the selected items and the corresponding character data in step 435.

FIG. 25A is a diagram illustrating a process of correcting error-prone text data for each selected item according to an embodiment of the present invention.

Referring to FIG. 25A, the error correction procedure performed in step 431 of FIG. 24A is performed. When a correction key is input, the controller 101 detects an error recognized in the third display area 73 of the display unit 115 in step 511 as shown in FIG. 28A. The character data of the item is displayed. When the text data to be corrected in the first display area 71 of the display unit 115 is clicked using the stylus pen in the state shown in FIG. 28A, the controller 101 detects this in step 513 and the text to be modified as shown in FIG. 28B in step 515. Display data.

In the first embodiment of the present invention, when the character data in which the recognition error occurs is corrected, the error may be corrected in the following two ways. That is, when an error recognized character is designated as shown in FIG. 28B, the controller 101 displays candidate characters for correcting the error recognized character in the third display area 73 of the display unit 115 as shown in FIG. 28B. Displays a recognition window for inputting handwritten characters in order to correct an error recognized character in the third display area 75, and displays a soft keypad capable of generating key data for correcting error characters in the fourth display area 77. . Accordingly, the user may use a method of selecting and modifying a desired letter from the candidate characters displayed in the third display area 73 or inputting and modifying a desired letter in the second display area 75 by handwriting. In addition, in addition to the recognition window for inputting the handwritten characters, soft keypads may be displayed, and error characters may be corrected by analyzing key data generated through the soft keypad.

Therefore, when any character among the candidate characters displayed in the third display area 73 is selected by the stylus pen in the state where the error recognized character is displayed as shown in FIG. 28B, the controller 101 detects this in step 517. In step 519, the error recognition character displayed in the first area is modified to the selected candidate character. In addition, if a handwritten character is input to the recognition window of the second display area 75 by the stylus pen in the state where an error recognized character is displayed as shown in FIG. 28B, the controller 101 detects this in step 521, and in step 523. The handwriting character recognizer of the character recognition unit 123 is driven. In step 525, the controller 101 modifies the error-recognized character data into character data recognized by the character recognition unit 123. In addition, when key data are generated through the soft keypad of the fourth display area in a state where an error recognized character is displayed as shown in FIG. 28B, the controller 101 detects this in step 521 and recognizes the character in step 523. Drives the softkey recognition module of sub-123. In step 525, the controller 101 modifies the error-recognized character data into character data recognized by the character recognition unit 123.

In addition, when a delete key is input, the controller detects this in step 527, and deletes the error recognized character selected in step 513 in step 529. If an additional key is input, the controller 101 detects this in step 531 and determines a position to add (insert) text data in step 533. In this case, the insertion may be a position before or after the character selected in step 513. In step 535, the controller 101 adds (inserts) a character to the determined position while performing the procedure of selecting a candidate character or inputting a handwritten character.

If the user selects another character to correct the selected item after performing the operation of selecting a candidate character or correcting an error recognized character as a cursive character, or deleting the selected character or adding a character as described above, the controller 101 Detects this in step 527 and returns to step 515 to repeat the above operation.

If the above operation is repeated, the controller 101 modifies the error-recognized text data of the selected item. Thereafter, when the corrected key is input, the controller 101 detects this in step 529 and ends the error correction procedure of the selected item, and returns to step 421 of FIG. 24A.

28A and 28B illustrate operations performed through candidate characters and handwritten character recognition when character data is corrected. However, error recognition characters can be corrected only by handwritten character recognition without using candidate characters. FIG. 28D illustrates a method of correcting by inputting a handwritten character or a soft key without using candidate characters when modifying character data.

The method as shown in FIGS. 24A and 25A selects an item, registers the item and the text data if there is no error in the text data of the selected item, and corrects the error text data if there is an error in the text data of the selected item. Registered text data with the corresponding item.

In the above-described method, as shown in FIG. 27B, a sentence of the first display area 71 is first selected by the stylus pen, and an item corresponding to the selected sentence is selected by the stylus pen in the second display area 75. In this case, the selected item and the corresponding sentence are displayed on the third display area 73. In this case, when the items and sentences displayed in the third display area 73 are correct, the confirmation key is clicked with a stylus pen as shown in FIG. 27B. In this case, the items and sentences displayed in the third display area 73 are registered. . However, when an error of a sentence displayed in the third display area 73 occurs, the correction key is clicked with a stylus pen as shown in FIG. 28A. Subsequently, as shown in FIG. 28B, when an error character displayed in the first display area 71 is clicked with a stylus pen, the clicked character is enlarged and displayed, and candidate characters of the error recognized character are displayed in the third display area 73. . A recognition window for inputting handwritten characters is displayed in the second display area 75 and a soft keypad is displayed in the fourth display area 77. The method for correcting the error recognized character in the above state may include selecting a candidate character displayed in the third display area 73, inputting a cursive character to be corrected in the recognition window of the second display area 75, or a fourth display area. You can enter character key data for error correction via the soft keypad of 77. When deleting or adding a character, enter a delete key or an additional key. If there is another error recognition character in the selected item, repeat the above process. When the modification is completed, the correction key is clicked with a stylus pen, and the display returns to the state as shown in FIG. 27A and allows the next item to be selected.

In the above description, a method of correcting an error character, inputting a handwritten character, selecting a candidate character, and performing the method through a soft key pad is described. However, the method may be implemented by using the handwritten character input method alone or by using a soft keypad alone. . In addition, the candidate character selection and handwritten character recognition may be implemented in parallel, or the candidate character selection and the soft keypad may be implemented in parallel.

24B and 25B are diagrams showing another item selection and error correction procedure of the first embodiment of the present invention.

A character recognition and an item selection operation will be described with reference to FIG. 24B. The character recognition and item selection process of FIG. 24B is performed by the same procedure as that of FIG. 24A. However, if a correction key is input, an error correction process is not immediately performed and only an indication that an error has occurred in the corresponding item is performed. That is, when a modification key is generated in the item selection process, the control unit 101 detects this in step 429, indicates that an error has occurred in the character recognition data of the corresponding item in step 450, and returns to step 421. The operation of FIG. 24B is operated in the same procedure as that of FIG. 24A except for the modification operation as described above. Therefore, if the character recognition and item selection process is performed by the procedure as shown in FIG. 24B, if an error of character recognition occurs in the selected item in the process of selecting the item after the character recognition, it indicates that an error has occurred in the corresponding item and returns the character. The character data of an item for which no recognition error has occurred is registered with the corresponding item and the character corresponding to the item.

In addition, in the process of selecting an item after the character recognition, it is possible to implement the method by selecting only items without performing an operation of checking and correcting. That is, after selecting the desired items in the recognized document without performing the checking and correcting operation, it is possible to implement a method for collectively sorting the errors by checking the text data of the selected items in the error correction process.

FIG. 25B is a diagram illustrating a process of correcting character data of items in which an error occurs after character recognition and item selection in the same manner as in FIG. 24B.

Referring to FIG. 25B, when the correction key is input, the controller 101 detects this in step 551. In step 553, the controller 101 displays an error-recognized item on the second display area 75 of the display unit 115. Character data of the error items is displayed in the first display area 71. When the text data to be corrected in the first display area 71 of the display unit 115 is clicked using the stylus pen in the state shown in FIG. 28A, the controller 101 detects this in step 513 and the text to be modified as shown in FIG. 28B in step 515. Display data. Then, when the user clicks an item for correcting the error-recognized text data using the stylus pen, the controller 101 detects this in step 555 and proceeds to step 557 to perform the operation as shown in FIG. 25A.

In FIG. 25A, an error recognized text data is corrected in the text data of the selected item. After performing the above error correction operation, when the correction is completed, the controller 101 stores the corresponding item and the modified character data in step 559. When the next item is selected by the user after the modification of the text data of the selected item is completed as described above, the controller 101 detects it in step 561 and returns to step 557 to correct the error-recognized text data of the selected item again. Repeat this. As described above, items having error-recognized character data are sequentially selected to perform a modification operation of the character data. . After the modification of the text data of all the items is completed, the user generates the correction completed key of the input unit 113. In step 561, the controller 101 detects this and displays and stores the modified items and the modified text data corresponding to each item on the display unit 115 in step 563.

When the character recognition, item selection, and error correction procedures are completed as described above, all the desired information recorded on the business card is entered. In this case, the character data of the selected item is displayed as shown in FIG. 29A. When the user clicks on the completion key with the stylus pen in the above state, the controller 101 detects that the business card recognition is finished, and displays the items and the text data corresponding to the items on the display 115 as shown in FIG. 29B. Display on the screen. The display data as shown in FIG. 29B is stored in the database 131. That is, after all of the above input process, recognition process, and modification process, document-recognized data of selected items are stored in a database of a desired area. The area of the database 131 may be various spaces such as a phone book, a notepad, and other application programs. Save all the data you want and exit the program.

In the second embodiment of the present invention, an error is corrected for each item when the document is recognized, and a speech recognition technique is introduced when the document is recognized and the error is corrected.

30 is a diagram illustrating an operation procedure according to a second embodiment of the present invention.

Referring to FIG. 30, the controller 101 performs a document image capturing operation in step 200. In this case, the video image photographed by the camera 107 is processed by the image processor 109 and then converted into digital data and displayed on the display 115. When a picture taking command is generated while the video image photographed on the display unit 105 is displayed, the controller 101 displays the video image displayed on the display unit 115 as a still image, and displays the video image displayed on the display unit 115. It is stored in the image memory area of the memory 103. In this case, the video image displayed on the display unit 115 may be a moving image or may be text image data such as a business card. The operation of step 200 may use a stored video screen or an input video screen as described with reference to FIGS. 24A and 24B.

In the above state, the user of the mobile terminal apparatus inputs a document recognition key corresponding to the currently displayed document through the input unit 113. Then, the controller 101 drives the preprocessing unit 121 to perform the preprocessing of the document image in step 210, and recognizes the character images of the preprocessed document image in step 220. The operation at this time may also be performed in the same manner as the operation described with reference to FIGS. 24A and 24B. Then, the text recognition unit 123 recognizes text images in the video screen displayed on the display unit 115 and converts the text images into text data. Thereafter, the controller 101 displays the character data recognized by the character recognition unit 123 in the first display area 71 of the display unit 115, and displays item information according to the type of the document input key in the second display area 75 of the display unit 115. do.

Then, when the user selects the recognized text data displayed in the first display area 71 of the display unit 115 and the item displayed in the second display area 75, the controller 101 selects the selected text data and the item in step 230. Is displayed on the third display area 73 of the display unit 115. In this case, the method for selecting an item may include a method of selecting an item displayed through the input unit 113 and a method of driving the voice recognition unit 129 to select an item.

If there is an error in the text data of the selected item after selecting the item as described above, the text data of the corresponding item is corrected. In this case, the error correction may be performed by selecting an error-prone text through the input unit 113 or by driving the speech recognition unit 129 to correct text data in which an error occurs by voice. If an error correction is requested in the above manner, the controller 101 detects this in step 241, and the controller 101 proceeds to step 240 to correct the error-prone text among the recognized text data.

When the modification is completed as described above, the controller 101 stores the modified character data in the database 131 as the character data of the corresponding item.

31 is a diagram illustrating a procedure of a document photographing process performed in step 200 of FIG. 30, and FIGS. 26A to 26E are diagrams of displaying a document image photographed in the document photographing process on the display unit 115. 31 may be implemented in the same manner as in FIG. 3.

Referring to the operation of the document photographing process, the user places the document to be recognized at an appropriate position and starts photographing using the camera 107 of the mobile terminal device. Then, the controller 101 is displayed on the display unit 115 as shown in FIGS. 26A and 26B in step 651. In this case, when the user of the portable terminal receives a camera adjustment key of the key input unit 105 (also also available from the input unit 113), the control unit 101 detects this in step 653 and controls the camera 107. In this case, adjustment of the camera 107 may be adjustment of exposure time and distance. Then, the document image as shown in FIG. 26A displayed on the display unit 115 of the document image photographed according to the exposure time and distance adjustment of the camera 107 is displayed with fine adjustment of the focus and distance. When the user presses the shooting key of the input unit 113 using the stylus pen in the above state, the control unit 101 detects this by taking a picture in step 655 and stops the document image at the time when the shooting key is input on the display unit 115. It is displayed as an image.

In step 659, the controller 101 displays the photographed document image. In this case, the document image displayed on the display unit 115 is as shown in FIG. 26C. When the document image displayed on the display unit 115 is satisfactory as shown in FIG. 26C, the user presses a storage key displayed on the input unit 113 using the stylus pen. When the storage key is generated, the controller 101 detects it in step 661 and stores the displayed document image in the image memory area of the memory 103 with a name. In this case, the document image displayed on the display unit 115 is as shown in FIG. 26E.

When the user clicks the business card recognition key in the above state, the control unit 101 detects this in step 663 and proceeds to the document recognition process in step 220, otherwise proceeds to step 665 to store the currently displayed document image and Quit.

As described above, in step 200 of capturing the document, a user inputs a desired image through a camera and sharply captures a document image through camera fine adjustment. If the corrected image is satisfactory, it is checked whether the character is extracted from the input image through text recognition and stored as text data or just as a photo. At this time, if the user of the portable terminal device requests the character recognition, the preprocessing and document recognition procedures of 210 and 220 are performed.

32 is a diagram illustrating a preprocessing and recognition process for a document image and an item selection process of recognized character data performed in steps 210 to 230 of FIG. 30, and FIGS. 27A to 27B illustrate the document recognition and It is a figure which shows the state of an item selection process.

Referring to FIG. 32, before the business card recognition key is generated, the controller 101 displays an image of the business card being stored on the display unit 115 as shown in FIG. 26E. When the user generates the business card recognition key of the input unit 113 in the above state, the control unit 101 detects this, and in step 751 the pre-processing unit 121 drives the pre-processing operation of the displayed document image. In this case, the preprocessing operation may be performed as described above. Then, the character recognition unit 123 converts the image of the business card as shown in FIG. 26E into text data, and the controller 101 displays the converted text data on the display unit 115 as shown in FIG. 27A. When the business card image is converted into text data, the controller 101 displays text data of the business card image on the first display area 71 of the display unit 115 and displays an item selection on the third display area 73 as shown in FIG. 27A. Items to be stored are displayed on the display area 75.

In the state where the recognized character data are displayed as shown in FIG. 27A, the user selects the character data (sentence) of the first display area 71 using a stylus pen as shown in FIG. 27B, and the second display area 75 When the stored item is selected, the controller 101 detects this in step 755 and displays the selected item and corresponding text data in the third display area 73 of the display unit 115 as shown in FIG. 27B in step 757. In addition, the method of selecting a stored item in step 757 may be executed by voice. In this case, the user of the portable terminal apparatus may select a voice recognition mode through the input unit 113 or the key input unit 105 and then input a desired storage item by voice.

When the correction key is generated from the input unit 113 in the state of displaying the text data as described above, the control unit 101 detects this in step 759 and proceeds to step 761 to perform an error correction process. However, if the correction key is not input, the controller 101 checks whether a selection key of the next item is input. When the selection of the item is input, the controller detects it in step 763 and proceeds to step 755 to select the next item. However, if the input of the completion key is detected in step 763, the controller 101 proceeds to step 765 to store the text data of the selected items in the database 131 and ends the document recognition procedure.

As described above, another document recognition process according to the second embodiment of the present invention performs a task of converting characters included in the input document image into text by driving a character recognizer. After displaying the converted text on the display unit 115, the user selects characters to be stored. At this time, a region (name, address, company, etc.) to store the selected characters is designated, and the characters are copied and input in the region. A process of designating a region to be stored in the item selection process will be described in detail later in the item selection process of FIG. 33. In addition, if there is a character to be corrected among the recognized characters, go to the error correction process, otherwise, ask if there is an additional item to save, repeat the item selection process, or go to the storage process and store it in the database And exit the program.

FIG. 33 is a diagram illustrating a detailed flow of an item selection process performed in steps 755 and 757 of FIG. 32.

Referring to FIG. 33, after performing character recognition, the display unit 115 is displayed as shown in FIG. 27A. In the display state as described above, the user may select the item displayed in the second display area 75 with the stylus pen or select the voice recognition mode through the input unit 113 or the key input unit 105 in order to select a desired storage item. In this case, when the voice recognition mode is selected, the controller 101 detects this in step 771, and in step 773, presses a record button and inputs a desired storage item and data by voice. Then, in step 773 and step 775, the controller 101 applies the voice signal received through the audio processor 111 to the voice recognition unit 123, and drives the voice recognition unit 129 to recognize the received voice. In step 777, the controller 101 displays and stores the text data of the item corresponding to the speech recognized signal as shown in FIG. 27B.

In addition, when the selection of the storage item is detected with the stylus pen in step 771, the controller 101 displays the storage item as shown in FIG. 27A in step 779, and when the desired item is clicked with the stylus pen in step 781, the selected item is stored as shown in FIG. 27B. The item and the text data are displayed, and the text data of the area specified in the storage item selected in step 783 is stored.

As described above, a method of selecting a storage item may be divided into a case of selecting by voice recognition and a case of selecting by a stylus pen. First, a method of selecting voice recognition is performed by pressing a record button and then pronouncing a desired item among the stored items displayed as shown in FIG. 27A and specifying the item through the voice recognition unit 129. At this time, if you select 'add item', the user does not move on to the next step, and the user adds a desired item to the storage item table. In the second method of selecting a stylus pen, a desired item is selected by using a stylus pen among the stored items displayed on the display unit 115 as illustrated in FIG. 27A. These two methods do not operate sequentially but may use one of them depending on the user's choice.

33 illustrates an example of selecting a desired item and text data by using a voice recognition or a stylus pen. However, the storage item may be selected by voice recognition and the text data may be selected by a stylus pen, or the stored item may be selected by a stylus pen, and the text data may be selected by voice recognition.

34A to 34D illustrate an operation procedure of operation 240 of FIG. 30 for correcting character data having an error for each selected item according to the second embodiment of the present invention.

Referring to FIG. 34A, the error correction procedure in step 240 of FIG. 30 is performed. When a desired item is selected, the controller 101 displays the selected item and the text data of the corresponding item in the third display area 73 of the display unit 115 as shown in FIG. 28A. do. At this time, when the text data of the selected item is misrecognized as shown in FIG. 28A, the user clicks a modification key using a stylus pen or selects a voice recognition mode to instruct modification. Then, the controller 101 detects this and inputs candidate characters most similar to the characters designated by the character recognition unit 123 in step 811, and displays them in the third display area 73 of the display unit 115 in step 813. In this case, the controller 101 displays candidate characters for correcting the error recognized character in the third display area 73 of the display unit 115 as shown in FIG. 28B, and also adds handwritten characters to correct the error recognized character in the second display area 75. Displays a recognition window for inputting a key or displays a soft keypad on the fourth display area. In this case, the user of the portable terminal apparatus clicks a corresponding character using a stylus pen if there are desired characters among the candidate characters displayed in the third display area 73 of the display unit 115. Accordingly, when any one of the candidate characters is selected by the stylus pen, the controller 101 detects this in step 815 and in step 817 corrects the error recognition character displayed in the first area to the selected candidate character.

However, if there are no desired characters among the candidate characters displayed in the third display area 73, the user of the portable terminal apparatus selects a voice recognition mode or uses the handwriting recognition window of the second display area 75 or the fourth display area. You can use the softkeypad. In this case, when the user selects the voice recognition mode through the input unit 113 or the key input unit 105, the control unit 101 proceeds to step 820 to perform the operation as shown in FIG. 34B and writes the desired handwritten characters in the handwriting recognition window of the second display area 75. If it is input, the operation as in step 34c is performed even in step 850.

The correction process using the stylus pen refers to the data value extracted from the character recognition unit 123. When one character is recognized in the character recognition process processed as shown in FIG. 33, the character recognition unit 123 determines the character most similar to the input character as the character data of the corresponding item, and then selects several similar characters. It is retained as candidate characters. Subsequently, in the error correction process performed in the procedure of FIG. 34A, candidate characters of characters to be corrected by the user are retrieved from the character recognition unit 123 and displayed in the third display area 73 of the display unit 115. At this time, the user of the portable terminal device selects and corrects a desired character from the candidate characters with a stylus pen. If there is no desired character, the user recognizes the voice recognition procedure of FIG. 34B or the handwriting recognition of FIG. 34C to input a new character. Procedure or the soft key recognition procedure of Fig. 34D. This switching is performed directly on one screen. The handwriting recognition area and the soft keypad are always driven at the bottom of the display unit 115 of the portable terminal device to wait for the user's selection, and when the record button is pressed, the voice recognition unit 129 is driven. . Accordingly, the character recognition unit 123 is designed to recognize printed characters, handwritten characters, and soft keys.

Referring to FIG. 34B, the voice recognition unit 129 should be configured differently according to the language of each country. The voice recognition unit 129 is not a word, but is input as a letter. Due to the characteristics of the language, one letter may form a word as in English, and several phonemes may form a letter as in Korean. For example, in the case of English, the word "KOREA" is composed of five letters, whereas in the case of Korean, two characters of "Korea" are composed of six phonemes, each of three phonemes. Therefore, in the case of a language such as Korean, the voice recognition unit 129 must input a phoneme unit in order to input a desired character unless the voice recognition engine is unlimited. Therefore, in the speech recognition mode, the language mode is first selected, and in the case of Hangul, an unlimited engine is selected to attempt speech recognition.

Therefore, during voice recognition, the controller 101 determines whether the character to be modified in step 821 is Korean or English. In this case, in the English mode, the user of the portable terminal device selects the English mode, presses the record button, and inputs text data for correction by voice. Then, the controller 101 detects that the English text data is input to the voice in step 835, and in step 837 drives the voice recognition unit 129 to recognize the English text data of the voice output from the audio processor 111 and outputs it to the controller 101. . Then, the controller 101 corrects the text data of the selected item with the voice-recognized English text data, and determines whether to correct the next text data in step 839. In this case, if there are more characters to be corrected in the text data of the selected item, the controller 101 returns to step 835 and repeats the above process. If there are no characters to be modified, the controller 101 proceeds to step 251 of FIG.

If the character to be corrected in step 821 is Korean, the controller 101 proceeds to step 823 and checks whether the voice recognition unit 123 is an unlimited voice recognition engine. In this case, if the unlimited voice recognition engine, the control unit 823 proceeds to step 823, and performs steps 835 to 839 as described above and performs the voice recognition operation of the Hangul characters. In this case, the Hangul speech recognition is performed in units of characters.

However, if the speech recognition unit 129 is not an unlimited speech recognition engine, the controller 101 proceeds to step 825 to perform Korean speech recognition in phoneme units. In this case, when the user modifies the text data of the selected item, the user presses the record button and sequentially inputs the phonemes constituting the Hangul text data by voice. When the voice input of the phonemes for the corresponding text is completed, the user presses a completion button. When the phonemes constituting the text as described above are inputted as voice, the controller 101 receives this in step 825, and in step 827 drives the voice recognition unit 129 to recognize the received phonemes. After recognizing the phonemes of the desired location to be corrected as described above, the controller 101 detects that the phoneme input of the text data is completed in step 829, and combines the phonemes in step 831 to complete the text and then converts the phonemes into the text data of the selected item. Correct it. In step 833, the controller 101 determines whether to modify the next character data. In this case, if there are more characters to be modified in the text data of the selected item, the controller 101 returns to step 825 and repeats the above process. If there are no characters to be modified, the controller 101 proceeds to step 251 of FIG.

As described above, the operation of the error correction process using voice recognition may be performed when the error correction is not completed in the process of correcting the error with the stylus pen. The error correction by the voice recognition is whether or not the character to be largely modified is a character having at least two constituent characters (for example, Korean: consonants and vowels, characters consisting of a plurality of phonemes of consonants, vocals, and voices). English: Alphabet characters are connected in sequence). Currently, portable terminal devices (for example, PDAs) cannot enter an unlimited voice recognizer that requires a very large capacity. Therefore, Korean characters cannot be recognized unless they have been previously inputted. Therefore, in the case of Hangul, the range that can be corrected at one time is divided into phoneme units (secondary / neutral / jongseong) and corrected by one part. If the unlimited voice recognizer is implemented in the portable terminal device in the future, it is not necessary to perform such a process, it is necessary to modify one by one immediately. On the other hand, in the case of English or special characters can be modified one by one, the user selects the alphabet or special characters that you want to modify, and if you press the record button to pronounce the desired character, you can modify the character through the voice recognizer. After the modification process, the process returns to the storage item selection process of FIG.

Referring to FIG. 34C, the error correction operation by handwritten character recognition is performed. In step 851, the error recognized character is displayed as shown in FIG. 8B, and the recognition window of the second display area 75 is touched by a stylus pen. When the handwritten character is input to the control unit 101, the control unit 101 detects this in step 853, and in step 855 drives the character recognition unit 123 to recognize the input handwritten character. The controller 101 modifies the error-recognized character data of the selected item to the character data recognized by the character recognition unit 123. In step 857, the controller 101 determines whether to modify the next character data. In this case, if there are more characters to be corrected in the text data of the selected item, the controller 101 returns to step 853 and repeats the above process. If there are no characters to be modified, the controller 101 proceeds to step 251 of FIG.

As described above, error correction by handwriting recognition is performed through a handwriting recognition window loaded in the second display area 75 of the display unit 115. In the error correction process of FIG. 34A, when the user fails to correct the desired character, the desired character is directly written in the handwriting recognition window.

Referring to FIG. 34D, the error correction operation by soft key recognition will be described. In step 871, the controller 101 displays an error recognized character as shown in 8b and displays a soft keypad on the fourth display area 77. In this case, when the soft key data received through the soft keypad is received, the control unit 101 detects this in step 873, and in step 875 drives the soft key recognizer of the character recognition unit 123 to recognize the characters corresponding to the input soft keys. do. The controller 101 modifies the error-recognized character data of the selected item into character data recognized by the soft key recognizer of the character recognizer 123. In step 877, the controller 101 determines whether to modify the next character data. In this case, if there are more characters to be corrected in the text data of the selected item, the controller 101 returns to step 853 and repeats the above process. If there are no characters to be modified, the controller 101 proceeds to step 251 of FIG.

As described above, error correction by soft key recognition is performed through a soft keypad loaded in the fourth display area 77 of the display unit 115. In the error correction process of FIG. 34A, if the user fails to correct the desired character, the user inputs the soft keys of the soft keypad and corrects the desired character.

When the process of selecting the items of the recognized text and the error correction process for the text data of the selected item is completed as described above, the user of the portable terminal device inputs a completion key through the input unit 113. In step 251, the controller 101 detects this and stores the document recognition result in the database 131. The database registers the items selected in the document and the text data of the items at the address designated by the user.

After the above input process, recognition process, and modification process, all of these data can be stored in the database of the desired area. In this case, the database area may be various spaces such as a phone book, a notepad, and other applications. Save all the data you want and exit the program.

In the second embodiment of the present invention as described above, the storage items of the recognized document are selected after document recognition. When an error occurs in the text data of the selected storage item, the error text is corrected and then the next storage item is selected. Therefore, in the process of storing the recognized document for each item, the character in which an error occurs is also modified and stored. In addition, in the second embodiment of the present invention, a voice recognizer may be used to select an item to store or to correct an error-prone text.

In addition, in the second embodiment of the present invention, when correcting an error, a candidate character is first selected to correct an error-prone character, and when the error correction is impossible using the candidate character, speech recognition or handwritten character and softkey recognition are performed. An example of modifying is described. However, it is also possible to select and implement some methods among a method of selecting a candidate character when correcting the error character, a voice input method by voice recognition, a handwritten character input method, and a text input method by a soft keypad. That is, the error character correction method may be implemented by directly inputting a voice, a handwritten character, and a soft key without selecting a candidate character. In addition, although the second embodiment of the present invention describes a method of correcting an error character by selecting a candidate character, speech recognition, and handwriting recognition, the method of selecting a candidate character and a speech recognition method, a speech recognition and a handwriting recognition method, and a speech recognition and software It is possible to implement only by the key recognition method, it is also possible to implement using only the voice recognition, soft key or handwritten character recognition method.

In addition, in the exemplary embodiment of the present invention, the document is assumed to be a business card and described.

As described above, in the case of registering information of a document in a device such as a portable terminal, text data can be registered through text recognition and / or voice recognition after scanning the image of the document. Operation can be minimized, and there is an advantage in that a character or a voice recognized during speech recognition can be easily modified. In addition, the information of the document may be input through the text and voice recognition method, and thus there is an advantage of efficiently inputting a large amount of document information.

Claims (33)

An apparatus for recognizing a character image of a document, An input unit for generating a command for performing the recognition mode, the correction mode, and the storage modes; A pre-processing unit configured to analyze pixels in the document image in the recognition mode, classify them into letter blocks and background blocks, and then binarize pixels of the letter blocks to generate preprocessed document images; A character recognition unit for recognizing preprocessed document images and converting them into character data; A recognition error processing unit for correcting misrecognized character data selected by the input unit in the correction mode to character data output from the input unit; A database for storing the recognized text data in the storage mode; And a display unit for displaying document images and text data generated during the mode execution. The method of claim 1, wherein the pretreatment unit Classify stripes having a length greater than or equal to a predetermined size in the document image, calculate a tilt angle of the classified stripes, determine a tilt of the subject, determine a rotation angle corresponding to the measured tilt, and tilt the subject. With subject tilt correction to correct, The document image whose slope of the subject is corrected is classified into a letter block and a background block, a text area is extracted by searching the positions of the classified letter blocks, and an image screen of the extracted text area is input to the size of the input document image. An image area expansion unit extending to And an image binarization unit configured to compare pixels of the letter blocks of the document image with a pixel reference value to binarize the pixel and background pixels with brightness values and to binarize the pixels of the background block with the brightness values of the background pixels. The method of claim 2, wherein the preprocessor classifies the input document image into a letter block and a background block, calculates an average energy ratio of the classified letter blocks, and compares the received document image with a predetermined reference value to determine whether a blood image is displayed. The apparatus further comprises an image blurring determining unit for determining. The apparatus as claimed in claim 2, wherein the preprocessor further comprises a noise removing unit which removes noise of an image screen having an extended image region and outputs the noise to the image binarization unit. The apparatus of claim 1, wherein the apparatus for recognizing the document image further comprises a camera that photographs a document and generates a document image. The voice recognition unit of claim 5, further comprising an input signal for selecting the item in the storage mode, and a voice recognition unit for generating an input signal for selecting and correcting character data misrecognized in the correction mode. Document information storage device of the portable terminal, characterized in that for converting the input voice signal into text data. The document of claim 5, wherein the character recognition unit further comprises a handwriting recognition unit, and recognizes the handwritten character image received in the correction mode and converts the character text to the corrected character data of the misrecognized character data. Information storage. The apparatus of claim 5, wherein the camera is capable of adjusting distance and exposure. An apparatus for storing document information using a camera, An input unit for generating a command to perform a shooting mode, a recognition mode, a correction mode, and storage modes; A display unit including a first display area for displaying an input document image and the recognized text data, a second display area for displaying items, a third display area for displaying text data of a selected item, and an area for displaying a mode menu Wow, A camera driven in the photographing mode to photograph the document image; A pre-processing unit configured to analyze pixels in the document image in the recognition mode, classify them into letter blocks and background blocks, and then binarize pixels of the letter blocks to generate preprocessed document images; A character recognition unit for recognizing a preprocessed document image and converting it into the character data; A recognition error processing unit for correcting misrecognized character data selected by the input unit in the correction mode to character data output from the input unit; A database for storing the recognized text data in the storage mode; And a display unit for displaying document images and text data generated when the respective modes are performed. The method of claim 9, wherein the pretreatment unit Classify stripes having a length greater than or equal to a predetermined size in the document image, calculate a tilt angle of the classified stripes, determine a tilt of the subject, and determine a rotation angle corresponding to the measured tilt to tilt the subject. With subject tilt correction to correct, The document image whose slope of the subject is corrected is classified into a letter block and a background block, a text area is extracted by searching the positions of the classified letter blocks, and an image screen of the extracted text area is input to the size of the input document image. An image area expansion unit extending to And an image binarization unit configured to compare pixels of the letter blocks of the document image with a pixel reference value to binarize the pixel and background pixels with brightness values and to binarize the pixels of the background block with the brightness values of the background pixels. The method of claim 10, wherein the preprocessor classifies the input document image into a letter block and a background block, calculates an average energy ratio of the classified letter blocks, and compares the received document image with a predetermined reference value to determine whether a blood image is displayed. The apparatus further comprises an image blurring determining unit for determining. The apparatus of claim 11, wherein the preprocessing unit further comprises a noise removing unit which removes noise of an image screen having an extended image region and outputs the noise to the image binarization unit. 13. The apparatus of claim 12, further comprising an input signal for selecting the item in the storage mode and a voice recognition part for generating an input signal for selecting and correcting character data misrecognized in the correction mode. Document information storage device of the portable terminal, characterized in that for converting the input voice signal into text data. The document of claim 12, wherein the character recognition unit further comprises a handwriting recognition unit, and recognizes the handwritten character image received in the correction mode, and converts the received character image into corrected character data of the misrecognized character data. Information storage. A method for recognizing a text image included in a document image in a terminal device, Specifying a mode for document recognition, In the document recognition mode, the pixels in the document image are analyzed and classified into a letter block and a background block, and the pixels of the letter block are binarized to generate a preprocessed document image. Recognizing the preprocessed document image and converting it into text data; Selecting a character data that is misrecognized in the correction mode, and correcting the misrecognized character data with the input character data; And storing the recognized text data in a storage mode. The method of claim 15, wherein the pretreatment process, Classify stripes having a length greater than or equal to a predetermined size in the document image, calculate a tilt angle of the classified stripes, determine a tilt of the subject, and determine a rotation angle corresponding to the measured tilt to tilt the subject. Calibration process, The document image whose slope of the subject is corrected is classified into a letter block and a background block, a text area is extracted by searching the positions of the classified letter blocks, and an image screen of the extracted text area is input to the size of the input document image. The process of expanding to And comparing the pixels of the text blocks of the document image with a pixel reference value to binarize the pixels and the background pixels with brightness values and the pixels of the background block with the brightness values of the background pixels. The method of claim 16, wherein the preprocessing process classifies the input document image into a letter block and a background block, calculates an average energy ratio of the classified letter blocks, and compares the document image with a predetermined reference value to determine whether the image is a blood image. The method of claim 1, further comprising the step of performing the pre-processing step if it is not a blood image. 18. The method as claimed in claim 17, wherein the preprocessing further comprises removing noise from an image screen having an extended image region and outputting the noise to the image binarization unit. The method of claim 18, wherein the error correction process, Displaying candidate characters corresponding to character data misrecognized in the correction mode; And modifying a character selected from the displayed candidate characters into character data of the misrecognized character. The method of claim 18, wherein the error correction process, Displaying a recognition window for inputting handwritten characters when the correction mode is requested; Recognizing a cursive character when the cursive character is input to the cursive recognition window; And modifying the recognized character into character data of the misrecognized character. The method of claim 18, wherein the error correction process, Displaying candidate characters of a character recognized incorrectly in the correction mode; Modifying the selected character among the displayed candidate characters into character data of the misrecognized character; Displaying a cursive recognition window when there is no character data to be corrected in the candidate character in the display process; Recognizing a cursive character when the cursive character is input to the cursive recognition window; And modifying the recognized character into character data of the misrecognized character. The method of claim 18, wherein the error correction process, Driving a voice recognition unit in the correction mode; Converting the input voice signal into text data by recognizing the voice recognition unit; And converting the converted character data into character data of the misrecognized character. A document having a display portion having a first display area for displaying text images and text data, a second display area for displaying items, a third display area for displaying text data of a selected item, and a menu area for displaying a mode menu. In the method for recognizing a text image included in the image, Displaying a document image taken by the camera, Analyzing the pixels in the document image in the document recognition mode, dividing them into a letter block and a background block, and binarizing the pixels of the letter block to generate a preprocessed document image and display the same in the first display area; Recognizing and converting the preprocessed document image into text data and displaying the text data on the first display area, and displaying the items of the document data on the second display area; Selecting an item to be stored from among the displayed items, selecting and storing text data of the selected item, and And repeating the process and storing the selected item and the corresponding text data. 24. The method of claim 23, further comprising an error correction process of correcting misrecognized text data after selecting the item and text data. The error correction process, Displaying candidate characters of characters that are incorrectly recognized in the third display area when an error correction request is made; And modifying a character selected from the displayed candidate characters into character data of the misrecognized character. 24. The method of claim 23, further comprising an error correction process of correcting misrecognized text data after selecting the item and text data. The error correction process, Displaying a handwriting recognition window on a second display area when the error correction request is made; Recognizing a cursive character when the cursive character is input to the cursive recognition window; And modifying the recognized character into character data of the misrecognized character. 24. The method of claim 23, further comprising an error correction process of correcting misrecognized text data after selecting the item and text data. The error correction process, Displaying candidate characters of characters that are incorrectly recognized in the third display area when an error correction request is made; Modifying the selected character among the displayed candidate characters into character data of the misrecognized character; Displaying a cursive recognition window on the second display area when there is no text data to be corrected in the candidate text; Recognizing a cursive character when the cursive character is input to the cursive recognition window; And modifying the recognized character into character data of the misrecognized character. 24. The method of claim 23, further comprising an error correction process of correcting misrecognized text data after selecting the item and text data. The error correction process, Driving the voice recognition unit upon the error correction request; Converting the input voice signal into text data by recognizing the voice recognition unit; And converting the converted character data into character data of the misrecognized character. A portable unit having a display unit having a first display area for displaying character data of a recognized business card, a second display area for displaying items, a third display area for displaying text data of a selected item, and an area for displaying a mode menu. In the business card image recognition method of the terminal device, Displaying an image of the business card photographed through a camera; Analyzing pixels in the displayed business card image in a recognition mode, dividing them into a letter block and a background block, and binarizing the pixels of the letter block to generate a preprocessed document image; Converting the text images of the preprocessed business card into text data, displaying the converted text data in a first display area and displaying items of the business card in a second display area; Selecting an item to be stored among the displayed items, selecting text data of the selected item and displaying the selected text data in a third display area; Selecting a character data that is misrecognized in the correction mode, and correcting the misrecognized character data with the input character data; And storing the recognized text data in a storage mode. The method of claim 28, wherein the storage item, Wherein said method comprises a name, a mobile phone number, a company phone number, an e-mail address, a position, and the like. The method of claim 29, wherein the performing of the modification mode comprises: Displaying candidate characters of character data misrecognized in the third display area; The method comprising the step of modifying the misrecognized character to the selected candidate character. The method of claim 29, wherein the correcting the misrecognized character, Displaying a handwriting recognition window on the second display area when the correction key is input; Recognizing a cursive character when the cursive character is input to the cursive recognition window; And a process of modifying the recognized character into character data of the misrecognized character. The method of claim 29, wherein the correcting the misrecognized character, Driving a voice recognition unit when the correction key is input; Converting the input voice signal into text data by recognizing the voice recognition unit; And modifying the converted character data into character data of the misrecognized character. The method of claim 29, wherein the error correction process, Displaying candidate characters of a character recognized incorrectly in the correction mode; Modifying the selected character among the displayed candidate characters into character data of the misrecognized character; Displaying a cursive recognition window when there is no character data to be corrected in the candidate character in the display process; Recognizing a cursive character when the cursive character is input to the cursive recognition window; And modifying the recognized character into character data of the misrecognized character.