WO2006025692A1

WO2006025692A1 - Method for processing document image captured by camera

Info

Publication number: WO2006025692A1
Application number: PCT/KR2005/002875
Authority: WO
Inventors: Seong Chan Byun; Sung Hyun Kim; Sang Wook Park; Yu Nam Kim
Original assignee: LG Electronics Inc
Current assignee: LG Electronics Inc
Priority date: 2004-08-31
Filing date: 2005-08-30
Publication date: 2006-03-09
Anticipated expiration: 2007-02-28
Also published as: KR20060050746A; US20060045342A1; US7623742B2

Abstract

A document image processing method includes providing a document image, selecting fields, which is to be recognized, from the document image, and performing character recognition process for the selected fields.

Description

METHOD FOR PROCESSING DOCUMENT IMAGE CAPTURED BY CAMERA

Technical Field

[1] The present invention relates to a method for recognizing characters on a document image captured by a camera and saving recognized characters. Particularly, the present invention relates to a method for recognizing characters on a name card image captured by a mobile camera phone with an internalized or externalized camera and automatically saving the recognized characters in corresponding fields of a pre¬ determined form such as a telephone directory database. Background Art

[2] An optical character recognition (OCR) system or a scanner-based character recognition system has been widely used to recognize characters on a document image. However, since these systems are dedicated system for recognizing characters on a document image, massive applications and hardware sources are required to process and recognize the document image. Therefore, it is difficult to simply apply the character recognition method used in the OCR system or scanner based recognition system to a device having a limited process and memory. A mobile camera phone may be designed to recognize the characters. That is, the camera phone is used to take a picture of a small name card, recognize the characters on the captured image, and auto¬ matically save the recognized characters in a phone number database. However, since the mobile camera phone has a limited processor and memory, it is difficult to accurately process the image and recognize the characters on the image.

[3] Describing a method for recognizing a name card using the mobile camera phone in more detail, a name card image is first captured by a camera of the mobile camera phone and the characters on the captured card image are recognized by fields using a character recognition algorithm. The recognized characters are displayed by fields such as a name, a telephone number, an e-mail address, and the like. Then, the characters displayed by fields are corrected and edited. The corrected and edited characters are saved in a predetermined form of a phone number database.

[4] Generally, when a user receives names cards from customers, friends and the like, the users opens a phone number editor of his/her mobile phone and inputs the in¬ formation on the name card by himself/herself using a keypad of the mobile phone. This is troublesome for the user. Therefore, a mobile camera phone having a character recognizing function has been developed to take a picture of the name card and auto¬ matically save the information on the name card in the phone number database. That is, a document/name card image is captured by an internalized or externalized camera of a mobile camera phone and characters on the captured image are recognized according to a character recognition algorithm. The recognized characters are automatically saved in the phone number database.

[5] However, when a relatively large number of characters are existed on image capture by the camera or scanner, since the mobile phone has a limited process and memory source, a relatively long process time is taken even when the recognition process is optimized. Furthermore, when the characters are composed in a variety of languages, the recognition rate may be deteriorated as compared with when they are composed in a single language.

[6] Fig. 1 shows a schematic block diagram of a prior mobile phone with a character recognizing function.

[7] A mobile phone includes a control unit 5, a keypad 1, a display unit 3, a memory unit 9, an audio converting unit 7c, a camera module unit 7b, and a radio circuit unit 7a.

[8] The control unit 5 processes data of a document (name card) image read by the camera module unit 7b, output the processed data to the display unit 3, processes editing commands of the displayed data, which are inputted by a user, and save the data edited by the user in the memory unit 9. The keypad 1 functions as a user interface for selecting and manipulating the function of the mobile phone. The display unit 3 displays a variety of menu screens, a run screen and a result screen. The display unit 3 further displays an interface screen such as a document image data screen, a data editing screen and a edited data storage screen so that the user edits the data and save the edited data. The memory unit 9 is generally comprised of a flash memory, a random access memory, a read only memory. The memory unit 9 saves a real time operating system and software for processing the mobile phone, and information on parameters and states of the software and the operating system and performs the data input/output in accordance with commands of the control unit 5. Particularly, the memory unit 9 saves a phone number database in which the information corresponding to the recognized characters through a mapping process.

[9] The audio converting unit 7c processes voice signal inputted through a microphone by a user and transmits the processed signal to the control unit 5 or outputs the processed signal through a speaker. The camera module unit 7b processes the data of the name card image captured by the camera and transmits the processed data to the control unit 5. The camera may be internalized or externalized in or from the mobile phone. The camera is a digital camera. The radio circuit unit 7a functions to connect to mobile communication network and process the transmission/receive of the signal.

[10] Fig. 2 shows a block diagram of a prior name card recognition engine. [11] A prior name card recognition engine includes a still image capture block 11, a character-line recognition block 12, and application software 13 for a name card recognition editor.

[12] The still image capture block 11 converts the image captured by a digital camera 10 into a still image. The character line recognition block 12 recognizes the characters on the still image, converts the recognized characters into a character line, and transmits the character line to the application software. The application software 13 performs the name card recognition according to a flowchart depicted in Fig. 3.

[13] A photographing menu is first selected using a keypad 1 (S31) and the name card image photographed by the camera is displayed on the display unit (S32). A name card recognition menu for reading the name card is selected S33. Since the recognized data is not accurate in an initial step, the data cannot be directed transmitted to the database (a personal information managing data base such as a phone number database) saved in the memory unit. Therefore, the name card recognition engine recognizes the name card, coverts the same into the character line, and transmits the character line to the ap¬ plication software. The application software supports the mapping function so that the character line matches with an input form saved in the database.

[14] The recognized name card data and the editing screen is displayed on the display unit so that the user can edits the name card data and performs the mapping process (S34 and S35). The user corrects or deletes the characters when there is an error in the character line. Then, the user selects a character line that he/she wishes to save and saves the selected character line. That is, when the mapping process is completed, the user selects a menu "save in a personal information box" to save the recognized character information of the photographed name card image in the memory unit (S36).

[15] Figs. 4 and 5 show an example of a name card recognition process.

[16] Fig. 4 is an editing screen by which the user can corrects or deletes the wrong characters when the user finds the wrong characters while watching the screens provided in the steps S34 and S35. In the editing screen, the user moves a cursor to a wrong characters "DEL" 40 to change the same to a correct characters "TEL". After the editing is finished, the user selects only character lines that he/she wishes to save in the database and saves the same in the memory unit. For example, as shown in Fig. 5, when a job title of the name card is "Master Researcher," the line "Master Researcher" 50 is blocked and a field "title" 61 is selected in a menu list 60. Then, the mapping process is performed to save the "Master Researcher" that is a recognition result in a title field of the database. Disclosure of Invention Technical Problem [17] In order to improve the recognition rate of the mobile phone, a clear, correct document image data (a photographed name card image data) must be provided to an input device of the character recognition system. In addition, the most efficient recognition can be realized while minimizing the load of the limited sources such as the processor and memory of the mobile phone.

[18] In order to recognize the characters on the name card image captured by the camera, the name card image is first binary-coded and a field required by the user is selected from the binary-coded image and saved in the memory unit. At this select, since the binary-coding process is performed even for an unnecessary portion of the name card image, the recognition efficiency is deteriorated. Particularly, since the character recognition and the field selection are performed after the binary-coding process for all of the name card image is performed, the manipulation frequency of the mobile phone is increased, thereby deteriorating the user-convenience. Technical Solution

[19] Accordingly, the present invention is directed to a document image processing method, which substantially obviate one or more problems due to limitations and dis¬ advantages of the related art.

[20] It is an object of the present invention to provide a document image processing method that can improve recognition efficiency and speed and omit an unnecessary recognition process by selecting a field, for which a character recognition process will be performed, from a document image captured by a camera before the character recognition process is performed and performing the character recognition process only for the selected field.

[21] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly selected out in the written description and claims hereof as well as the appended drawings.

[22] To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, there is provided a document image processing method, comprising: providing a document image; selecting fields, which is to be recognized, from the document image; and performing character recognition process for the selected fields.

[23] According to another aspect of the present invention, there is provided a document image processing method of a mobile phone, comprising: selecting fields, which is to be character-recognized, from a document image captured by a camera; performing a character-recognition process for the selected fields; and saving recognized character data.

[24] According to still another aspect of the present invention, there is provided a document image processing method of a mobile phone, comprising: extracting fields from a document image; selecting fields, which is to be character-recognized, from the extracted fields; designating different binary-coding methods for the selected fields; performing the character-recognition process for the selected fields according to the designated binary-coding methods; and editing and storing recognized data.

[25] The document image is a name card image photographed by a camera associated with the mobile phone and the fields are character lines written on the name card.

[26] The selected fields are separated into different groups and different binary coding methods are applied for the two groups.

[27] The different binary-coding methods are designated for the selected fields in advance or selected by a user.

[28] It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. Advantageous Effects

[29] The present invention provide a document image processing method that can improve recognition efficiency and speed and omit an unnecessary recognition process by selecting a field, for which a character recognition process will be performed, from a document image captured by a camera before the character recognition process is performed and performing the character recognition process only for the selected field.

[30] According to the present invention, since the binary-coding process is performed only for the selected fields, the recognition speed and rate can be increased and the number of user's button manipulation can be reduced. Furthermore, since the spelling check function is provided for the recognition results, the recognition errors can be easily corrected. Brief Description of the Drawings

[31] The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principle of the invention. In the drawings:

[32] Fig. 1 is a schematic block diagram of a prior mobile phone with a character recognizing function.

[33] Fig. 2 is a schematic block diagram of a prior name card recognition engine;

[34] Fig. 3 is a flowchart illustrating a prior name card recognition process; [35] Figs. 4 and 5 are views of an example of a name card recognition process depicted in Fig. 3;

[36] Fig. 6 is a flowchart illustrating a name card recognition process according to an embodiment of the present invention;

[37] Figs. 7 and 8 a view illustrating a process for selecting a field in a name card recognition process according to an embodiment of the present invention;

[38] Fig. 9 is a view illustrating a process for selecting a plurality of field in order in a name card recognition process according to an embodiment of the present invention; and

[39] Fig. 10 is a view illustrating a recognition result of characters written on a name card according to an embodiment of the present invention. Best Mode for Carrying Out the Invention

[40] Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

[41] Fig. 6 shows a flowchart illustrating a name card recognition process according to an embodiment of the present invention.

[42] A user first take a picture of a name card using a camera associated with a mobile phone. At this select, it is assumed that a size of the name card is 176X144 in a preview state (S601). The name card image captured by the camera may be a portion of the name card or a whole portion of the name card. Fields are extracted from the name card image (S602). Then, the user selects fields, for which he/she wishes to perform a character recognition process, from the extracted fields (S603). The recognition process is performed only for the selected fields (S604). By doing this, an unnecessary recognition process can be omitted. At this select, one of high-speed and precise binary-coding methods is performed according to importance of the selected fields and difficulty in recognition. That is, the binary-coding method may be varied according to the fields.

[43] The high-speed binary-coding method is processed in a state where a size of the photographed image containing the selected fields is reduced. That is, by reducing the photographed image, the amount of calculation for the selected fields can be reduced in the binary-coding process. On the contrary, the precise binary-coding process method performs the binary coding for the selected fields without reducing the size of the pho¬ tographed image. Therefore, although the high-speed binary-coding method can quickly perform the binary-coding process, the recognition rate may be reduced. Although the precise binary-coding method can slowly perform the binary-coding process, the recognition rate may be increased.

[44] For the characters such as numbers, names and the like that are easy to recognize, the high-speed binary-coding method may be applied. However, for the characters such as symbols that are not easy to recognize, the precise binary-coding method may be applied. One of the high-speed and precise binary-coding methods may be designated for each field in advance. Alternatively, the user can select one of the high¬ speed and precise binary-coding methods for each field. For example, the important fields are designated with the precise binary-coding method in advance while the relatively less important fields are designated with the high-speed binary-coding method. Alternatively, the user designates one of the high-speed and precise binary- coding methods for each field.

[45] When the selected fields are recognized by the recognition engine, the recognition results are displayed on the screen of the mobile phone (S605). Next, it is determined if there is a need to further take a picture of the name card for additional fields that is to be added (S606). When there is a need to further take a picture, the above-described processes are repeated. This is generally required when the fields necessary for the user are existed on both surfaces of the name card. When it is determined that there is no need to further take a picture, it is determined if there are wrong characters in the recognized field that are selected in advance. When it is determined there are wrong characters, the user corrects the wrong characters (S607 and S610).

[46] When there is no wrong character, it is determined if there is a need to perform a re¬ capture for the name card. When it is determined that there is a need to perform the re¬ capture, the recognition results of the image that is early taken is saved in a temporary buffer (S608 and S611). The re-capture may be performed when the user determines that the image early taken is not the desired image. However, even when the re-capture is performed, the recognition results for the image early taken are saved in the temporary buffer. After the re-capture is performed, the above-described processes such as the field extraction, the field selection and the like are repeated to obtain a new recognition image (S611, S601 and S602).

[47] Lastly, the recognized character data are stored in a personal information-managing database such as a phone number database in response to proper fields (S609).

[48] As described above, when the recognized image obtained by performing one of the high-speed and precise binary-coding process for the selected fields, the addition pho¬ tographing process, the wrong character determining process, the retaking process are selectively performed before the recognition results are saved in the database. The recognition results may be saved in the form of a phone book so that the user can easily perform the data search, data edit, SMS transmission, phone call, group designation and the like. [49] Figs. 7 and 8 are views illustrating a designation of the fields in the name card recognition process according to an embodiment of the present invention.

[50] As shown in Fig. 7, the name card image captured by the camera and camera sensor is displayed on the screen of the mobile phone. For example, an image 701, an enlarged window 702, and a selection section 704 having a variety of selection fields are displayed on the screen. The selection section 704 includes a name, a phone number field, a mobile phone number field, an email address field, a facsimile number field, and a memo field to which a title, an address and the like can be added. When the name card is photographed and displayed, as shown in Fig. 8, the user extracts desired fields from the name card image. When the "name YU NAM KIM" is selected by moving the cursor 800, the "YU NAM KIM" is enlarged and displayed on the enlarged window 803. The user identifies one of the fields, which corresponds to the selected field, and selects the identified field. In the drawing, since the cursor 800 selects the "YU NAM KIM" which is the name, the user selects the number "1" so that the name field "1" recognizes the name "YU NAM KIM" and saves the same.

[51] Fig. 9 illustrates a process for selecting a plurality of field in order in a name card recognition process according to an embodiment of the present invention.

[52] As described in Fig. 9, when the cursor 900 selects the telephone number, the telephone number is enlarged and displayed on the enlarged window 902 and the user selects the telephone number field in the selection section 903 by selecting the number "2" so that the telephone number field "2" recognizes the telephone number and saves the same.

[53] Fig. 10 illustrates a recognition result of characters written on a name card according to an embodiment of the present invention.

[54] As shown in the drawing, the fields are selected and the recognition is realized for the selected field. That is, the name, telephone number, mobile phone number, facsimile number, and email address are recognized in the corresponding fields. In addition, a company name is recognized in the memo field. The recognized image is saved or it is determined if there is a need to further take a picture of the name card or to reselect the card image to add addition recognition fields.

[55] In the present invention, a spelling check function may be provided to correct the fields having a recognition error. In the drawing, possible words 1002 that can replace a wrong spelling "JIM" 1001. When the user selects a word "KIM," the wrong word "JIM" is changed into the correct word "KIM."

[56] It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents. Industrial Applicability

[57] The recognition results are saved in the data storage unit.

[58] According to the present invention, since the binary-coding process is performed only for the selected fields, the recognition speed and rate can be increased and the number of user's button manipulation can be reduced. Furthermore, since the spelling check function is provided for the recognition results, the recognition errors can be easily corrected.

[59] As described above, since the recognition process is performed only for the characters, for which the user wishes to recognize, the recognition efficiency and the user's convenience can be improved.

[60]

Claims

[I] A document image processing method, comprising: providing a document image; selecting fields, which is to be recognized, from the document image; and performing character recognition process for the selected fields.

[2] A document image processing method of a mobile phone, comprising: selecting fields, which is to be character-recognized, from a document image captured by a camera; performing a character-recognition process for the selected fields; and saving recognized character data.

[3] The document image processing method according to claim 2, wherein the document image is a name card image photographed by a camera associated with the mobile phone and the fields are character lines written on the name card.

[4] The document image processing method according to claim 2, wherein the selected fields are separated into different groups and different binary coding methods is applied for the two groups.

[5] The document image processing method according to claim 2, wherein the recognized characters are saved by corresponding fields by a personal in¬ formation-managing database.

[6] The document image processing method according to claim 2, further comprising correcting and editing the recognized characters when there is any error.

[7] The document image processing method according to claim 2, wherein possible words are proposed to correct a wrong spelled word so that the automatic correction can be realized by the user selecting one of the possible words.

[8] The document image processing method according to claim 2, wherein a plurality of document images for a single document can be photographed and recognition results of selected fields of each document images are handled as recognition results subordinated in the document.

[9] The document image processing method of claim 4, wherein the different binary- coding methods comprise a high-speed binary-coding method and a precise binary-coding method.

[10] The document image processing method of claim 4, wherein the different binary- coding methods are designated in advance or selected by a user.

[II] The document image processing method of claim 9, wherein the high-speed binary-coding method is performed by reducing the document image and performing the binary-coding process only for the selected fields.

[12] The document image processing method according to claim 9, wherein the precise binary-coding method performs the binary-coding process without reducing a size of the document image. [13] A document image processing method of a mobile phone, comprising: extracting fields from a document image; selecting fields, which is to be character-recognized, from the extracted fields; designating different binary-coding methods for the selected fields; performing the character-recognition process for the selected fields according to the designated binary-coding methods; and editing and storing recognized data. [14] The document image processing method according to claim 13, wherein the different binary-coding methods comprise a high-speed binary-coding method and a precise binary-coding method. [15] The document image processing method according to claim 14, wherein the high-speed binary-coding method is applied for the fields having a relatively high character recognition rate and the precise binary-coding method is applied for the fields having a relatively low character recognition rate. [16] The document image processing method according to claim 13, wherein the selected fields comprise a number, a character, a symbol, and a figure.