EP1553517B1 - Character recognition processing device, character recognition processing method, and mobile terminal device - Google Patents

Character recognition processing device, character recognition processing method, and mobile terminal device Download PDF

Info

Publication number
EP1553517B1
EP1553517B1 EP03784485A EP03784485A EP1553517B1 EP 1553517 B1 EP1553517 B1 EP 1553517B1 EP 03784485 A EP03784485 A EP 03784485A EP 03784485 A EP03784485 A EP 03784485A EP 1553517 B1 EP1553517 B1 EP 1553517B1
Authority
EP
European Patent Office
Prior art keywords
character
recognized
unit
recognizing
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP03784485A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1553517A1 (en
EP1553517A4 (en
Inventor
Yukihiro c/o Matsushita Elect. Ind.Co. Ltd FUJITA
Kouji Sawamura
Katsuyuki Ito
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Corp
Original Assignee
Panasonic Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Corp filed Critical Panasonic Corp
Publication of EP1553517A1 publication Critical patent/EP1553517A1/en
Publication of EP1553517A4 publication Critical patent/EP1553517A4/en
Application granted granted Critical
Publication of EP1553517B1 publication Critical patent/EP1553517B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/142—Image acquisition using hand-held instruments; Constructional details of the instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/1444—Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition

Definitions

  • the present invention relates to a character recognition processing device, a character recognition processing method and a portable terminal device that are preferably used for a portable terminal device such as a portable telephone, a portable information terminal or the like.
  • a feature amount is extracted from a character whose image is inputted and the character is recognized on the basis of the extracted feature amount.
  • the character whose image is inputted is scanned at prescribed intervals, the number of times of the intersections of scanning lines and the character are obtained. Further, the sum total of the differences of the obtained number of times of the intersections between the respective adjacent scanning lines is obtained as one of the feature amount to recognize the character. Scanning operations relative to the character to be recognized are carried out at prescribed intervals in at least two different directions or carried out at prescribed intervals in two directions intersecting at right angles with each other.
  • the outline data of inputted image data is formed.
  • Data obtained by normalizing the formed outline data is collated with dictionary data formed by extracting the outline data of a previously registered standard character to recognize a character.
  • the outline data of the image data is deformed to a typeface that can be recognized by the dictionary data or deformed so that the distortion is corrected before the data is collated with the dictionary data.
  • a character string whose characters are to be recognized is entirely photographed at a time
  • the number of pixels of a photographing device for instance, a CCD camera or the like
  • a host CPU or a MIPS (Million Instructions Per Second) value of a DSP or a work RAM undesirably needs to be increased.
  • the present invention is devised by considering the above-described circumstances and an object of the present invention is to provide a character recognition processing device, a character recognition processing method and a portable terminal device that can improve a rate of recognition and a recognition speed, can reduce the degree of a specification required for a device forming a photographing device or a system and can decrease consumed electric power and a cost.
  • WO 02/2557582 discloses a portable device that is capable of capturing imagery and is also capable of detecting and extracting text information from the captured imagery.
  • the text information which needs to be detected and processed is highlighted by a cursor control.
  • US 2001/0032070 A1 discloses digital video or still camera including optical character recognition and translator functions to translate text included in captured images. Cursor control allows the user to identify the desired text with the image. Translation is also possible on text included in images replayed from the camera's memory.
  • a character recognition processing device is defined in the claims.
  • the character image as the object to be recognized is displayed on a display part formed with a liquid crystal display element together with the character frame for recognizing the character. While a cursor of the character frame for recognizing the character is set to a character string as an object to be recognized, the character image is photographed by an image pick-up part such as a CCD camera. Then, the image data of the character image is fetched by the image fetching unit. The cursor position information is collated with the fetched image data of the character image by the layout analyzing unit to analyze the arrangement of lines or characters. The character is extracted and recognized relative to the character image by the character cuting unit and the character recognizing unit.
  • the layout analyzing unit the arrangement information of the lines or the characters of the character string as the object to be recognized can be precisely analyzed by using the cursor position information. As a result, a rate of recognition and a recognition speed can be improved. Further, even when the number of pixels of the image data for recognizing the character is not increased, the character can be highly accurately recognized by using the cursor position information. Accordingly, a specification of high performance that a photographing part has many pixels is not required and a specification required for the device can be reduced. Further, the throughput of the data in the device can be reduced, so that a specification necessary for a device forming a system such as a CPU, a DSP, a memory or the like can be reduced. Thus, the degree of the specification of each of parts of the device can be reduced, so that a consumed electric power and a cost can be lowered.
  • the image fetching unit fetches the image data of the character images respectively for prescribed areas from all of the character images.
  • the layout analyzing unit collates the cursor position information with each of the fetched image data of the plurality of character images to analyze the arrangement of the characters.
  • the image data of the character image is fetched respectively for prescribed areas from the entire part of the character image by the image fetching unit.
  • the cursor position information is collated with each of the fetched image data of the plurality of character images by the layout analyzing unit to analyze the arrangement of the lines or the characters.
  • the character image is divided into still images of some frames and the still images are fetched together with the cursor position information. Consequently, even when the character string as the object to be recognized is long, the arrangement information of the character string as the object to be recognized can be precisely analyzed by employing the cursor position information in the layout analyzing unit.
  • the image fetching unit fetches the image data of the character images respectively for prescribed areas from all of the character images.
  • the layout analyzing unit collates the cursor position information with the image data in which the plurality of fetched character images are connected together to analyze the arrangement of the characters.
  • the image data of the character image is fetched respectively for prescribed areas from the entire part of the character image by the image fetching unit.
  • the image data is collated with the cursor image information to analyze the arrangement of lines or characters by the layout analyzing unit.
  • the character image is divided into still images of some frames and the still images are fetched together with the cursor position information. Consequently, even when the character string as the object to be recognized is long, the arrangement information of the character string as the object to be recognized can be precisely analyzed by employing the cursor position information in the layout analyzing unit.
  • the present invention provides a portable terminal device having the character recognition processing device as described above.
  • the portable terminal device on which the character recognition processing device is mounted while a rate of recognizing and a recognition speed for recognizing a character is improved, a specification required for a device forming a system such as a photographing part, a CPU, a DSP, a memory or the like can be reduced.
  • a specification required for a device forming a system such as a photographing part, a CPU, a DSP, a memory or the like can be reduced.
  • the degree of the specification of each of parts of the device can be reduced and a low consumed electric power and a low cost can be realized.
  • a portable terminal device comprises a photographing unit for photographing a character image as an object to be recognized for recognizing a character; an image fetching unit for fetching the image data of the photographed character image; a cursor information output unit for outputting cursor position information showing the position of a character frame for recognizing the character coordinated with the character image; a layout analyzing unit for collating the cursor position information with the fetched image data of the character image to analyze the arrangement of the character; a character cutting unit for extracting the character image on the basis of the analyzed result of the layout analyzing unit; and a character recognizing unit for recognizing the extracted character image as the character and converting the character image to character information.
  • the layout of the character corresponding to the character frame for recognizing the character is analyzed on the basis of the image data of the character image and the cursor position information showing the position of the character frame for recognizing the character to recognize the character.
  • a specification required for a device forming a system such as a photographing part, a CPU, a DSP, a memory or the like can be reduced. Accordingly, the degree of the specification of each of parts of the device can be reduced and a low consumed electric power and a low cost can be realized.
  • a character frame display unit is further provided for displaying the character frame for recognizing the character overlapped on the character image upon photographing by the photographing unit.
  • a recognized character display unit is further provided for displaying the character information as a recognized result by the character recognizing unit.
  • the user can easily recognize the character information of the recognized result by a display.
  • the recognized character display unit individually selectively displays the character information as the recognized result for each of prescribed character units.
  • the user can individually select and utilize the recognized character information for each unit such as the types of the characters on the basis of the displayed character information.
  • a recognized character storing unit is further provided for storing the character information as the recognized result by the character recognizing unit.
  • the user can store and utilize the recognized character information in a desired form.
  • the recognized character storing unit stores the character information in a recognized character storing area.
  • the character information of a plurality of recognized results can be stored in the recognized character storing area and used.
  • the recognized character storing unit registers the character information in a data base corresponding to a type of each character information when the type of the character information is any of a telephone number, a mail address, a URL (Uniform Resource Location).
  • the character information can be registered in the data base such as a telephone directory or a book mark depending on the type of the character information and utilized.
  • a recognized character utilizing unit is further provided for utilizing the character information as the recognized result by the character recognizing unit in accordance with the type of the character information.
  • the user can effectively employ the character information for each of the types such as the telephone number, the mail address, the URL or the like in accordance with the type of the recognized character information.
  • the recognized character utilizing unit displays a transmitting screen to the telephone number.
  • the transmitting screen to the telephone number is displayed so that the user can simply carry out a transmitting operation.
  • the recognized character utilizing unit displays an electronic mail preparing screen to the mail address.
  • the electronic mail preparing screen to the mail address is displayed so that the user can simply prepare and transmit an electronic mail.
  • the recognized character utilizing unit displays a network connecting screen to the URL.
  • the network connecting screen to the URL is displayed so that the user can simply carry out a connecting operation to a network such as the internet to obtain desired information.
  • the photographing unit has a function for continuously photographing the character image as the object to be recognized.
  • the image fetching unit fetches the image data of the plurality of character images that are connected together.
  • the character recognizing unit recognizes the characters of the image data of the connected character images to convert the image data to the character information.
  • the character images are continuously photographed by coordinating the character images with the character frame for recognizing the character and the plurality of the character images thus obtained is connected together.
  • a rate of recognition and a recognition speed for recognizing the character can be improved.
  • the photographing unit has a function for continuously photographing the character image as the object to be recognized.
  • the image fetching unit fetches the image data of the plurality of character images.
  • the character recognizing unit recognizes the characters of the fetched image data respectively for the plurality of character images to convert the image data to the character information.
  • the character images are continuously photographed by coordinating the character images with the character frame for recognizing the character.
  • the characters of the image data are recognized respectively for the plurality of character images thus obtained.
  • the character recognizing unit has a plurality of recognizing modes that meet the types of the characters as the objects to be recognized and carries out a character recognizing process suitable for the corresponding type of the character in accordance with a preset recognizing mode.
  • the user sets the recognizing mode corresponding to the type of the character that is to be recognized by the user.
  • the character recognizing process suitable for the corresponding type of the character can be carried out to improve the rate of recognition.
  • a character recognition processing method comprises: a step of displaying and photographing a character image as an object to be recognized together with a character frame for recognizing a character; a step of fetching the image data of the photographed character image; a step of outputting cursor position information showing the position of the character frame for recognizing the character coordinated with the character image; a step of collating the cursor position information with the fetched image data of the character image to analyze the arrangement of the character; a step of extracting the character image on the basis of the analyzed result of the arrangement of the character; and a step of recognizing the extracted character image as the character and converting the character image to character information.
  • the arrangement information of lines or characters of a character string as an object to be recognized can be precisely analyzed by using the cursor position information to improve a rate of recognition or a recognition speed. Further, the cursor position information is used so that a specification required for each part of the device can be reduced. Since the throughput of the data in the device can be reduced, the degree of a specification of the device can be reduced and a low consumed electric power and a low cost can be realized.
  • the present invention provides a character recognition processing program in which the above-described steps are respectively executed by a computer.
  • the layout of the character corresponding to the character frame for recognizing the character can be precisely analyzed on the basis of the image data of the character image and the cursor position information showing the position of the character frame for recognizing the character to recognize the character. While the rate of recognition and the recognition speed for recognizing the character are improved, the degree of the specification of each part of the device can be reduced and a low consumed electric power and a low cost can be realized.
  • reference numeral 1 designates a portable terminal device.
  • 2 designates a character recognition processing device.
  • 3 designates an image fetching part.
  • 4 designates a cursor control part.
  • 5 designates a layout analyzing part.
  • 6 designates a character cutting part.
  • 7 designates a character recognizing part.
  • 8 designates a format output part.
  • 9 designates a photographing part.
  • 10 designates a display control part.
  • 11 designates an application part.
  • 12 designates a display.
  • 30 designates a portable telephone.
  • 33 designates a first display part.
  • 34 designates a camera.
  • 35 designates a second display part.
  • 36 designates an operating key.
  • 41 designates a photographing part.
  • 42 designates an operating part.
  • 45 designates an audio processing part.
  • 46 designates a storing part.
  • 47 designates a control processing part.
  • 51 designates a character recognizing part.
  • 52 designates a character using part.
  • 61 designates a temporarily storing part.
  • 62 designates a storing part.
  • 63 designates an output part.
  • 64 designates a type deciding part.
  • Fig. 1 is a block diagram showing the structure of a portable terminal device having a character recognition processing device according to a first embodiment of the present invention.
  • a structural example that the character recognition processing device 2 is mounted on the portable terminal device 1 is described.
  • the character recognition processing device 2 of this embodiment includes an image fetching part 3, a cursor control part 4, a layout analyzing part 5, a character cutting part 6, a character recognizing part 7 and a format output part 8. These component elements are realized by the operation of a software program, the combination of software and hardware or an exclusive hardware in a control processing part including a processor and a memory or the like.
  • the portable terminal device 1 includes a photographing part 9 composed of, for instance, a CCD camera, a display control part 10 and an application part 11 in addition to the character recognition processing device 2.
  • the photographing part 9 photographs an object whose character is to be recognized and fetches the object as photographed data to input the photographed data to an image fetching part 3 of the character recognition processing device 2.
  • the image fetching part 3 corresponds to an image fetching unit to convert the inputted photographed data to image data of a prescribed format and input the image data to the display control part 10.
  • the cursor control part 4 corresponds to a cursor information output unit to input cursor position information related to a cursor that functions as a character frame for recognizing a character to the display control part 10.
  • the display control part 10 converts the inputted image data and the cursor position information to display data, further processes the display data to a format suitable for a display 12 composed of a liquid crystal display element and outputs the processed display data to display information on the display 12.
  • the layout analyzing part 5 corresponds to a layout analyzing unit to receive the image data of the prescribed format from the image fetching part 3 and the cursor position information from the cursor control part 4 upon recognizing the character. Then, the layout analyzing part 5 collates the cursor position information with the image data to analyze a layout.
  • the character cutting part 6 corresponds to a character cutting unit to cut and extract the image data for a character unit on the basis of the analyzed result of the layout.
  • the character recognizing part 7 corresponds to a character recognizing unit to recognize the cut image data for a character unit and convert the image data to character data.
  • the format output part 8 converts the character data recognized in the character recognizing part 7 to a proper format.
  • the application part 11 uses the character data converted to the format in the format output part 8 as data for an application and inputs the data to the display control part 10 as required to display the data on the display 12.
  • FIG. 2 is an explanatory view of an operation at the time of a character recognizing process according to the first embodiment and shows a state that a character string as an object to be recognized is photographed.
  • a character string 17 as an object to be recognized in an object 16 whose character is to be recognized is photographed by the photographing part 9 of the portable terminal device 1 to recognize the character.
  • a cursor 18 as a character frame for recognizing a character is displayed together with the character string 17 as the object to be recognized. While a user adjusts a photographing position, the user sets the character string 17 as the object to be recognized to the cursor 18, the user operates an operating button 19 to photograph a still image under this state.
  • the photographed still image of the character string 17 as the object to be recognized is converted to the image data of a determined format and fetched in the image fetching part 3 and the image data is supplied to the layout analyzing part 5.
  • the layout analyzing part 5 receives the cursor position information from the cursor control part 4 to collate the cursor position information with the image data of a character image. In such a way, the layout analyzing part 5 can precisely analyze the arrangement information of lines and characters of the character string 17 as the object to be recognized by using the cursor position information.
  • the character cutting part 6 cuts the image data for a character unit.
  • the image data extracted for a character unit is converted to the character data to recognize the character.
  • the character recognition processing device of the first embodiment when the character of the character string 17 as the object to be recognized of the object 16 whose character is to be recognized is recognized, the image of the character string 17 as the object to be recognized and the cursor 18 as the character frame for recognizing the character are displayed together on the display 12. While the user sets the cursor 18 to the character string 17 as the object to be recognized, the still image is photographed. Thus, the image data of the photographed character image is fetched together with the cursor position information.
  • the layout analyzing part 5 the arrangement information of the lines or the characters can be easily and precisely analyzed by using the cursor position information so that a rate of recognition and a recognition speed can be improved in recognizing the character.
  • Figs. 3 and 4 are explanatory views of an operation during a character recognizing process according to the second embodiment.
  • Fig. 3 shows a state that a character string as an object to be recognized is photographed.
  • Fig. 4 shows a manner of the character recognizing process relative to the character string as the object to be recognized that is divided into a plurality of parts. Since the structure of the character recognition processing device is the same as that of the first embodiment, Fig. 1 is applied thereto.
  • a character string 23 as an object to be recognized in an object 22 whose character is to be recognized is photographed by a photographing part 9 of a portable terminal device 1 to recognize a character in the same manner as that of the first embodiment.
  • a photographing part 9 of a portable terminal device 1 to recognize a character in the same manner as that of the first embodiment.
  • a cursor 18 as a character frame for recognizing a character is displayed together with the character string 23 as the object to be recognized. While a user adjusts a photographing position, the user sets a part of the position of a start end of the character string 23 as the object to be recognized (a part of a left end in an example shown in Fig. 3 ) to the cursor 18. Under this state, an operating button 19 is operated to start a photographing operation of a continuous still image.
  • the user slides the portable terminal device to a direction opposite to the position of the start end upon starting a photographing operation (a right direction in the example shown in Fig. 3 ) in parallel with the character string 23 as the object to be recognized so as to be directed to a writing direction of a character.
  • a photographing operation a right direction in the example shown in Fig. 3
  • the photographing operation of the continuous still image is finished.
  • the user keeps pressing down the operating button 19, for instance, during the photographing operation of the continuous still image.
  • the continuous still image of the photographed character string 23 as the object to be recognized is divided into a plurality of image data and fetched in an image fetching part 3 as shown in Fig. 4 .
  • a character image of one line is divided into still images of some frames (1) to (4) and the still images are extracted from all of character images and converted to the image data of a determined format.
  • the image data is delivered to a layout analyzing part 5.
  • cursor position information is received from a cursor control part 4 to collate the cursor position information with the image data of the character image.
  • characters may not be possibly respectively accommodated in the character frame of the cursor with respect to the character images of each frame used for analyzing a layout. Accordingly, the cursor position information can be used without a problem to analyze arrangement information relative to a "line".
  • the cursor position information may not be used for analyzing the arrangement information for each character. In this case, the cursor position information may be shifted to the moving direction of the continuous still image to analyze the arrangement information of each character.
  • the arrangement information of the character string 23 as the object to be recognized can be precisely analyzed by employing the cursor position information.
  • the arrangement information of the lines can be precisely analyzed.
  • a character cutting part 6 the image data is cut for a character unit.
  • a character recognizing part 7 the extracted image data for a character unit is converted to character data to recognize a character.
  • "ABCDEFGHUJKL” is obtained in the frame (1).
  • "FGHIJKLMNOP” is obtained in the frame (2).
  • "JKLMNOPQRSTU” is obtained in the frame (3).
  • "OPQRSTUVW” ⁇ is obtained.
  • the recognized results of the characters respectively for the frames are synthesized so that "ABCDEFGHIJKLMNOPQRSTUVW" is obtained as a final recognized result of the characters.
  • the character recognition processing device of the second embodiment when the characters of the character string 23 as the object to be recognized of the object 22 whose character is to be recognized are recognized, the cursor 18 as the character frame for recognizing the character is displayed on the display 12 together with the image of the character string 17 as the object to be recognized. Then, the user moves the portable terminal device 1 to photograph the continuous still image. Then, the photographed character image is divided into some still images and the image data of each frame is fetched together with the cursor position information.
  • the layout analyzing part 5 the arrangement information of the lines can be easily and accurately analyzed by using the cursor position information, and a rate of recognition and recognition speed can be improved in recognizing a character.
  • the portable terminal device when the continuous still image photographing is carried out to obtain a plurality of image data, the portable terminal device is moved on the character string as the object to be recognized by keeping the operating button 19 of the portable terminal device 1 pressed down.
  • the present invention is not limited thereto. For instance, while the user moves the main body of the portable terminal device 1 to move a photographing range, the user may press down the operating button 19 each time to intermittently obtain the plurality of image data. The user may also obtain the image data of a continuous wide range as the image of the character string as the object to be recognized.
  • the layout analyzing part 5 may analyze the layout for each character image of a plurality of character images (still images) fetched by the image fetching part 3.
  • the layout analyzing part may temporarily connect the plurality of character images to one character image, and then, analyze the layout.
  • the character can be highly accurately recognized by using the cursor position information. Accordingly, a specification of high performance, that the number of pixels of the photographing part 9 is increased, is not required and a specification required for the device can be reduced. Further, the throughput of the data in the device can be reduced, so that a specification necessary for the device forming a system such as a CPU, a DSP, a memory or the like can be reduced. Accordingly the degree of a specification of each part of the device can be reduced and a low consumed electric power and a low cost can be realized.
  • Fig. 5 is a view showing the structure of an external appearance of a portable telephone according to the third embodiment of the present invention.
  • Fig. 5(A) is a front view
  • Fig. 5(B) is a rear view.
  • the portable telephone 30 of this embodiment includes a first casing 31 and a second casing 32.
  • a first display part 33 composed of a liquid crystal display element is provided on a front surface side of the first casing 31, that is, on a surface that mainly faces a user when the user opens the first casing 31 and the second casing 32 and holds them.
  • a first display part 33 composed of a liquid crystal display element is provided.
  • a camera 34 as a photographing part and a second display part 35 composed of a liquid crystal display element are provided in a rear surface side opposite to the front surface of the first casing 31, a camera 34 as a photographing part and a second display part 35 composed of a liquid crystal display element are provided.
  • an operating key 36 as an operating part having a plurality of key buttons such as an on-hook/off-hook key, a numeric character key, a function key or the like is disposed.
  • Various kinds of functions are assigned to the function keys PF1 to PF5 depending on operation modes or display screens of the device.
  • Fig. 6 is a block diagram showing the structure of the portable telephone according to the third embodiment.
  • the portable telephone includes a photographing part 41 having a function of a photographing unit for photographing the image of an object whose character is to be recognized, an operating part 42 for performing an operation and input by a user, a display part 43 having functions of a character frame display unit and a recognized character display unit for displaying various kinds of information such as character recognized results, a communication part 44 for performing a radio communication with a base station of a mobile communication system, an audio processing part 45 for performing an audio process of an audio speech or a call receiving sound, a storing part 46 having a function of a recognized character storing unit for storing various kinds of information and a control processing part 47 for carrying out a character recognizing operation, an application process such as the use of recognized character data or controlling respective parts.
  • a speaking voice is inputted, transmitted/received and outputted by the communication part 44, the audio processing part 45 and the control processing part 47.
  • a message such as an electronic mail is transmitted or received or various kinds of information are obtained by accessing a network such as an internet
  • character data is inputted and information such as the character data or image data is transmitted and received and displayed by the operating part 42, the display part 43, the communication part 44 and the control processing part 47.
  • the image is photographed, an object to be photographed is photographed to obtain the image data of a photographed image by the photographing part 41 and the control processing part 47.
  • the storing part 46 stores various kinds of information such as telephone directory registered information including the telephone number or the mail address of a communication mate, the character data such as an electronic mail, the image data such as the photographed image, audio data such as a call receiving sound.
  • Fig. 7 is a block diagram showing a functional structure related to a character recognizing process in the portable telephone according to the third embodiment.
  • a photographing part 41 and a character recognizing part 51 similar to those of the first and second embodiments are provided.
  • a character using part 52 is also provided.
  • the character using part 52 has a function of a recognized character utilizing unit for using character data as a recognized result in the function of each part is also provided.
  • the character recognizing part 51 and the character using part 52 are realized by hardware including a processor and a memory and a software program operating by using them.
  • Fig. 8 is an explanatory view showing a functional structure of the character using part in the structure shown in Fig. 7 .
  • the character using part 52 includes a temporarily storing part 61, a storing part 62 and an output part 63.
  • the temporarily storing part 61 is a recognized character storing unit to temporarily store the character data after the character recognizing process.
  • the storing part 62 is a recognized character storing unit and has a plurality of storing areas as a recognized character storing area to respectively store a plurality of character data.
  • the output part 63 has a type deciding part 64 for deciding the type of the character data to perform an output operation corresponding to the type of the character data of a recognized result.
  • the character data obtained by recognizing a character is a telephone number
  • an operation related to an audio calling to the telephone number or a registration in a telephone directory as a database is carried out.
  • the character data is a mail address
  • an operation related to a preparation of a mail to the mail address or the registration in the telephone directory is carried out.
  • the character data is a URL (Uniform Resource Locator)
  • an operation related to an internet connection for accessing the URL or a registration in a book mark as a database is carried out.
  • Fig. 9 is an explanatory view showing the flow of an entire process of the recognition of the character and the use of the character.
  • Fig. 9 shows the transition of each of operating states on a display screen displayed on the display part 43 as a center.
  • OCR Optical Character Recognition
  • a main menu screen 71 corresponding to an initial screen is displayed.
  • Fig. 11 shows one example of the main menu screen 71.
  • the list of the registered contents of the plurality of character data is displayed.
  • a plurality (for instance, 20) of character data of character strings to be recognized can be registered.
  • the photographing part 41 is activated to photograph the image of a new character string as an object to be recognized to become a waiting state for photographing by a camera.
  • a preview display screen 72 is displayed.
  • Fig. 12(A) shows one example of the preview display screen 72.
  • a character image 81 as an object to be recognized that is photographed by the photographing part 41 is displayed.
  • a character frame 82 for recognizing a character showing a character recognizing area is superimposed and displayed.
  • the user views the character image 81 and adjusts a photographing range so that the character string as the object to be recognized is accommodated in the character frame 82 for recognizing the character and presses down a "photographing" button assigned to the function key PF1
  • the character image 81 is photographed.
  • a series of character recognizing processes described in the first embodiment such as the analysis of the layout of the character string as the object to be recognized, cutting of the character, the recognition of the character or the like are carried out by the character recognizing part 51.
  • a mode shifts to a function setting mode of the photographing part 41 to display a camera function setting screen 73.
  • Fig. 12(B) shows one example of the camera function setting screen 73.
  • any of a recognizing mode, a brightness adjustment and a photo-light (photographing light) can be selected as a target to be set.
  • a recognizing mode setting screen 74 is displayed.
  • Fig. 12(C) shows one example of the recognizing mode setting screen 74.
  • the recognizing mode setting screen 74 when any of an automatic mode, a URL mode, a mail address mode and a telephone number mode is selected as the recognizing mode, a character recognizing process specified by the selected mode is carried out for all the character strings of the photographed character image.
  • a free mode When a free mode is selected, the character is recognized relative to an arbitrary character string as an object and all the character strings that can be recognized are outputted as recognized character data.
  • the character recognizing part 51 automatically decides a suitable recognizing mode depending on the contents of the character image 81 from among the four modes to recognize the character under the suitable recognizing mode.
  • the screen returns to the preview display screen 72.
  • the character recognizing process specified by the selected mode is carried out for all the character strings of the photographed character image.
  • the URL mode, the mail address mode and the telephone number mode is selected, only the character string corresponding to the URL or the like may be outputted as the recognized character data.
  • a recognized character display screen 75 is displayed.
  • Fig. 13(A) shows one example of the recognized character display screen 75.
  • the recognized character display screen 75 the character data of the character string as a recognized result is displayed.
  • the recognized character string includes the telephone number, the mail address and the URL
  • an anchor display is carried out in which the above-described items are respectively underlined to focus a selected character string.
  • the recognized character string is analyzed to identify the telephone number, the mail address and the URL. The telephone number, the mail address and the URL can be individually selected and used in accordance with the type of the character string.
  • a function menu display screen 76 is displayed.
  • Fig. 13(B) shows one example of the function menu display screen 76.
  • any of connecting, editing, copying, storing, newly registering in a telephone directory, additionally registering in a telephone directory and registering a book mark can be selected.
  • a function of "connecting" is selected in the function menu display screen 76.
  • the preview display screen 72 is displayed again so that the character string can be photographed.
  • the photographing and character recognizing processes of the character string as the object to be recognized are completed subsequently to the previous time, the preview display screen is switched to the recognized character display screen 75.
  • the character string recognized at this time is additionally displayed.
  • the character may be recognized for a plurality of images or after the plurality of images are connected together to one image, the character may be recognized.
  • a function screen 77 corresponding to each function is displayed so that each function can be performed.
  • a text editor is activated to display an editing screen.
  • the user can correct an erroneously recognized character in the editing screen.
  • a copying function is called to display a copying range selecting screen.
  • the user can copy the character data a range of which is designated and use the copied character data in other character input screen.
  • "storing" is selected, a storing process of the character data of the recognized character string is carried out.
  • a storing operation completion informing screen such as "stored” is displayed.
  • the recognized character string obtained by the character recognizing process at this time is stored in the column of the "non-registered" selected in the main menu screen 71.
  • a telephone directory registering screen is displayed so that the user can register the recognized character strings such as the telephone number, the mail address, a name, or the like in desired columns of the telephone directory.
  • a book mark registering screen is displayed so that the user can register the recognized character string such as the URL as a book mark.
  • a using screen 78 corresponding to the type of each of the character strings is displayed.
  • an audio transmitting screen 78a is displayed as the using screen.
  • an audio transmission to the telephone number whose character is recognized can be carried out by a simple operation.
  • the URL is selected, as shown in Fig. 13(D)
  • an internet connecting screen 78b is displayed as the using screen.
  • a mail preparing screen 78c is displayed as the using screen.
  • the character frame for recognizing the character is displayed on the preview display screen upon photographing the object to be recognized to photograph the character image of the object to be recognized.
  • the character of the image in the frame is recognized so that a load applied to the character recognizing process can be reduced.
  • the recognizing mode is selected to perform the character recognizing process in accordance with the type of a preset character as the object to be recognized, so that a character recognizing accuracy can be improved.
  • the character data of a plurality of types are arranged and displayed as the recognized results.
  • the character data can be divided for prescribed character units and individually selected, used and stored.
  • the recognized character data corresponding to the types of the character strings can be easily employed, for instance, the audio transmission in the case of the telephone number, the preparation of the mail in the case of the mail address or the like.
  • a character recognition processing device and a character recognition processing method can be provided which can improve a rate of recognition and a recognition speed, can reduce the degree of a specification required for a device forming a photographing device or a system and can decrease consumed electric power and a cost.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Character Discrimination (AREA)
  • Character Input (AREA)
  • Telephone Function (AREA)
EP03784485A 2002-08-07 2003-07-25 Character recognition processing device, character recognition processing method, and mobile terminal device Expired - Lifetime EP1553517B1 (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
JP2002230312 2002-08-07
JP2002230312 2002-08-07
JP2003133171 2003-05-12
JP2003133171 2003-05-12
PCT/JP2003/009483 WO2004015619A1 (ja) 2002-08-07 2003-07-25 文字認識処理装置及び文字認識処理方法、並びに携帯端末装置

Publications (3)

Publication Number Publication Date
EP1553517A1 EP1553517A1 (en) 2005-07-13
EP1553517A4 EP1553517A4 (en) 2007-10-03
EP1553517B1 true EP1553517B1 (en) 2009-12-09

Family

ID=31719845

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03784485A Expired - Lifetime EP1553517B1 (en) 2002-08-07 2003-07-25 Character recognition processing device, character recognition processing method, and mobile terminal device

Country Status (7)

Country Link
US (1) US7903875B2 (ja)
EP (1) EP1553517B1 (ja)
JP (1) JP4566740B2 (ja)
CN (1) CN100338619C (ja)
AU (1) AU2003252263A1 (ja)
DE (1) DE60330484D1 (ja)
WO (1) WO2004015619A1 (ja)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4036168B2 (ja) * 2003-09-09 2008-01-23 株式会社日立製作所 携帯電話
JP2005267000A (ja) * 2004-03-17 2005-09-29 Sony Corp 電子機器および機能割り当て方法
JP4587165B2 (ja) * 2004-08-27 2010-11-24 キヤノン株式会社 情報処理装置及びその制御方法
CN1787662A (zh) * 2004-12-07 2006-06-14 北京银河视讯传媒广告有限公司 控制移动终端显示的装置及方法
JP4453016B2 (ja) * 2005-01-31 2010-04-21 株式会社カシオ日立モバイルコミュニケーションズ 携帯端末、文字読取方法および文字読取プログラム
US20090251338A1 (en) * 2008-04-03 2009-10-08 Livescribe, Inc. Ink Tags In A Smart Pen Computing System
US9349046B2 (en) 2009-02-10 2016-05-24 Kofax, Inc. Smart optical input/output (I/O) extension for context-dependent workflows
CN101498592B (zh) * 2009-02-26 2013-08-21 北京中星微电子有限公司 指针式仪表的读数方法及装置
JP2010218322A (ja) * 2009-03-18 2010-09-30 Nec Corp 携帯通信端末及びそれに用いる通信先情報取得方法並びにそのプログラム
JP2010244211A (ja) * 2009-04-02 2010-10-28 Nippon Telegr & Teleph Corp <Ntt> 情報入力装置および情報入力方法
KR20100120753A (ko) * 2009-05-07 2010-11-17 (주)실리콘화일 문자인식을 위한 이미지 센서 및 이미지 센싱 방법
US8972563B2 (en) * 2010-11-30 2015-03-03 Red Hat, Inc. Updating changes to caches
US8358843B2 (en) * 2011-01-31 2013-01-22 Yahoo! Inc. Techniques including URL recognition and applications
TW201310355A (zh) * 2011-08-19 2013-03-01 Newsoft Technology Corp 經由資訊及指令關聯影像來瀏覽或執行指令的方法及其程式產品
CN102609735B (zh) * 2012-02-06 2014-03-12 安徽科大讯飞信息科技股份有限公司 一种字符书写规范度评测的方法和装置
JP5989479B2 (ja) * 2012-09-20 2016-09-07 シャープ株式会社 文字認識装置、文字認識装置の制御方法、制御プログラム、および制御プログラムを記録したコンピュータ読み取り可能な記録媒体
JP6080586B2 (ja) * 2013-02-13 2017-02-15 株式会社ファインデックス 文字認識システム、文字認識プログラム及び文字認識方法
CN104077582A (zh) * 2013-03-25 2014-10-01 腾讯科技（深圳）有限公司 访问互联网的方法、装置及移动终端
US20150006360A1 (en) * 2013-06-28 2015-01-01 Google Inc. Threshold Confidence Levels for Extracted Card Data
JP6525523B2 (ja) * 2013-07-31 2019-06-05 キヤノン株式会社 情報処理装置、制御方法およびプログラム
CN103729636B (zh) * 2013-12-18 2017-07-25 小米科技有限责任公司 字符切割方法、装置及电子设备
JP6303531B2 (ja) * 2014-01-22 2018-04-04 富士ゼロックス株式会社 画像処理装置及び画像処理プログラム
EP3132381A4 (en) * 2014-04-15 2017-06-28 Kofax, Inc. Smart optical input/output (i/o) extension for context-dependent workflows
CN105740863A (zh) * 2014-12-08 2016-07-06 阿里巴巴集团控股有限公司 信息处理方法及装置
US9852348B2 (en) * 2015-04-17 2017-12-26 Google Llc Document scanner
CN104850849A (zh) * 2015-04-27 2015-08-19 小米科技有限责任公司 发送文字的方法、装置及终端
CN105373791B (zh) * 2015-11-12 2018-12-14 中国建设银行股份有限公司 信息处理方法及信息处理装置
WO2018124849A1 (ko) * 2017-01-02 2018-07-05 삼성전자 주식회사 텍스트를 인식하는 방법 및 단말기
JP6448674B2 (ja) * 2017-01-26 2019-01-09 キヤノン株式会社 文字認識可能な画像を撮影するためのガイド表示を行う、カメラ機能を有する携帯可能な情報処理装置、その表示制御方法、及びプログラム
JP6889393B2 (ja) * 2017-03-08 2021-06-18 コニカミノルタ株式会社 画像処理装置、設定値設定方法および設定値設定プログラム
CN108200102A (zh) * 2018-03-15 2018-06-22 贵州电网有限责任公司 一种基于屏幕字符识别的数据安全获取系统
CN109934210B (zh) * 2019-05-17 2019-08-09 上海肇观电子科技有限公司 版面分析方法、阅读辅助设备、电路和介质

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07220025A (ja) 1994-01-31 1995-08-18 Canon Inc 画像処理装置
US5517586A (en) * 1994-08-16 1996-05-14 Wang Laboratories, Inc. Method and apparatus for automatically specifying a portion of text from a bitmap image of the text
JPH0962786A (ja) * 1995-08-28 1997-03-07 Oki Electric Ind Co Ltd 文字認識装置
JPH1069524A (ja) * 1996-08-28 1998-03-10 Sumitomo Metal Ind Ltd 文字切り出し方法
GB9711022D0 (en) * 1997-05-28 1997-07-23 Rank Xerox Ltd Text/image selection from document images
GB9809679D0 (en) * 1998-05-06 1998-07-01 Xerox Corp Portable text capturing method and device therefor
JP4128275B2 (ja) * 1998-07-06 2008-07-30 オリンパス株式会社 翻訳機能付カメラ
JP2000076378A (ja) 1998-08-27 2000-03-14 Victor Co Of Japan Ltd 文字認識方法
JP2000207487A (ja) * 1999-01-20 2000-07-28 Sony Corp 情報処理装置および方法、並びに提供媒体
JP4697819B2 (ja) * 1999-05-14 2011-06-08 ソニー株式会社 情報処理装置、情報処理方法及び記録媒体
JP2001008072A (ja) * 1999-06-22 2001-01-12 Casio Comput Co Ltd 電子カメラ及びその制御方法
JP2001060250A (ja) 1999-08-24 2001-03-06 Glory Ltd 文字認識方法および装置
US20010032070A1 (en) 2000-01-10 2001-10-18 Mordechai Teicher Apparatus and method for translating visual text
JP2002024762A (ja) * 2000-06-30 2002-01-25 Toshiba Corp 文書認識装置及びその方法
US7031553B2 (en) * 2000-09-22 2006-04-18 Sri International Method and apparatus for recognizing text in an image sequence of scene imagery
WO2002025575A2 (en) 2000-09-22 2002-03-28 Sri International Method and apparatus for portably recognizing text in an image sequence of scene imagery
JP2002111841A (ja) * 2000-10-04 2002-04-12 Sony Corp 携帯通信端末および文字読取装置
JP4095243B2 (ja) * 2000-11-28 2008-06-04 キヤノン株式会社 Ｕｒｌ取得および処理システムおよびその方法およびその処理を実行するプログラムを記憶した記憶媒体。
US20020131636A1 (en) * 2001-03-19 2002-09-19 Darwin Hou Palm office assistants
JP4240859B2 (ja) * 2001-09-05 2009-03-18 株式会社日立製作所 携帯端末装置及び通信システム
US7343049B2 (en) * 2002-03-07 2008-03-11 Marvell International Technology Ltd. Method and apparatus for performing optical character recognition (OCR) and text stitching

Also Published As

Publication number Publication date
EP1553517A1 (en) 2005-07-13
AU2003252263A1 (en) 2004-02-25
CN100338619C (zh) 2007-09-19
US20060177135A1 (en) 2006-08-10
JP4566740B2 (ja) 2010-10-20
EP1553517A4 (en) 2007-10-03
US7903875B2 (en) 2011-03-08
CN1675652A (zh) 2005-09-28
WO2004015619A1 (ja) 2004-02-19
DE60330484D1 (de) 2010-01-21
JPWO2004015619A1 (ja) 2005-12-02

Similar Documents

Publication Publication Date Title
EP1553517B1 (en) Character recognition processing device, character recognition processing method, and mobile terminal device
US9589198B2 (en) Camera based method for text input and keyword detection
US20050221856A1 (en) Cellular terminal image processing system, cellular terminal, and server
KR100767921B1 (ko) 휴대 단말 장치 및 휴대 전화기
US7623742B2 (en) Method for processing document image captured by camera
JP3443146B2 (ja) 画像合成方法および画像記録装置
JP4467519B2 (ja) 携帯情報端末
US20060290789A1 (en) File naming with optical character recognition
JP4453016B2 (ja) 携帯端末、文字読取方法および文字読取プログラム
JP2005065286A (ja) カメラ付き携帯用端末機での住所録管理装置及び方法
US20080018772A1 (en) Input apparatus for image
JP2000032111A (ja) 携帯型情報読取処理装置及び制御方法
JP4251643B2 (ja) 画像撮像装置、および、プログラム
CN109617946A (zh) 将图像中自定义文字识别结果数据自动上传的方法及拍照设备
JP2001008072A (ja) 電子カメラ及びその制御方法
US7336319B2 (en) Digital camera apparatus having a recognizing function
JP2003069925A (ja) 付帯情報入力方法および装置並びにプログラム
JP3772367B2 (ja) 情報処理装置
JP2019186766A (ja) 画像処理装置、操作支援方法および操作支援プログラム
JP5999582B2 (ja) 情報出力装置及びプログラム
WO2005071944A1 (ja) 文字読み取り装置及び画像表示方法
JPS6120040A (ja) 電子式写植システムにおける画像合成方法
JP2004355518A (ja) 携帯通信機器
JP2000236426A (ja) ファクシミリ装置
JP2010218322A (ja) 携帯通信端末及びそれに用いる通信先情報取得方法並びにそのプログラム

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050207

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

DAX Request for extension of the european patent (deleted)
RBV Designated contracting states (corrected)

Designated state(s): DE FR GB

RIN1 Information on inventor provided before grant (corrected)

Inventor name: SAWAMURA, KOUJI

Inventor name: FUJITA, YUKIHIROC/O MATSUSHITA ELECT. IND.CO.,LTD

Inventor name: ITO, KATSUYUKI

A4 Supplementary search report drawn up and despatched

Effective date: 20070905

RIC1 Information provided on ipc code assigned before grant

Ipc: G06K 9/22 20060101AFI20070830BHEP

Ipc: G06K 9/20 20060101ALI20070830BHEP

Ipc: H04M 1/00 20060101ALI20070830BHEP

Ipc: G06K 9/03 20060101ALI20070830BHEP

Ipc: H04M 1/56 20060101ALI20070830BHEP

Ipc: G06K 9/34 20060101ALI20070830BHEP

17Q First examination report despatched

Effective date: 20080102

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: PANASONIC CORPORATION

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 60330484

Country of ref document: DE

Date of ref document: 20100121

Kind code of ref document: P

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20100910

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20120725

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20120718

Year of fee payment: 10

Ref country code: FR

Payment date: 20120719

Year of fee payment: 10

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20130725

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20140331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130725

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140201

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60330484

Country of ref document: DE

Effective date: 20140201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130731