JP2006227768A - Method and program for character recognition in portable terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem to read a character string of a language like Japanese in which columnar writing and lateral writing are mixed in real time with a portable terminal with low CPU performance and limited available memory even if a user does not specify the direction of character string. <P>SOLUTION: An automatic determination means for the character line direction is set up as preprocessing of character recognition in a portable terminal. In the automatic determination means for the character line direction is provided with a reading area extraction means to detect figures, photographs, ruled lines, and blanks and extract an area of images as a reading area excluding the above. A character line direction determination means is also made to perform determination of character line direction in the area extended vertically and horizontally in a prescribed extent centering on the position specified in imaging the character string to be read in order to operate in high speed and saving memory. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a portable terminal having a camera, a user interface thereof, and a character recognition technology.

  In recent years, small mobile phones, PDAs (Personal Digital Assistance) and the like (hereinafter collectively referred to as mobile terminals) have become widespread. In mobile terminals, it is often necessary to input characters for e-mail creation, address registration, memo writing, schedule registration, and the like. A key is generally used as a means for inputting characters. However, a key input on a small portable terminal requires a user to press the key many times, which places a heavy burden on the user.

  Japanese Patent Laid-Open No. 2003-78640 (Patent Document 1) describes a method using character recognition as a means for replacing key input. In this method, a character string to be input is captured as an image with an attached camera, a character string to be read is designated, the character string is read using a character recognition technique, and the read character string is input.

However, in Japanese, both vertical writing and horizontal writing are used. Therefore, prior to character recognition, it is necessary to designate in advance the direction of the character string, whether to read a vertically written character string or a horizontally written character string.

  As described above, even if the user does not specify the direction of the character string, a character string in a language such as Japanese, in which vertical writing and horizontal writing are mixed, has a low CPU processing capacity and a small amount of memory that can be used. Reading in real time with a terminal is a problem. Conventionally, there has been a technique for determining whether a document is written vertically or horizontally from an image. However, in the conventional method, it is necessary to process a wide range of images, and it has been difficult to apply to a portable terminal with limited memory capacity and processing capability.

  In order to solve the above-described problem, in the present invention, a character line direction automatic determination unit is provided as preprocessing for character recognition in a portable terminal. The automatic character line direction determining means is provided with a reading area extracting means for detecting a figure, a photograph, a ruled line or a blank and extracting an image area excluding these as a reading area.

  Further, the character line direction determining means determines the character line direction in a region that is widened in a predetermined range from the top, bottom, left, and right centering on the position specified when imaging the character string to be read so as to operate at high speed and save memory. To do. The position to be specified is specified by aligning the marker with the position of the character string to be read as shown in Japanese Patent Application Laid-Open No. 2003-78640 (Patent Document 1).

  Further, the character line direction automatic determination means uses less area by using an area where the reading area extracted by the reading area extraction means coincides with an area expanded from a designated position to a predetermined range. The processing speed can also be increased.

  By providing the reading area extracting unit and the automatic character line direction determining unit, a language such as Japanese that has a low CPU processing capacity and a small amount of memory that can be used at a high speed, memory saving, and vertical writing and horizontal writing. Can be read in real time.

  According to the present invention, a character string in a language such as Japanese that uses both vertical writing and horizontal writing on a portable terminal with low CPU power and a small amount of memory that can be used does not have to set the direction of the character string to be read in advance. However, it is possible to recognize the character string by automatically determining the direction of the character string at a high speed and saving memory.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a data flow showing an embodiment of a portable terminal to which the present invention is applied. The program 101 of the present invention receives an image from a camera mounted on a portable terminal as input, reads a word in a document copied in the image, and displays the result. The reading area extraction module 102 inputs an input image, detects a figure, a photograph, a ruled line, and a blank area in the input image, extracts only the area to be read, and outputs the extracted area coordinates. The character line direction determination module 103 inputs an input image and a reading area, determines whether the character line is vertical writing or horizontal writing, and outputs information indicating whether the character line is vertical writing or horizontal writing. The character line extraction module 104 extracts the character line by inputting the input image, the character line direction, and the reading area coordinates, and outputs the character line area. The character line display module 105 inputs an input image, a character line direction, and a character line area, and displays them on the display. The character string reading module 106 recognizes a character string by inputting the character line area when the user confirms that the character line direction and the character line area are correct with respect to the input image. Output the result. Here, when the character line direction and the character line area displayed by the character line display module 105 are a character string that the user wants to read, the user instructs an instruction to perform character string recognition on the character line. The display module 107 inputs the input image and the character string reading result and displays them on the display.

  FIG. 2 is a block diagram illustrating a hardware configuration. The portable terminal 201 includes a camera 202 that captures a character string to be read, a CPU 203 that performs processing operations of the program of the present invention, an image captured by the camera 202, a program of the present invention, a character identification dictionary, and a character for character string matching. Memory 204 for storing the column dictionary, display of the character string to be read, display of the character line extraction result, display of the character string recognition result, determination of imaging of the camera 202 and confirmation of the character line extraction result, and character string recognition It consists of a button 206 that conveys the result confirmation to the program.

  FIG. 3 is a flowchart showing a schematic processing procedure of the character recognition program shown in FIG. In the image input process 301, an image captured by the camera is taken into the internal memory. In the image display process 302, the image captured in the image input process 301 is displayed on the display. In a reading area extraction process 303, a figure, a photograph, a ruled line, or a blank is detected from the image captured in the image input process 301, and only the reading area excluding these areas is extracted. In the character line direction determination process 304, the character line direction is determined from the reading area extracted by the reading area extraction process 303 whether the character line is vertical writing or horizontal writing.

  In the character line extraction process 305, a character line is extracted based on the reading area extracted in the reading area extraction process and the character line direction determined in the character line direction determination process 304. In the character line area display process 306, the character line area extracted by the character line extraction process 305 is displayed on the display in the form of a rectangle on the image displayed by the image display process 302. At this time, in order to clarify whether the character line direction is vertical writing or horizontal writing, the color of the rectangle of the character line area may be changed between vertical writing and horizontal writing. In the character line determination judgment 307, an input as to whether or not the character line area displayed in the character line area display process 306 is a character string that the user wants to read is accepted. 307 is performed. Otherwise, the process returns to the image input process 301. In the character string reading process 308, a character string is read from the character line extracted by the character line extraction process 305. The character line extraction process 305 and the character string reading process 307 can be realized by using, for example, the method disclosed in Patent Document 1 based on the character line direction determined by the character line direction determination process from the position specified by the user. In the reading result display process 309, the character string read in the character string reading process 308 is displayed on the display. In the determination or end determination 310, the user confirmed and read the input image displayed in the image display process 302, the character line area displayed in the read character line display process 306, and the character string displayed in the character string read result display process 309. When the character string is determined or the end of the character recognition program is selected, the process ends. Otherwise, the process returns to the next image input process 301.

  FIG. 4 is a diagram showing details of the reading area extraction processing. Reference numeral 401 denotes an input image, and reference numeral 402 denotes a marker that is displayed so as to be superimposed on the image when the user designates the reading area. A portion having the same coordinates as the marker is determined as a temporary center of the reading area. Like 403, a figure and a photograph are detected from the input image 401, and it excludes from a reading area. Further, as indicated by 403, ruled lines 405 are detected from the area 404 of the image on the designated coordinate side and excluded from the reading area. Also, as indicated by 405, a blank 407 is detected from the image area 406 on the designated coordinate side and excluded from the reading area. In this way, the reading area 408 is determined. Note that it is not necessary to use all the above items to be excluded from the reading area, and other conditions may be used together.

  FIG. 5 is an example of a detailed data flow of the reading area extraction process. A figure / photo detection 501 detects a figure / photo area from the input image. For example, a circumscribed rectangle of a connected component of pixels is used to detect a connected component having a size equal to or larger than a certain size. However, it is assumed that the connected component close to the designated coordinates is not handled as a figure or a photograph. The ruled line detection 502 detects a long ruled line in an area other than the area detected by the figure / photo detection 501 from the input image. The ruled line may be detected by detecting a connected component having a certain aspect ratio or more of the circumscribed rectangle of the connected component of the pixel. The area outside the image from the detected ruled line is outside the reading area. In the blank detection 503, a blank area is detected from an area other than the figure or photograph area detected by the figure / photo detection from the input image and the ruled line area detected by the ruled line detection 502. Here, the order of the above three detections may be changed.

  FIG. 6 is an example of a detailed data flow of character line direction determination processing. In order to perform character line direction determination at high speed and in a memory-saving manner, in the character line direction determination area limiting process 601, an area widened in a predetermined range from the reading position designation coordinate in at least one of up, down, left, and right directions is input Take out. Further, the coordinates of the reading area are used for the coordinates of the reading area where the coordinates of the reading area are closer to the reading position designation coordinates in an area expanded in at least one of the upper, lower, left and right directions within a predetermined range. The predetermined range extending from the reading position designation coordinate to the top, bottom, left, and right is sufficiently small with respect to the size of the input image (to the extent that a sufficient reduction in the amount of processing computation can be expected as compared with the entire processing target). A certain ratio (for example, about 4 characters should be about 3 lines. If 10 lines are included in the image at the time of importing, if about 4 characters are about 3 lines, the input image will be 4 minutes each vertically and horizontally. 1).

  In the histogram generation process 602, pixel histograms are generated in the vertical direction and the horizontal direction for the area limited by the character line direction determination area limiting process 601. Here, the coordinates of the circumscribed rectangle of the connected component instead of the pixel may be used for generating the histogram. In that case, the height value of the circumscribed rectangle of the connected component is added in the case of the histogram in the vertical direction, and the width value of the circumscribed rectangle of the connected component is added in the case of the histogram in the horizontal direction. Character line direction determination processing 603 determines whether a character line is vertical or horizontal from the vertical and horizontal histograms and designated coordinates.

  The criteria for determining the direction follow the following order of conditions. Note that the valleys of the histogram under the following conditions are locations where the value in the histogram is equal to or less than the threshold value. The threshold varies depending on the amount of noise in the input image.

(1) If the maximum pixel value in the vertical histogram is wMax and the maximum pixel value in the horizontal histogram is hMax,
When wMax> hMax × a, vertical text line,
When hMax> wMax × a, horizontal text line,
And Here, a is an arbitrary value, for example, 1.3.

(2) If the average width of the valleys in the vertical histogram is wWAve and the average width of the valleys in the horizontal histogram is hWAve,
When wWAve> hWAve, vertical text line
When hWave> wWAve, horizontal text line
And

(3) When the number of valleys in the vertical histogram is wNum and the number of valleys in the horizontal histogram is hNum,
When wNum <hNum, vertical text line,
When hNum <wNum, horizontal text line,
Here, in the conventional method, since the entire document image captured by the camera is targeted for character recognition, the entire document image is analyzed for a so-called document layout such as a paragraph or a column and the direction is determined for each character line. Although it is necessary, in the process assumed in the present invention, it is only necessary to know the direction of the character string designated by the user. Therefore, it is not necessary to analyze the entire document image, and character line direction determination may be performed only near the reading designated position. For the determination of the character line direction, if there are about three character lines of about four characters, the character line direction can be determined with sufficient accuracy. If there are about 10 character lines in the input image, it is considered that if the size of the input image is about a quarter each, then about 3 lines of 4 characters will be included. As a result, even in a portable terminal having a smaller memory capacity or arithmetic processing capability than in the conventional method, the character line direction determination and the character recognition can be performed with high accuracy.

  If neither of the above conditions can be determined, the process returns to the character line direction determination area limiting process 601 to further widen the area extending from the reading position designation coordinate to at least one of up, down, left, and right, and the histogram generation process 602 and the character line direction. The determination process 603 is repeated. The number of repetitions varies depending on the CPU power. The size that extends from the reading position designation coordinate to at least one of up, down, left, and right is about 1.5 times larger than the area that could not be determined.

  FIG. 7 is a diagram showing the character line direction determination processing shown in FIG. Reference numeral 701 denotes an input image. Reference numeral 702 denotes a reading area obtained by reading area extraction processing. Reference numeral 703 denotes a read character string designation position that designates a character string that the user wants to read. Reference numeral 704 denotes a character line determination limited area that is spread from the position of the read character string designated in 703 to the upper, lower, left, and right sides. Reference numeral 705 denotes the coordinates of the reading area 702 and the character line determination limited area 704, which is an AND area of the reading character string designation position 703. A histogram is taken for the area 705. Reference numeral 706 denotes the result of taking a pixel histogram in the horizontal direction with respect to the area 705. Reference numeral 707 denotes a result of taking a histogram of pixels in the vertical direction with respect to the area 705. In the input image 701, from the histograms 706 and 707, according to the result of using, for example, (2) of the condition in the description of 603 in FIG. Determined.

  FIG. 8 is a display example 1 of a vertically written character string reading result. An image and a character line 802 are displayed on the display 801. The character string reading result display 803 is displayed vertically when it is determined that the character line direction is determined to be vertically written.

  FIG. 9 is a display example 2 of the vertically written character string reading result. An image and a character line 902 are displayed on the display 901. The character string reading result display 903 is displayed in horizontal writing even when vertical writing is determined in the character line direction determination. Further, in order to distinguish from the result of reading a horizontally written character line, the color of the character line 902 is changed between the case of reading a vertically written character string and the case of reading a horizontally written character string.

  FIG. 10 is a display example of a horizontally written character string reading result. An image, a character line 1002, and a character string reading result display 1003 are displayed on the display 1001.

It is a data flow which shows the data flow of the program of this invention. It is a block diagram of hardware. It is a flowchart which shows the general | schematic process sequence in the program of FIG. It is a figure which shows the reading area extraction process. It is an example of the data flow of reading area extraction processing. It is an example of the data flow of a character line direction determination process. It is a figure which shows the character line direction determination process which FIG. 6 shows. It is the example 1 of a display of the vertical character string read result. It is the example 2 of a display of the vertical character string read result. It is an example of a display of the horizontal character string reading result.

Explanation of symbols

  102: Reading area extraction module, 103 Character line direction determination module, 104: Character line extraction module, 105: Character line display module, 106: Character string reading module, 107: Processing result display module

Claims (4)

A program executed in a character recognition device having a camera that inputs electronic data of a document image, a display unit that displays the input document image and a processing result of the document image, and a processing operation unit, In the calculation part,
A document image obtained by capturing a paper surface of a document and a reading position designated by a user are input, and a reading area extraction process for extracting a reading target area from the document image and a direction of a character line in the reading target area are determined. A character line direction determining process, a character line extracting process for extracting a character line from the reading target area according to a character line direction determining result, and a character string reading process for reading a character in the extracted character line;
In the character line direction determination process, a character recognition program for performing processing by extracting an area sufficient for determining a character line direction around a reading position designated by the user.   The character recognition program according to claim 1, wherein the reading target extraction processing is to select a divided region including a reading position after dividing the image based on a blank portion in the image.   The character recognition program according to claim 1, wherein the reading target extraction process is to select a divided region including a reading position after dividing an image by linear components in the image.   In addition to the configuration of claim 1, the apparatus has a character line extraction result display process for displaying a character line extraction result superimposed on a document image, and when there is no reading instruction, an image input, a reading area extraction process, and a character line direction determination. A character recognition program which repeats processing, character line extraction processing, and character line extraction result display, and executes character string reading processing when there is a reading instruction.

Images