JP2003331217A - Character input device, character input method and character input program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a character input device capable of realizing input by accurate character recognition even when a high-quality photographed image can not be acquired because of resolution of a camera, deformation of a lens or the like. <P>SOLUTION: This device is equipped with a data processing device 50 comprising a character extraction device 51 for extracting an image of a character part from an image photographed by a photographing device 10, a character recognition device 52 for performing character recognition from the image of the character part, and adding reliability to the character recognition result to acquire character recognition data, and a recognition data synthesis device 53 for detecting and connecting overlapped parts in the character recognition data acquired from a plurality of images in the same document, selecting highly- reliable character recognition data relative to the overlapped parts, and thereby synthesizing a plurality of the character recognition data into one synthesized recognition data, and also equipped with a recognition data storage device 60 for storing the synthesized recognition data and a display device 40 for displaying the photographed image or the character recognition result. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a character input device, a character input method, and a character input program, and more particularly to a character input device and a character input device capable of accurately inputting a large amount of character information using a photographing device such as a camera. The present invention relates to an input method and a character input program.

[0002]

2. Description of the Related Art When a document is photographed by a device such as a digital camera and the data is taken into a computer, the resolution of this type of camera is limited. However, if it is captured, there is a problem that the characters cannot be identified. For this reason, in the past, when it was desired to capture and capture a document containing a large amount of text information at high resolution, instead of capturing the entire document at once, dividing the document into several areas The method is to take a magnified image by dividing it into two parts, and then combine them after importing them into the computer.

As a conventional character input method, for example, Japanese Patent Laid-Open No.
There are some disclosed in 2000-278514 and Japanese Patent Laid-Open No. 10-91722. In these character input methods, characters are recognized from an image taken by a handy scanner or the like instead of a camera, the characters are aligned by a character code, and then the images are combined to obtain the entire image. These methods enable effective input for high-quality images taken with a handy scanner or the like. However, for images taken with the camera in your hand, the magnification may change depending on the distance to the paper surface, or the image may tilt depending on how you hold the camera.
Satisfactory results cannot be obtained with these conventional methods.

When inputting a document into a computer, it is often important that the text information, not the image, is important. Therefore, a method of obtaining character information of the entire document by combining not the image but the character recognition result has been considered. For example, in Japanese Patent Laid-Open No. 11-196255 and Japanese Patent Laid-Open No. 11-232378, character recognition is performed after a high-quality camera-captured image is obtained by a method such as automatically moving the camera or correcting the captured image. , A method of synthesizing character information by determining overlapping of character recognition results is disclosed.

However, in the above-described method, the divided photographing is performed in a predetermined order, and the character recognition and the composition are performed at the end, so that the result of the character recognition is also confirmed at the end. For this reason, it takes a lot of time and effort to check the entire recognition result with the original document, and even if the captured image is saved and checked on the screen, all the images are saved. Storage capacity is required. Further, even when an error is noticed, it is necessary to select the character to be corrected by some means and perform the correction work in order to correct it, which is also a heavy burden on the user.

[0006] Further, when capturing characters in a computer, it is considered that there are many occasions where a character string (URL, mail address, etc.) of a part of a document is input and used.
As a means for photographing and inputting a part of a character string in a document with a camera, there is a conventional technique such as Japanese Patent Laid-Open No. 2000-207338. This is to input a partial character string on the screen by performing character recognition processing only on an area set by a user using a pointing device or the like on an image captured by a camera.

However, this does not consider inputting a long character string that does not fit on one screen. Even if the images are divided into a plurality of times and the images are connected later, it is troublesome to specify the area of the character string to be input for each image pickup.

Also, the case where the character string to be input in the document has a line break in the middle is not taken into consideration, and it is necessary to combine them into one character string by using the document editing software or the like after photographing them separately. It was

Further, when extra characters are printed adjacent to each other before and after the character string to be input in the document, it is necessary to strictly adjust the photographing range to the input character string so that these characters are not photographed, which is troublesome. Met.

[0010]

The above-described conventional character input method has the following problems.

The first problem is that, in order to obtain an image without distortion or distortion, the photographing method must be devised, or the user must perform image processing on the photographed image to correct it.

The reason for this is that an image taken by a camera has a low resolution and is distorted and is not suitable for character recognition, unlike the case where a device dedicated to capturing a document image such as a scanner is used. Further, as described in the second problem, it is difficult to correct the erroneous recognition, so that it is necessary to acquire an image more suitable for the recognition.

The second problem is that it is difficult to confirm and correct a recognition error.

The reason is that the character recognition is performed after the entire document is divided and photographed, so that it takes a lot of time and effort to check the recognition error, and even if the error is noticed, the character is designated by some means and the correct character is recognized. This is because the troublesome work of selecting or inputting is required.

A third problem is that when a document is divided and photographed, if a photographed image that does not completely overlap the photographed image taken up to that time is photographed, it cannot be combined correctly. That is the point.

The reason is that the information of the overlapping portions of the respective images is used when the results of the divided photographing are combined.

The fourth problem is that, in the case of divisional photographing, the work of re-holding the camera and pressing the shutter button many times each time the photographing is performed is very burdensome for the user.

The reason is that in order to divide and photograph a document having a large area, it is necessary to reposition the camera and press the shutter many times by the number of divisions, which is very troublesome and time-consuming. This is because of this. In addition, there is a problem that the operation of pressing the shutter button causes camera shake.

The fifth problem is that it is not suitable for inputting a part of a long character string from a document.

The reason is that the input of a long character string that cannot be photographed at one time is not taken into consideration. Even if the photograph is divided into several times, the photographed image is recorded every time the photograph is taken. You must specify the input string of
Because it is troublesome.

The sixth problem is that it is difficult to input a character string having a line break in the middle.

The reason is that it is not taken into consideration to input a line-broken character string, and it is necessary to input them separately and then combine them with character editing software or the like.

The seventh problem is that if an extra character string is printed closely before and after the character string to be input, the input is troublesome and the transfer is necessary.

The reason is that it is necessary to move the camera to finely adjust the photographing range and to strictly specify the processing range so that an extra character string is not input.

An object of the present invention is to realize accurate character recognition input when a character is input using a camera, even if a high-quality photographed image due to camera resolution, lens distortion, etc. cannot be obtained. It is to propose a character input device, a character input method, and a character input program.

Another object of the present invention is to provide a character input device, a character input method, and a character input device capable of inputting characters without wasteful work such as correcting a photographed image by image processing in order to obtain an image suitable for recognition. To propose a character input program.

Another object of the present invention is to provide a character input device and a character input method which can easily correct an input error caused by erroneous recognition of a character when inputting a character using a camera.

Still another object of the present invention is to use the image without wasting it even if a region having no overlapping portion is photographed by the divided photographing when inputting characters using the camera. It is to propose a character input device, a character input method, and a character input program that can be performed.

Still another object of the present invention is to propose a character input device, a character input method, and a character input program that eliminate the need for a series of operations such as re-holding and pressing the shutter button many times when dividing and photographing a document. To do.

Still another object of the present invention is to provide a character input device, a character input method, and a character input device capable of continuously capturing a long character string without designating an object each time a character is input using a camera. To propose a program.

Still another object of the present invention is to provide a character input device, a character input method, and a character input program which can easily input a character string printed with a line feed in the middle when a character is input using a camera. To propose.

Still another object of the present invention is to finely adjust the shooting range so as not to shoot an extra character string printed closely in front and back when inputting characters using a camera.
It is to propose a character input device, a character input method, and a character input program that do not require a strict specification of the processing range.

[0033]

SUMMARY OF THE INVENTION To achieve the above object, the present invention provides a character input device for inputting characters by extracting and recognizing characters from a photographed image of a document by a photographing device. In addition, the reliability indicating the certainty of recognition is added and stored, and by comparing the character recognition results from images obtained by repeatedly photographing the whole or a part of the document, the recognition result of the character with the highest reliability can be obtained. Characteristic is based on.

According to the character input device of the present invention of claim 2, in the character input device for inputting characters by extracting and recognizing characters from a captured image of a document by the image capturing device, the image capturing device captures an image. A character extracting unit that extracts an image of a character portion from an image, a character recognizing unit that performs character recognition from the image of the character portion, and adds the reliability to the character recognition result to obtain character recognition data; Of the character recognition data obtained from the image of the character recognition data is detected and connected, and by selecting the character recognition data having high reliability with respect to the overlapped portion, a plurality of the character recognition data are combined into one character recognition data. And a recognition data storage unit for storing the combined character recognition data.

According to the character input device of the third aspect of the present invention, the reliability is at least information on the degree of matching between the dictionary referred to in character recognition and the character extracted from the photographed image, and the distortion of the photographed image. It is characterized in that it is determined based on.

According to the character input device of the present invention of claim 4, the recognition data synthesizing means relates to the character recognition data obtained from a plurality of images of the same document, and the character recognition obtained from the latest image. The reliability of the character recognition data obtained from the latest image is detected by detecting the overlapping portion of the data and the character recognition data obtained from the previous image stored in the recognition data storage means. If the value is high, the character recognition data obtained from the previous image in the recognition data storage means is rewritten.

According to the character input device of the present invention of claim 5, at least one or more other recognition candidates of the character designated from the characters stored as the character recognition data are read out from the recognition data storage means and presented. The character recognition data of the specified character is replaced with the character recognition data of the character selected from the recognition candidates, thereby correcting the character recognition data.

According to the character input device of the present invention of claim 6, the character correction means recognizes the character of the character positioned near the center of the photographed image among the character recognition data displayed on the display means. The feature is that data is selected as a character to be corrected.

According to the character input device of the present invention of claim 7, the recognition data synthesizing means relates to the character recognition data obtained from a plurality of images of the same document, and the character recognition obtained from the latest image. Detecting an overlapping portion between the data and the character recognition data obtained from the previous image stored in the recognition data storage means, and if the overlapping portion is not detected, the character recognition obtained from the latest image It is characterized in that the data is stored as the reserved data, and the character recognition data obtained from the next input image is also detected as an overlapping portion with the reserved data.

According to the character input device of the present invention as defined in claim 8, the moving amount of the photographing device is detected from the difference between continuously input images, and the stationary state of the photographing device is detected based on the moving amount. It is characterized in that it is provided with a means for automatically discriminating the image capturing instruction of the image.

According to the character input device of the present invention of claim 9, an image temporary storage means for temporarily storing a photographed image by the photographing device, an image stored in the image temporary storage means and the photographing device. An image difference detecting unit that detects a difference from an input image, determines that the photographing device is in a stationary state when the detected difference is equal to or less than a predetermined threshold, and sends a photographing command to the photographing device. It is characterized by including.

According to the character input device of the tenth aspect of the present invention, the image difference detecting means determines that the photographing device is in a stationary state when the difference is equal to or less than a predetermined threshold for a predetermined time. It is characterized by making a judgment.

According to the present invention of claim 11, in a character input method for inputting characters by extracting and recognizing characters from a photographed image of a document by a photographing device, the recognition result of the characters indicates the recognition accuracy. It is characterized in that the recognition result of the character having the highest reliability is selected by adding the reliability indicating the likelihood and comparing the character recognition results from the images obtained by repeatedly photographing the whole or a part of the document.

According to the twelfth aspect of the character input method, the reliability is based on at least the degree of coincidence between the dictionary referred to in character recognition and the character extracted from the photographed image and information on the distortion of the photographed image. It is characterized by setting.

According to the character input method of the thirteenth aspect, the overlapping portions of the character recognition data obtained from a plurality of images of the same document are detected and connected, and the character recognition data having high reliability with respect to the overlapping portion is detected. It is characterized in that a plurality of the character recognition data are combined into one of the character recognition data by selecting.

According to the character input method of claim 14, regarding the character recognition data obtained from a plurality of images of the same document in the recognition data composition, the character recognition data obtained from the latest image and the character recognition data previously obtained Of the character recognition data obtained from the image, the character recognition obtained from the previous image when the reliability of the character recognition data obtained from the latest image is higher. Characterized by rewriting data.

According to the character input method of claim 15, by designating the character stored as the character recognition data, another candidate of the designated character is presented, and the character recognition of the character selected from the candidates is presented. It is characterized in that the character recognition data is corrected by replacing the character recognition data of the designated character with data.

According to the character input method of claim 16, correction is possible by moving the cursor to the character recognition data of the character located near the center of the photographed image among the displayed character recognition data. It is characterized by showing that it is a highly character.

According to the character input method of claim 17, regarding the character recognition data obtained from a plurality of images of the same document in the recognition data composition, the character recognition data obtained from the latest image and the character recognition data previously obtained The overlapping portion with the character recognition data obtained from the image is detected, and if the overlapping portion is not detected, the character recognition data obtained from the latest image is stored as reserved data, and then input. It is characterized in that the character recognition data obtained from the image is also detected in an overlapping portion with the reserved data.

According to the character input method of the eighteenth aspect, the moving amount of the photographing device is detected from the difference between continuously input images, and the stationary state of the photographing device is determined based on the moving amount. It is characterized by automatically capturing images.

According to a nineteenth aspect of the character input method, an image photographed by the photographing device is temporarily stored, and a difference between the temporarily stored image and the image input from the photographing device is detected. When the detected difference is less than or equal to a predetermined threshold value, it is determined that the photographing device is in a stationary state, and a photographing instruction is sent to the photographing device.

According to the character input method of the twentieth aspect, when the difference is equal to or less than a predetermined threshold value for a predetermined time, it is determined that the photographing device is in a still state.

According to a twenty-first aspect of the present invention, in a character input program for inputting a character by performing extraction on a captured image of a document by a photographing device and recognition of the character, the recognition result of the character is obtained. , A function of adding a reliability indicating the certainty of recognition and a function of comparing character recognition results from images obtained by repeatedly capturing the whole or a part of the document, and selecting a recognition result of the character having the highest reliability. It is characterized by having.

According to a thirtieth aspect of the present invention, in the character input device of the present invention, a mark drawing means for drawing a mark indicating a position of a specified specific character string in a photographed image by the photographing device, and the photographed image for each line. It is characterized by further comprising a line extracting unit that divides the line image into line images and sends the divided line images to the character extracting unit in order from the line image closest to the mark.

In the character input device of the present invention according to claim 31, the recognition data synthesizing means is obtained from the character recognition data obtained from the latest image and the previous image stored in the recognition data storing means. Further, it is characterized by comprising a forcible connecting means for forcibly connecting the character recognition data in accordance with an instruction from the user when an overlapping portion with the character recognition data is not detected.

According to a thirty-second aspect of the present invention, in the character input device of the present invention, based on the character recognition data stored in the recognition data storage means, it is determined whether or not it is the character recognition data by the first shot image, and it is determined whether it is the first time. For example, based on the position control means that clearly indicates the position of the end of the recognized character string at the beginning of the recognized character string, and the position at the beginning or end of the recognized character string,
And a deleting unit that deletes a character of the character recognition data.

[0057]

DETAILED DESCRIPTION OF THE INVENTION A first embodiment of the present invention will be described in detail below with reference to the drawings.

Referring to FIG. 1, a character input device 1 according to the first embodiment of the present invention includes a photographing device 10 for inputting an image and a data processing device 50 for performing a process for inputting a character from the input image. A recognition data storage device 60 for storing the input and character-recognized data, an input device 20 for a user to operate the device, a display device 40 for outputting a process or a result, and a display device 40. A display image control device 30 for controlling an output image is provided.

The data processing device 50 is realized by a program-controlled computer, and the character extracting device 51.
And a character recognition device 52 and a recognition data synthesizing device 53.

The photographing device 10 is a photographing means realized by a CCD or the like, and normally outputs the imaged image to the display image control device 30, but when a photographing instruction is input,
A still image is taken and the image is sent to the character extracting device 51.

The input device 20 inputs an instruction from the user to the photographing device 10, the data processing device 50 and the display image control device 30.

The display image control device 30 is connected to the input device 20, the photographing device 10, the character recognition device 52, and the recognition data storage device 60, and displays any of the data sent from each device on the display device 40. It is controlled according to the command input from the input device 20.

The display device 40 displays the image data sent from the display image control device 30.

The character extracting device 51 of the data processing device 50 is connected to the photographing device 10 and performs image processing on the image photographed by the photographing device 10 to extract a character image for each character to extract the character image. Send to 52.

The character recognition device 52 performs character recognition processing on the character image generated by the character extraction device 51,
The recognition result obtained for each character and the information such as reliability to be described later are included and output as character recognition data.

The recognition data synthesizing device 53 synthesizes the data by detecting the overlapping portion of the two data of the character recognition data input from the character recognizing device 52 and the synthetic recognition data input from the recognition data storage device 60. Then, the combined result is output to the recognition data storage device 60 as combined recognition data.

The recognition data storage device 60 stores the combined recognition data sent from the recognition data combining device 53.

Next, the overall operation of the character input device 1 according to the present embodiment will be described in detail with reference to the flowcharts of FIGS.

First, when the present apparatus starts its operation, a captured image (moving image) from the image capturing apparatus 10 is displayed on the display image control apparatus 3
It is displayed on the display device 40 through 0 (step A
1).

When a photographing instruction to photograph an image is input from the input device 20 by pressing the photographing button or the like (step A2), a still image is photographed, and the image at that time is displayed on the display device 40 and the data processing device 50. Is sent to the character extracting device 51 (step A3).

When the character extracting device 51 receives the image, it performs image processing to extract (cut out) a character-by-character image from the image and send it to the character recognizing device 52 (step A4). The character extraction process is, for example, Japanese Patent Application No. 06-2.
It can be realized by applying the technique described in Japanese Patent Publication No. 85930. If the extraction of the character image from the captured image fails, the camera image output is resumed (step A5).

The character recognition device 52 is, for example, Japanese Patent Application No. 06-301.
This is realized by using the conventional technique as described in Japanese Patent No. 802, character recognition is sequentially performed on character-by-character images sent from the character extraction device 51, and recognition is performed on a predetermined number of candidates obtained as a recognition result. Information such as reliability is output as character recognition data (step A6).

That is, the character recognition data output as the character recognition result of a certain character includes a predetermined number of candidate data, and also includes reliability information indicating the certainty of character recognition. In addition, the number of candidate data may be one (first candidate) or plural (second candidate or less). That is, when there are a plurality of candidates, the character recognition data output as the character recognition result of a character includes not only the data of the first candidate of the character but also a predetermined number of candidates equal to or less than the second candidate. The data is included, and the reliability information indicating the certainty of character recognition is included.

Here, the reliability of recognition means the character recognition device 5
2 the matching condition between the character template included in the dictionary referred to and the character image obtained by shooting (obtained based on the distance between the character spaces of the character images in character recognition),
It is a value that indicates the certainty of character recognition that is comprehensively obtained from the position on the image (for example, the distance from the center of the captured image of the character) and the information about the distortion that the lens has. The higher the value, the higher the result of character recognition. Let's be certain.

For example, in the information on the position on the image, which is a factor for determining the reliability of recognition, the image is more likely to be distorted as the character is farther from the center of the photographed image. Therefore, when the distance from the center is small, the reliability is high. Is higher, and the reliability is lower as the distance from the center is larger.

Further, as a factor for determining the reliability of recognition, it is possible to add information on the photographing time in a series of character input operations of a certain document. When a character is photographed a plurality of times, the reliability of the character is higher as the recognition processing is performed on the image photographed later.

The character recognition data obtained by the character recognition device 52 is sent to the recognition data synthesis device 53.

The recognition data synthesizing device 53 has a large print text in which both the synthetic recognition data input so far and stored in the recognition data storage device 60 and the character recognition data newly input in this photographing are both present. Assuming that a part of the image is photographed in an overlapping manner, the overlapping portion is discriminated and detected (step A8).

In the determination of the overlapping portion, not only the first character candidate but also a plurality of character candidates included in the synthetic recognition data and the character recognition data (a predetermined number of candidates below the first candidate) are determined. By doing so, more accurate results can be obtained.

If the synthetic recognition data does not exist in the recognition data storage device 60, the recognition data synthesis device 53 sends the new character recognition data as it is to the recognition data storage device 60 as the synthetic recognition data (step A
7).

If the overlapping portion is not detected even if the composite recognition data exists, it is determined that the character recognition data cannot be composited, and the process returns to the camera image output (step A9).

If an overlapping portion is found, the position of the overlapping portion is used as a clue to determine the relationship between the shooting positions of the combined recognition data and the input character recognition data, and the input character recognition data is combined with the combined recognition data. To do. At that time, the character of the overlapping part is determined by referring to the reliability of the character, and the character recognition data of the newly captured part (the part not included in the composite recognition data) of the input character recognition data is combined. Add to recognition data. For the overlapping part (the part included in the synthetic recognition data), the synthetic recognition data and the character recognition data are compared for each character, and the synthetic recognition data has a lower reliability of recognition. Replace with something. (Steps A10, A1
1).

The combined recognition data obtained by combining the character recognition data in this manner is sent again to the recognition data storage device 60 and stored and updated (step A12).

The composite recognition data input so far is output to the display device 40 through the display image control device 30 (step A13).

The character recognition data combining processing in steps A10 and A11 works effectively in both cases where the combined recognition data and the character recognition data partially or entirely overlap. Therefore, not only when connecting the images obtained by split shooting, but also when shooting again the place where it was once shot, or when shooting the whole part at once and then taking a close-up part of it again, It is possible to improve the recognition accuracy of the entire input sentence by synthesizing the character recognition data having high reliability from both of them in a form giving priority.

Next, the effect of the input device according to the present embodiment, which is constructed and operates as described above, will be described.

In the present embodiment, the recognition reliability is set for each character at the time of character recognition, and the recognition reliability is saved together with the character recognition result. Then, if the result obtained by recognizing the new captured image is higher in reliability than the stored combined recognition data, the combined recognition data is updated with the recognition result having high reliability. Image deterioration caused by photographing with the photographing device 10 (camera) is often local such as screen edge distortion and hand shadow, and images near the center of the screen have little distortion and high recognition reliability. It is expected. Since the input result of such a highly reliable portion remains (is stored) during continuous shooting, the user does not have to worry about the image quality of each shot image or a part of distortion.

Further, even if there is a recognition error, the character will be replaced correctly if the image photographed with good image quality is input again. Therefore, the user re-photographs the entire image due to some misrecognition. You can greatly save the trouble of making corrections such as inputting correct characters later.

Next, a first embodiment of the present invention will be described with reference to the drawings. The first example corresponds to the first embodiment of the present invention.

In this embodiment, the character input device is realized as the portable terminal 100 as shown in FIGS. FIG. 12 is a perspective view of the mobile terminal 100 viewed from the front,
FIG. 13 is a perspective view seen from the back.

The portable terminal 100 includes buttons 111 as the input device 20, a liquid crystal display 112 as the display device 40, and a camera 11 as the photographing device 10.
Equipped with 3. Further, a central processing unit that functions as the data processing device 50 and a storage device that functions as the data storage device 60 are provided inside.

In the present embodiment, consider a case where a sentence 200 as shown in FIG. 14 is photographed by the portable terminal 100 and input. Here, since the resolution of the camera 113 is low, it is necessary to take close-up pictures to some extent in order to photograph individual characters at a resolution that allows character recognition, and the range that can be photographed at one time (photographing range PR) is the dotted line It is the size represented by a rectangle. In other words, it is not possible to shoot and recognize the entire sentence at once.

Therefore, first, as shown in FIG. 15, the left half of the sentence 200 is photographed.

The captured image data is stored in the portable terminal 100.
A character image is extracted by a data processing device 50 incorporated in the character recognition device, and thereafter character recognition is performed character by character, and the character image and the reliability of recognition are output as character recognition data. FIG. 16 shows an example of the recognition result of the character string on the second line. The recognition result character of FIG. 16 represents the character indicated by the character code of the first recognition candidate of the extracted character. For the sake of simplicity, only the result of the second row will be shown, but the same processing is performed for the other rows.

This is an input by the first photographing, and the synthetic recognition data is not stored in the recognition data storage device 60. Therefore, the data processing device 50 stores the current character recognition data as it is in the recognition data storage device 60 as the combined recognition data, and the recognition result is displayed on the liquid crystal display 112.

At this time, as a display method, for example, it is preferable to output a screen in which the photographed image and the recognized characters are in one-to-one correspondence on the screen because the user can easily confirm the erroneous recognition.

Subsequently, the right half of the document 200 is photographed as shown in FIG. At this time, it is necessary to shoot so that there may be an overlap with the previously captured text image.

The photographed image data is subjected to character extraction by the character extraction device 51 of the data processing device 50 as shown in FIG. 18, and is converted to character recognition data via the character recognition device 52.

Here, the recognition data synthesizing device 53 of the data processing device 50 selects all the synthetic recognition data (character "abcdefghj") input and stored last time and the character recognition data (character "fghijklmn") input this time. The method of duplication is checked, and the method of duplication having the largest number of matching characters for the most duplicated portion is searched. 4 characters to the right of the data you entered last time,
In the case where the left four characters of the data input this time overlap, the matching ratio is the highest with three out of the four characters, and it is determined that this is the overlapping portion.

Thereafter, as shown in FIG. 19, the character recognition data (the character recognition data input this time) is combined with the combined recognition data (the character recognition data input and stored last time).

First, for the right two characters "hi" of the overlapping four character parts, the one of the character recognition data having high recognition reliability is adopted. Similarly, for the left two characters "fg", the one of the synthetic recognition data with high recognition reliability is adopted. As a result, the character "j" that was erroneously recognized the first time was corrected to "i" by the second shooting.

Next, the portion of "jklmn" which is a portion newly photographed by this photographing is connected to the end of the combined recognition data.

The data synthesized (corrected / concatenated) as described above is again stored in the recognition data storage device 60 and displayed as an input result on the liquid crystal display 112.

Here, since there is an error in the input result, the user can try the correction by photographing the character again.

FIG. 20 shows a state in which the portable terminal 100 is moved so that the character "b" is at the center of the screen and re-photographing is performed in order to correct the mistaken recognition of b as 6. The closer the camera image is to the center, the less the distortion due to the lens, and the focus is also generally centered. Therefore, by shooting at the center of the screen, an image suitable for character recognition can be obtained.

The photographed image data is subjected to character extraction, character recognition, conversion to character recognition data shown in FIG. 21, and then combination with combined recognition data.

Here, since all the character recognition data of this time are included in the combined recognition data, no new character is added to the combined recognition data, and the overlapping portion is processed.

As shown in FIG. 22, the synthetic recognition data and the character recognition data are compared, and “bc” having a lower character recognition reliability in the synthetic recognition data is replaced with that of the character recognition data.

By the above operation, the synthesized recognition data can be stored in the recognition data storage device 60 by dividing and photographing the sentence 200 which cannot be photographed once. Moreover, the recognition error could be corrected by retaking the document 200.

In this example, the image is photographed twice, but in the case of a sentence printed on a wider area, it is possible to input by changing the number of divisions according to the size of the sentence. is there.

Further, in this example, the image is divided horizontally and photographed. However, even in the case of a vertically long sentence, it is possible to similarly vertically divide and photograph and combine.

In addition, this embodiment uses the camera 1 having a low resolution.
Although an example in which a large document is divided and captured using 13 is described, the input method realized in the present embodiment is not limited to this.

For example, if the camera 113 has a high resolution and a mechanism capable of changing the focal length automatically or manually, first, as shown in FIG.
The whole can be photographed.

If there is an erroneous recognition, it is possible to try the correction by taking a close-up so that the erroneously recognized character becomes the center as shown in FIG. A character image taken in close-up and having a large size has a high resolution and a high degree of reliability in character recognition, and thus is likely to be replaced correctly.

Next, a character input device according to the second embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 3, the character input device 1A according to the second embodiment of the present invention is capable of correcting characters by operating the input device 20 in addition to the configuration of the first embodiment shown in FIG. It differs from the first embodiment in that the character correction device 25 for performing is connected to the recognition data storage device 60. The character correction device 25 has a function of directly correcting the combined recognition data stored in the character correction device 25 recognition data storage device 60.

The operation of the character input device 1A according to this embodiment will be described with reference to the drawings. FIG. 4 is a flowchart showing the operation of the character input device 1A according to the second embodiment. Here, in the operation of the present embodiment, the difference from the operation of the first embodiment of the present invention shown in FIG. 2 is that, as shown in FIG. 4, the recognition by the character correction device 25 after step A13. This is that a recognition result correction process (step B1) for correcting the result is added. Therefore, description of steps common to those in FIG. 2 is omitted.

The operation of the recognition result correction processing (step B1) by the character correction device 25, which is a feature of this embodiment, will be described in detail with reference to FIG.

When the recognition result correction process is started, a cursor for recognition result correction is displayed at the position of the recognition result corresponding to the character extracted at the position closest to the center in the last photographed image (step C1). When the user tries to make a correction and re-shoots, the character to be corrected often comes near the center of the re-shot screen, so by displaying the cursor on the character extracted near the center of the re-shot screen, the user can It is possible to omit or reduce the operation of moving the cursor while searching for the character to be corrected.

Then, this cursor can be moved on the displayed document of the recognition result by using the input device 20 (steps C2 and C3).

When the input device 20 is operated to indicate the display of the next candidate, the next recognition candidate of the character designated by the cursor is retrieved from the combined recognition data stored in the recognition data storage device 60 and displayed. Yes (steps C4, C
5). As described in the first embodiment, when there are a plurality of candidates, the character recognition data (also the synthetic recognition data) output as the character recognition result of a character is the first candidate of the character. Data of a predetermined number of candidates below the second candidate. The candidates below the second candidate included in the list are searched and presented.

Alternatively, even when a character is directly input from the input device 20, the input character is displayed as a candidate instead of the character designated by the cursor (step C).
6, C7).

When the input device 20 is operated to indicate a candidate decision, the displayed recognition candidate is overwritten on the corresponding synthetic recognition data in the recognition data storage device 60 as a correct recognition candidate. If no candidate is determined, the process returns to the candidate selection (steps C8-C9).

The correction process is continued until an input is made from the input device 20 to end the correction process (step C).
10).

These cursor movement, candidate selection, character input, etc. can be realized by, for example, a small keyboard with up, down, left, right arrow keys, or
It can also be realized by a device such as a touch panel and a pen that can directly specify characters on the display device 40.

Next, the effect of the second embodiment constructed and operated as described above will be described.

In the present embodiment, as described above,
It was possible to select the next candidate for recognition of the specified character or to select the correct recognition candidate and overwrite it on the corresponding synthetic recognition data, so I tried to correct the character recognition error again, but it was not corrected. Even in such a case, the correct character can be easily input. Also, since the cursor is automatically displayed at the position of the character that was re-photographed in the center of the screen when the user tried to correct it, there is no need to search for the corrected character and manually move the cursor. Can be done easily.

Next, a second embodiment of the present invention will be described in detail with reference to the drawings. This example corresponds to the second embodiment.

This embodiment has the same configuration as the first embodiment and is realized by the portable terminal 100 device shown in FIGS. 12 and 13, but in that the central processing unit functions as the character correction device 25. different.

In order to input the sentence 200 shown in FIG. 14 as in the first embodiment, first, the left half is photographed as shown in FIG. 15, and then the right half is photographed as shown in FIG. Then, it is assumed that the character recognition data is combined, and further, the character "b" is photographed with the character "b" at the center as shown in FIG. 20 in order to correct an error in which the character "b" is recognized as the character "6". .

However, unlike the case of the first embodiment,
It is assumed that the character “b” is erroneously recognized as the character “6” even in the second shooting and is not corrected.

Here, on the liquid crystal display 112, as shown in FIG. 25, the photographed image 214 and the input result 21 are displayed.
Both 5 are displayed. Further, since the image was taken centering on the character "b", the square cursor 216 is located at the corresponding character "6" in the input result.

Here, the user can display the next candidate for character recognition by pressing the next candidate display button provided on the buttons 111. It is assumed that the character recognition candidate “b” and the reliability of the character recognition corresponding thereto are stored in the form as shown in FIG.

Since the first candidate is being displayed now, by pressing the next candidate button twice, the third candidate "b" is displayed on the screen.

Here, when the user presses the enter button provided on the buttons 111, the recognition data storage device 60 stores that the character "b" is the correct recognition result.

Next, a character input device according to the third embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 6, a character input device 1B according to a third embodiment of the present invention has a reserved data storage device 70 in addition to the configuration of the first embodiment shown in FIG. Is different.

The reserved data storage device 70 is connected to the recognition data synthesizing device 53 and stores the input character recognition data when the overlapping portion of the character recognition data and the synthetic recognition data is not detected and the synthesizing is not performed. It is to be stored temporarily. Then, when the next shooting is performed and the new character recognition data is combined with the combined recognition data, the combined recognition data and the character recognition data returned from the reserved data storage device 70 to the recognition data combining device 53 again. The synthesis process is tried again.

Next, the operation of the character input device 1B according to the third embodiment will be described in detail with reference to the drawings.

Referring to FIG. 7, steps A1 to A8 and steps A10 to A in the operation of the present embodiment.
The operation of No. 13 is the same as the operation of the first embodiment shown in FIG.

In step A9, when the overlapped portion of the combined recognition data and the character recognition data is searched for, the overlapped portion is found. Then, the process proceeds to step A10 to perform the combining. However, if the overlapped portion is not found, the overlapping portion is found. The character recognition data is reserved data, and the reserved data storage device 70
And is returned to the camera image output (step D2).

Then, after succeeding in synthesizing another character recognition data and the synthetic recognition data (steps A10 to A1).
3) If the reserved data is stored in the reserved data storage device 70, the data is regarded as a new character string and the combining process is performed again. When the hold data is not stored, the process returns to the camera image output (step D
1).

Next, the effect of the third embodiment will be described.

In the present embodiment, the reserved data storage device 7
Even if the input data fails to be merged because 0 is provided and the overlapping part is not found, it is stored as pending data until the data can be merged by another shooting. Even if the user moves his hand too much and shoots an image with no overlap, the character recognition data recognized up to that point can be used for data synthesis without wasting.

Next, a third embodiment of the present invention will be described with reference to the drawings. Such an example corresponds to the third embodiment of the present invention.

Like the first embodiment, this embodiment is realized by the portable terminal 100, but is different in that the storage device also functions as the reserved data storage device 70.

Now, in order to input the print text shown in FIG. 14 as in the first embodiment, first the left half is photographed as shown in FIG. 27 and then the right half is photographed, but the terminal is moved to the right. It is assumed that the image is photographed as shown in FIG. 28 because it has moved too much. This cannot be combined because there is no overlap with the first input result. Therefore, the character recognition data for this input is temporarily stored in the reserved data storage device 70.

Next, if the user takes a picture for the third time as shown in FIG. 29, since the picture is taken at the same time as the first picture, it can be combined.

Here, with respect to the first and third combining results, the second pending character combining data is tried to be combined. In this case, since there is an overlapping portion, the composition is successful and the entire sentence can be input.

Next, a character input device according to the fourth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 8, a character input device 1C according to a fourth embodiment of the present invention is the same as the first embodiment shown in FIG. Difference in having 90.

The image temporary storage device 80 is used in the image pickup device 1.
The image output from the image capturing apparatus 10 is temporarily stored in each frame when connected to 0.

The difference detection device 90 is the image temporary storage device 8
0 and the image capturing device 10 are connected to each other, and the amount of movement of the image capturing device 10 is detected by obtaining the difference between the image one frame before stored in the image temporary storage device 80 and the latest frame image.
A still image is photographed by determining the photographing timing from the movement of the photographing device 10 and sending a photographing command to the photographing device 10.

Next, the operation of the character input device 1C according to the fourth embodiment will be described in detail with reference to the drawings.

The operation of this embodiment shown in FIG. 9 differs from the operation of the first embodiment shown in FIG. 1 in that instead of step A2, step E1 and step E2.
This is the point where processing is performed.

In the present embodiment, while the image output from the photographing device 10 is displayed on the display device 40, the image temporary storage device 80 and the image difference detection device 90 described above are used to change the previous frame for each frame. The difference amount of
Then, it is determined from the difference amount whether or not the photographing device 10 is stationary (step E1).

When it is determined that the photographing device 10 is stationary, the image difference detection device 90 issues a photographing instruction to the photographing device 10 (step E2), and the process proceeds to step A3 to automatically photograph the image. It That is, the shooting is automatically performed without performing the shooting instruction operation (shutter) by the input device 20.

Here, as a method of judging whether or not the photographing apparatus 10 is stationary, for example, as shown in FIG. 10, when the inter-frame difference amount F is below a certain threshold p (t1, t2, t3). ) Can be determined to be the stationary state of the photographing device 10, and the photographing instruction can be automatically issued.

With the method described above, while the user can quickly perform continuous shooting, there is a possibility that shooting will occur unexpectedly even if the shooting device 10 happens to be still for a moment.

Therefore, for example, as shown in FIG. 11, the amount of difference between frames falls below a fixed threshold p and a fixed time q
It is also possible to set the shooting timing when the inter-frame difference amount does not exceed the threshold value during the period (t4, t5).

By using this method, since the photographing is performed when the user holds the photographing apparatus 10 in a stationary state for a certain period of time, the photographing is more explicitly instructed and the unexpected photographing is avoided. it can.

Next, the effect of the fourth embodiment will be described.

In the present embodiment, the movement amount of the photographing device 10 is detected from the photographed image of the photographing device 10 (camera),
Shooting is automatically performed when the shooting device 10 is stationary.

Accordingly, it is not necessary to press the shutter (operate the input device 20 to issue a shooting instruction) to shoot a document image, and the shooting device 10 may be moved to aim at the shooting target. Even when photographing a large document, the photographing device 10 is moved up and down, left and right, and the photographer is photographed one after another by holding the hand at a constant interval, and the input result is displayed in real time on the display device 40 (liquid crystal). Since it is displayed on the screen, the user can more easily input characters by shooting. Further, camera shake caused by pressing the shutter (operating the input device 20) can be prevented.

Next, a fourth embodiment of the present invention will be described with reference to the drawings. Such an example corresponds to the fourth embodiment of the present invention.

Like the first embodiment, this embodiment is realized by the portable terminal 100, but the central processing unit also functions as the image difference detection device 90, and the storage device functions as the image temporary storage device 80. They differ in that they do.

This embodiment is different from the first embodiment only in that the terminal is allowed to stand still for a while without pressing the shutter button or the like at the time of photographing. As shown in FIG. 30, a document can be input by a natural operation such as tracing on a terminal. Similarly, when correcting an input error, return to the place of the same input error character, and
Should be stationary.

Next, a character input device according to a fifth embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 31, the character input device 1E according to the fifth embodiment of the present invention is the first character input device 1E shown in FIG.
In addition to the configuration of FIG.
5 is different in that it has 5.

The mark drawing device 31 of the display image control device 30
Has a function of drawing a mark for a user to capture a target character in a moving image sent from the image capturing device 10 and sending the mark to the display device 40.

The line extraction device 55 of the data processing device 50 is
It has a function of being connected to the photographing device 10, decomposing an image photographed by the photographing device 10 for each line, and sending the divided line images to the character extracting device 51.

Next, the operation of the character input device 1E according to the fifth embodiment will be described in detail with reference to the drawings.

In the operation of this embodiment shown in FIG. 32, steps A2, A3, steps A6 to A8, and steps A10 to A13 are the same as the operation of the first embodiment shown in FIG. Therefore, the description is omitted.

When the present apparatus starts its operation, the photographing apparatus 10
The captured image (moving image) from is sent to the display image control device 30. The mark drawing device 31 of the display image control device 30 is
A mark is drawn at a specific position of the photographed image sent from the photographing device 10 and displayed on the display device 40 (step F1).

When the photographing operation is performed, the photographed image output by the photographing device 10 is first sent to the row extracting device 54. The line extracting device 54 decomposes the received image into line images, and sends the line image closest to the position on the screen designated by the photographing device 10 to the character extracting device 51. (Step F2) This row extraction process can be realized by using a conventional technique as described in, for example, Japanese Patent Laid-Open No. 05-182024.

If the character cannot be extracted in step A5 or if the overlapping portion of the combined recognition data and the character recognition data is not found in step A9, the line extracting device 54 determines the line image processed next to the image capturing device. 10
A line image close to the position designated by is extracted and sent to the character extracting device 51 again. If there is no other line image to be processed, the process returns to the camera image output (step F3).

Next, the effect of the fifth embodiment will be described.

In the present embodiment, the photographed image of the photographing device 10 is first decomposed into line images, and the image of the line photographed at the position closest to the mark displayed on the screen is extracted and processed.

With this, the user can input a character string of a part of a document by a simple operation of simply taking a desired character string in accordance with a mark on the screen and taking a picture.

In addition, in the divided photographing, when the input result cannot be combined with the previous input result, a line that can be combined is searched while sequentially processing vertically adjacent lines. Therefore, if only the beginning of the character string to be input can be input, it is not necessary to accurately match the character string with the mark in subsequent shooting. Also,
Since the lines are extracted and processed in the order of being closer to the mark, if the user matches the character strings near the mark, the target character string can be found and combined with a small amount of processing.

Next, a fifth embodiment of the present invention will be described with reference to the drawings. Such an example corresponds to the fifth embodiment of the present invention.

Like the first embodiment, this embodiment is realized by the portable terminal 100, but is different in that the central processing unit also functions as the row extracting device 54 and the mark drawing device 31.

In this embodiment, a business card 3 as shown in FIG.
00 email address part "yamada@nec.c
o. Consider the case of inputting "jp".

Liquid crystal display 112 of portable terminal 100
As shown in FIG. 38, a camera image or a photographed image 214A from the camera 113 and a cross-shaped mark 311 are displayed as a mark for designating a photographing target. Hereinafter, in the present embodiment to the seventh embodiment, the image from the camera 113 is displayed as a moving image on the liquid crystal display 112 in real time before the shutter of the camera 113 is pressed, and when the shutter is pressed, the displayed image is displayed. It is displayed as a captured image.

When the camera position is adjusted so that the first half of the mail address is aligned with the mark 311, and shooting is performed, the captured image is subjected to line extraction processing, the line closest to the mark is subjected to recognition processing, and the recognition result is displayed.

Next, when photographing the second half of the mail address, it is no longer necessary to accurately match the mail address to the mark, as shown in FIG. 39, the mail address may be entered in the screen.

The data processing device 50 extracts the rows of the photographed images, and tries to combine the extracted rows in order from the side closer to the mark. In the example, first, the character string "AX (03) 3454-222
I try to combine 2 ", but I can't because there is no overlap. Then, next, the character string "a@nec.co.j
Attempts to join p "succeed. This way you can enter your email address correctly

Next, a character input device according to a sixth embodiment of the present invention will be described in detail with reference to the drawings.

In the character input device according to the sixth embodiment of the present invention, only the configuration of the recognition data synthesizer 53 is different.
Since the configuration is different from that of the first embodiment shown in FIG. 1, description of other components will be omitted.

Referring to FIG. 33, the recognition data synthesizing apparatus 53 in the fifth embodiment is the same as the overlapping part discriminating apparatus 5 in FIG.
3A, an overlapping portion selection device 53B, and a concatenation device 53C, which are also included in the recognition data synthesis 53 device in the first embodiment, but in the present embodiment, a forced concatenation control device is further included. It differs in having 53D.

The overlapping portion discriminating apparatus 53A has a function of discriminating an overlapping portion between the character recognition data and the synthetic recognition data. This corresponds to step A9 in the operation of the first embodiment shown in FIG.

The overlapping portion discriminating device 53B synthesizes the data by selecting the highly reliable data with respect to the overlapping portion of the character recognition data and the synthetic recognition data, and synthesizes the remaining portion of the character recognition data with the synthetic recognition data. Coupling device 53C
It has a function to send to. This corresponds to step A10 in the operation of the first embodiment shown in FIG.

The connecting device 53C has a function of connecting the character recognition data after the combined recognition data. This is step A in the operation of the first embodiment shown in FIG.
Corresponds to 11.

The forced connection control device 53D is connected to the overlapping part discriminating device 53A and the input device 20, and when it is judged that there is no overlapping part in the combined recognition data and the character recognition data, the character recognition data is transmitted in accordance with the input device 20. Connection device 5
It has a function to send to 3C.

Next, the operation of the character input device 2A according to the sixth embodiment will be described in detail with reference to the drawings.

In the operation of this embodiment shown in FIG. 34, steps A1 to A8 and steps A10 to A1.
The operation of No. 3 is the same as the operation of the first embodiment shown in FIG.

In step A9, when the overlapped portion of the combined recognition data and the character recognition data is searched for, the overlapped portion is found, and the process proceeds to step A10 to perform the combining, but if the overlapped portion is not found, , Display a message on the output device asking the user whether to force the connection. When the user selects not to perform the forced connection, the camera image output is returned again (step G).
1).

When the user inputs that the forced connection is to be performed, the character recognition data is the forced connection control device 53D.
To the coupling device 53C.

The connection device 53C connects the character recognition data sent from the forced connection control device 53D after the combined recognition data sent from the character recognition data storage device 60,
The combined data is sent again to the recognition data storage device 60 (step G2).

Next, the effect of the sixth embodiment will be described.

In this embodiment, the forced connection control device 53 is used.
D is provided, and when the synthesis of the character recognition data fails, the character recognition data can be directly connected after the synthetic recognition data according to the user's instruction.

As a result, even if the character string to be input is printed with a line break in the middle, the entire line can be input as a single character line by forcibly connecting after the line break is photographed.

The forced concatenation process in this embodiment is particularly effective when inputting a character string written with a line break, but the present invention is not limited to this and, for example, it is written in two columns. This is also effective when inputting the first row of a document and then the second row.

Next, a sixth embodiment of the present invention will be described with reference to the drawings. Such an example corresponds to the sixth embodiment of the present invention.

Like the first embodiment, this embodiment is realized by the portable terminal 100, but is different in that the central processing unit also functions as the forced connection control device 53D.

In this embodiment, consider the case where the URL written on the card 400 as shown in FIG. 40 is input.

Similar to the first embodiment, it is assumed that the URLs are photographed in order from the left and the input is completed up to the right end of the first line as shown in FIGS.

As shown in FIG. 42, when the continuous portion of the URL printed on the second line is photographed, there is no overlapping portion between the character recognition data recognized this time and the stored combined recognition data. Connection confirmation message 4 as shown in
10 is displayed.

If "YES" is selected here to instruct forcible concatenation, the character recognition data is concatenated at the end of the stored composite recognition data, and the URL printed with a line break as shown in FIG. Can be entered as a column.

Next, a character input device according to the seventh embodiment of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 35, in a character input device 1F according to a seventh embodiment of the present invention, in addition to the configuration of the first embodiment shown in FIG. 1, the cursor position control device 32 includes a display image control device 30. This is different from the fifth embodiment in that the character deletion device 35 is connected to the recognition data storage device 60.

The cursor position control device 32 determines whether the current input is the first input (photographing) or not based on the combined recognition data sent from the recognition data storage device 60. The recognition result is output to the display device 40 in a state where the cursor is moved to the beginning and is moved to the end of the recognized character string if it is not the first time.

The character deletion device 35 has a function of directly deleting the combined recognition data stored in the recognition data storage device 60 by the input from the input device 20.

The operation of the character input device 1F according to this embodiment will be described with reference to the drawings.

FIG. 36 is a flow chart showing the operation of the character input device 2B according to the seventh embodiment. Here, of the operations of the present embodiment, the first embodiment of the present invention shown in FIG.
The difference from the operation of the above embodiment is that steps H1 to H3 are performed after step A13. Therefore, the description of the steps common to FIG. 2 is omitted.

After the input result is displayed in step A13, the cursor position control device 32 further displays the cursor by aligning the cursor with one character in the input result character string.
(Step H1)

Here, when the current photographing is the first input, that is, when nothing is stored in the recognition data storage device 60, the cursor is positioned at the beginning of the input character string.

This is because in the first input, the characters printed to the left of the character string to be input are often photographed together, so that they can be easily deleted.

In the case of the second and subsequent photographing, the cursor is positioned at the last character of the displayed input result.

This is because in the final shooting in the split shooting, the characters printed to the right of the character string to be input are often taken together, so that they can be easily deleted.

Next, when a character deletion operation is performed by the input device 20, one character at the cursor position is deleted,
Subsequent character strings are shifted one character to the left. If the deleted character is the last character in the recognition result, the character at the cursor position is deleted and the cursor moves one character to the left. Character deletion operations can be performed continuously. When the character deletion operation is completed, the screen returns to the camera image output (step H
2, H3).

By combining this embodiment with the second embodiment, it is possible to perform both deletion and correction of the character at the cursor position.

In this case, the cursor position control device 32
In addition to the above operation, when the last shooting is re-shooting to return to the place where the shooting was performed, the cursor is moved to the character shot closest to the center of the last shooting, as in the second embodiment. Is good.

This is more desirable because the user can easily delete the extra characters before and after, and at the same time, the misrecognized character can be corrected immediately at the time of returning shooting.

Next, the effect of the seventh embodiment will be described.

In the present embodiment, the cursor position control device 32 and the character deletion device 35 are provided, and the character at the cursor position can be deleted by the user operation. Here, when shooting for the first time, the cursor moves to the beginning of the character string, and in subsequent shootings, the cursor moves to the end of the character string, so the user can simply press the delete key several times, Since the mixed extra characters can be deleted, the operability is improved.

Next, a seventh embodiment of the present invention will be described with reference to the drawings. Such an example corresponds to the seventh embodiment of the present invention.

Like the first embodiment, this embodiment is realized by the portable terminal 100, but is different in that the central processing unit also functions as the cursor control device 32.

In this embodiment, consider the case where the telephone number written on the card 500 as shown in FIG. 45 is input.

First, when the first half of the telephone number was photographed,
As shown in FIG. 46, it is assumed that, in addition to the telephone number, the character string “EL:” printed on the left is also photographed and mixed in the recognition result.

Here, the cursor control device 32 determines that the current photographing is the first photographing because the new synthetic recognition data is stored in the state where the data is not stored in the synthetic data storage device, and the character photographing is performed. Move the cursor 216 to the beginning of the recognition result character string on the recognition result

The user who sees the recognition result can delete the mixed “EL:” as shown in FIG. 47 by merely pressing the delete key included in the input buttons 111 three times.

Next, it is assumed that the latter half of the telephone number is photographed, and the character string "(alternative)" printed after the telephone number is mixed in the latter half of the input result as shown in FIG.

Since the cursor control device 32 updates the synthetic recognition data, the cursor 21 is displayed on the character recognition result.
Move 6 to the end of the recognition result character string

[0235] Here, the user can delete the mixed "(alternative)" by pressing the delete key twice, as shown in Fig. 49. Thus, since the mixed character can be deleted by a simple operation, You can easily enter a part of the complicated printed text.

The functions of the character extracting device 51, the character recognizing device 52, the recognition data synthesizing device 53 of the data processing device 50 in the character input device of each of the above embodiments, the character correcting device 25, and the image difference detecting device 90. For other functions and other functions, install a control program that has each function,
It can be realized by software by loading it in the memory of the computer processing device. This control program 300 is stored in a magnetic disk, a semiconductor memory, or another recording medium. Then, it is loaded from the recording medium to the computer processing device and controls the operation of the computer processing device to realize the above-described functions.

Although the present invention has been described with reference to the preferred embodiments and examples, the present invention is not necessarily limited to the above-described embodiments and examples, and various modifications are possible within the scope of the technical idea. It can be modified and implemented.

In the above-described embodiments and examples, the case where the camera is used as the image taking apparatus has been described. However, the case where a scanner (handy scanner or the like) is used as the image taking apparatus is also included in the present invention. The present invention is effective not only when a document is photographed by a photographing device such as a camera having a low resolution to input characters, but also when a character is inputted by a photographing device such as a scanner having a high resolution.

[0239]

As described above, according to the character input device and the character input method of the present invention, the following effects can be obtained.

First, when a character is input using the camera, accurate character recognition can be realized even if a high-quality photographed image due to the camera resolution or lens distortion cannot be obtained. In addition, it is possible to eliminate useless processing such as correcting a captured image by image processing in order to obtain an image suitable for recognition.

The reason for this is that the reliability of recognition is stored for each character, and each time a photograph is taken, it is replaced with a more reliable result.

Secondly, when a character is input using the camera, an input error due to an erroneous recognition of the character can be easily corrected.

The reason is that it is expected that the characters that have been input by mistake will be replaced by the correct characters by re-shooting, and if they are not corrected by it,
This is because correct characters can be input with a few key operations.

Thirdly, when characters are input using the camera, even if a range where there is no overlapping portion is photographed in divided photographing, the image can be used without wasting.

The reason is that even if the result cannot be combined with the input result until then because there is no overlapping portion, the result is held and the combination is tried again after another input is made. Because.

Fourthly, when a document is divided and photographed, a series of operations such as re-holding the document and pressing the shutter button becomes unnecessary. Also, camera shake caused by pressing the shutter can be prevented.

The reason is that the frame difference of the input image from the camera is examined so that the photographing is automatically performed at the timing when the motion of the camera is stationary for a predetermined time.

Fifth, it is possible to easily input a partial character string in the document.

The reason is that the character string to be input may be photographed according to the mark on the screen, and in the case of a long character string, if the character string to be inputted is photographed according to the mark, the subsequent photographing is performed. Then, only the lines that can be combined with the input up to the previous time are extracted and automatically combined.

Sixth, it is possible to easily input a character string having a line break in the middle.

The reason for this is that, in the same way as when shooting a character string in a divided manner, if you shoot consecutive character strings that are line feed, a message asking whether to connect to the screen will be displayed, and one key operation is required. This is because the character strings before and after the line break are connected.

Seventh, even if an extra character string is printed closely before and after the desired character string, it can be easily corrected.

The reason for this is that once the image is taken including the extra character string, the cursor will automatically move to the beginning of the line when shooting for the first time, and to the end of the sentence for subsequent shooting, so simply press the delete key. This is because the extra characters before and after can be easily deleted.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a character input device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation of the first exemplary embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration of a character input device according to a second embodiment of the present invention.

FIG. 4 is a flowchart illustrating an operation of the second exemplary embodiment of the present invention.

FIG. 5 is a flowchart illustrating an operation of a recognition result correction process according to the second embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration of a character input device according to a third embodiment of the present invention.

FIG. 7 is a flowchart illustrating an operation of the third exemplary embodiment of the present invention.

FIG. 8 is a block diagram showing a configuration of a character input device according to a fourth embodiment of the present invention.

FIG. 9 is a flowchart illustrating an operation of the fourth exemplary embodiment of the present invention.

FIG. 10 is a diagram illustrating a first method of determining a shooting timing by a shooting device according to a fourth embodiment of the present invention.

FIG. 11 is a diagram illustrating a second method for determining the shooting timing by the shooting device according to the fourth embodiment of the present invention.

FIG. 12 is a front perspective view of the mobile terminal according to the first embodiment of the present invention.

FIG. 13 is a perspective view of the mobile terminal according to the first embodiment of the present invention as seen from the back side.

FIG. 14 is a diagram illustrating a relationship between a shooting target and a shooting range according to the first embodiment of this invention.

FIG. 15 is a diagram illustrating a state in which the left side of a printed document is photographed according to the first embodiment of this invention.

FIG. 16 is a diagram illustrating a state of character recognition on the left side of the second column of a sentence according to the first embodiment of this invention.

FIG. 17 is a diagram illustrating a state in which the right side of a printed document is photographed according to the first embodiment of this invention.

FIG. 18 is a diagram illustrating a state of character recognition on the right side of the second column of a sentence in the first example of the present invention.

FIG. 19 is a diagram for explaining a state of recognition data composition in the first embodiment of the present invention.

FIG. 20 is a diagram illustrating a state of correction by re-imaging according to the first embodiment of the present invention.

FIG. 21 is a diagram illustrating a state of character recognition in a range centered on b in the second column of a sentence in the first example of the present invention.

FIG. 22 is a diagram for explaining how recognition data is combined by retaking in the first embodiment of the present invention.

[Fig. 23] Fig. 23 is a diagram for describing the manner of shooting with a high-resolution camera according to the first embodiment of the present invention.

FIG. 24 is a diagram illustrating a state of correction by close-up photography according to the first embodiment of this invention.

FIG. 25 is a diagram illustrating a display state of an input result according to the second embodiment of this invention.

FIG. 26 is a diagram illustrating a state of recognition data of a character “b” according to the second embodiment of the present invention.

FIG. 27 is a diagram illustrating a state in which the left side of a printed document is photographed according to the third embodiment of the present invention.

FIG. 28 is a diagram illustrating a manner of photographing the right side of a print document according to the third embodiment of the present invention.

FIG. 29 is a diagram illustrating a manner of photographing the vicinity of the center of a print document according to the third embodiment of this invention.

FIG. 30 is a diagram illustrating a state of automatic shooting according to the fourth embodiment of the present invention.

FIG. 31 is a block diagram showing a configuration of a character input device according to a fifth embodiment of the present invention.

FIG. 32 is a flowchart illustrating an operation of the fifth embodiment of the present invention.

FIG. 33 is a block diagram showing a configuration of a recognition data synthesizing device in a character input device according to a sixth embodiment of the present invention.

FIG. 34 is a flowchart illustrating an operation according to the sixth embodiment of the present invention.

FIG. 35 is a block diagram showing a configuration of a character input device according to a seventh embodiment of the present invention.

FIG. 36 is a flow chart explaining the operation of the seventh embodiment of the present invention.

[Fig. 37] Fig. 37 is a diagram illustrating a manner of photographing the left side of a mail address of a business card according to the fifth embodiment of the present invention.

FIG. 38 is a diagram illustrating a manner of displaying an input result on the left side of a mail address of a business card according to the fifth embodiment of the present invention.

FIG. 39 is a diagram illustrating a manner of displaying an input result on the right side of a mail address of a business card according to the fifth embodiment of the present invention.

FIG. 40 is a diagram illustrating a manner of photographing the first line of a card on which a URL is printed according to the sixth embodiment of the present invention.

FIG. 41 is a diagram illustrating a manner of displaying the input result of the first line of the card on which the URL is printed according to the sixth embodiment of the present invention.

FIG. 42 is a diagram illustrating a manner of photographing the second line of a card on which a URL is printed according to the sixth embodiment of the present invention.

FIG. 43 is a diagram illustrating a state of a message asking whether or not forced connection processing is possible according to the sixth embodiment of this invention.

FIG. 44 is a diagram illustrating the result of the forced connection process according to the sixth embodiment of the present invention.

FIG. 45 is a diagram illustrating a situation in which the left side of the card on which the telephone number is printed is photographed in the seventh embodiment of the present invention.

FIG. 46 is a diagram illustrating a manner of displaying an input result on the left side of a card on which a telephone number is printed according to the seventh embodiment of the present invention.

[Fig. 47] Fig. 47 is a diagram for explaining a manner of deleting an extra character mixed to the left of the input result of a telephone number in the seventh example of the present invention.

FIG. 48 is a diagram illustrating a manner of displaying an input result on the right side of a card on which a telephone number is printed according to the seventh embodiment of the present invention.

FIG. 49 is a diagram illustrating a state of deleting an extra character mixed to the right of a telephone number input result according to the seventh embodiment of this invention.

[Explanation of symbols]

10 Imaging device 20 Input device 25-character correction device 30 Display image control device 31 Mark drawing device 35 character deletion device 40 display device 50 data processor 53A Overlapping part discriminating device 53B Overlapping part selection device 53C coupling device 53D Forced connection control device 54 line extractor 60 recognition data storage device 70 Reserved data storage device 80 image temporary storage 90 Image difference detection device 100 mobile terminals 111 Input button 112 LCD display 113 camera 200 documents 214 photographed image 214A camera image or captured image 215 Input result 216 cursor 300 business cards 311 Mark to match the input character strings Card with 400 URL printed 410 Connection confirmation message Card with 500 phone numbers

Continued front page    F term (reference) 5B029 BB02 CC30                 5B064 BA01 EA07 EA11 EA12 EA14                       EA15 FA04 FA05 FA06 FA13

Claims (32)

[Claims] 1. A character input device for inputting a character by extracting and recognizing a character from a captured image of a document by the image capturing device, wherein the character recognition result is added with a reliability indicating a certainty of recognition. Characters characterized by performing character input based on the recognition result of the character with the highest reliability by comparing the character recognition results from images obtained by repeatedly capturing the whole or part of the document. Input device. 2. A character input device for inputting characters by extracting and recognizing characters from a photographed image of a document by the photographing device, wherein the character extracting means extracts a character portion image from the image photographed by the photographing device. And character recognition means for performing character recognition from the image of the character portion and adding the reliability to the character recognition result as character recognition data, and duplication of the character recognition data obtained from a plurality of images of the same document. By detecting and concatenating the parts, and selecting the highly reliable character recognition data for the overlapping parts,
A character input device comprising: a recognition data combining means for combining a plurality of the character recognition data into one piece of the character recognition data; and a recognition data storage means for storing the combined character recognition data. 3. The reliability is determined based on at least a degree of matching between a dictionary referred to in character recognition and a character extracted from the captured image and information about distortion of the captured image. Alternatively, the character input device according to claim 2. 4. The recognition data synthesizing means stores the character recognition data obtained from the latest image of the character recognition data obtained from a plurality of images of the same document, and the character recognition data stored in the recognition data storage means. When the overlapping portion with the character recognition data obtained from the previous image is detected and the reliability of the character recognition data obtained from the latest image is higher, the previous recognition data storage means The character input device according to claim 2, wherein the character recognition data obtained from an image is rewritten. 5. The character of the character selected from the recognition candidates, which is read out from the recognition data storage means and presents at least one other recognition candidate of at least one of the characters designated from the characters stored as the character recognition data. The character according to any one of claims 2 to 4, further comprising character correction means for correcting the character recognition data by replacing the character recognition data of the designated character with the recognition data. Input device. 6. The character correcting means selects, as the character to be corrected, the character recognizing data of the character located near the center of the photographed image among the character recognizing data displayed on the display means. The character input device according to claim 5. 7. The recognition data synthesizing means stores the character recognition data obtained from the latest image of the character recognition data obtained from a plurality of images of the same document and the character recognition data stored in the recognition data storage means. If the overlapping portion with the character recognition data obtained from the previous image is detected, and if the overlapping portion is not detected, the character recognition data obtained from the latest image is stored as pending data and then input. The character input device according to any one of claims 2 to 6, wherein the character recognition data obtained from the image is also detected with an overlapping portion with the reserved data. 8. A moving amount of the photographing device is detected from a difference between images which are continuously input, a still state of the photographing device is determined based on the moving amount, and an image photographing instruction is automatically issued. The character input device according to claim 1, further comprising means for performing. 9. An image temporary storage unit for temporarily storing a captured image by the image capturing apparatus, and detecting and detecting a difference between an image stored in the image temporary storage unit and an image input from the image capturing apparatus. The image difference detecting means for determining that the photographing device is in a stationary state and sending a photographing command to the photographing device when the difference is less than or equal to a predetermined threshold value. Item 9. The character input device according to any one of items 8. 10. The image difference detection means determines that the photographing device is in a stationary state when the difference is equal to or less than a predetermined threshold for a predetermined time. Character input device. 11. A character input method for inputting a character by extracting and recognizing a character from a captured image of a document by an image capturing device, wherein the character recognition result is added with a reliability indicating a certainty of recognition. A character input method comprising: comparing character recognition results from images obtained by repeatedly photographing the whole or a part of the document and selecting a recognition result of the character having the highest reliability. 12. The reliability is determined based on at least a degree of matching between a dictionary referred to in character recognition and a character extracted from the captured image and information about distortion of the captured image. Character input method described in. 13. A plurality of the characters are detected by detecting and concatenating overlapping portions of the character recognition data obtained from a plurality of images of the same document, and selecting the highly reliable character recognition data for the overlapping portions. 2. The recognition data is combined into one character recognition data.
The character input method according to claim 1 or claim 12. 14. Regarding the character recognition data obtained from a plurality of images of the same document in the recognition data combination, the character recognition data obtained from the latest image and the character recognition obtained from a previous image. The character recognition data obtained from the previous image is rewritten when an overlapping portion with the data is detected and the reliability of the character recognition data obtained from the latest image is higher. Item 13. The character input method according to item 13. 15. A character stored as character recognition data is designated to present another candidate of the designated character, and the character recognition data of the character selected from the candidate is used to display the character of the designated character. 14. The character input method according to claim 13, wherein the character recognition data is corrected by replacing the recognition data. 16. Among the displayed character recognition data, by moving the cursor to the character recognition data of the character located near the center of the photographed image, the character is highly likely to be corrected. The character input method according to claim 15, wherein: 17. In the recognition data composition, regarding the character recognition data obtained from a plurality of images of the same document, the character recognition data obtained from the latest image and the character recognition obtained from a previous image. If an overlapping portion with data is detected and the overlapping portion is not detected, the character recognition data obtained from the latest image is stored as reserved data, and the character recognition data obtained from the next input image is stored as above. The character input method according to claim 13, wherein an overlapping portion with the reserved data is also detected. 18. The amount of movement of the photographing device is detected from the difference between images that are continuously input, the still state of the photographing device is determined based on the movement amount, and the image is automatically photographed. The character inputting method according to claim 11, wherein 19. An image captured by the image capturing device is temporarily stored, a difference between the temporarily stored image and an image input from the image capturing device is detected, and the detected difference is a predetermined threshold value. 14. The character input method according to claim 13, wherein in the following cases, the photographing device is determined to be in a stationary state and a photographing command is sent to the photographing device. 20. The character input method according to claim 19, wherein when the difference is equal to or smaller than a predetermined threshold value for a predetermined time, it is determined that the imaging device is in a stationary state. 21. In a character input program for executing a character input by extracting characters from a captured image of a document by a photographing device and recognizing the character by a photographing device, the recognition result of the character indicates the certainty of recognition. And a function of adding a confidence level, and a function of comparing character recognition results from images obtained by repeatedly capturing the whole or a part of the document and selecting a recognition result of the character with the highest reliability level. Character input program 22. The reliability is determined based on at least a degree of matching between a dictionary referred to in character recognition and a character extracted from the captured image and information about distortion of the captured image. The character input program described in. 23. A function of detecting and connecting overlapping portions of the character recognition data obtained from a plurality of images of the same document, and selecting a plurality of character recognition data having high reliability with respect to the overlapping portions, 23. The character input program according to claim 21 or 22, having a function of combining the character recognition data into one piece of the character recognition data. 24. The recognition data synthesizing function relates to the character recognition data obtained from a plurality of images of the same document,
The reliability of the character recognition data obtained from the latest image is detected by detecting an overlapping portion between the character recognition data obtained from the previous image and the character recognition data obtained from the previous image. 24. The character input program according to claim 23, wherein the character recognition data obtained from the previous image is rewritten when the value is high. 25. By designating a character stored as character recognition data, another candidate of the designated character is presented, and the character of the designated character is selected from the character recognition data of the character selected from the candidates. The character input program according to claim 23, having a character correction function for correcting the character recognition data by replacing the recognition data. 26. The character correction function moves a cursor to the character recognition data of a character located in the vicinity of the center of a photographed image among the displayed character recognition data, so that the character having a high possibility of being corrected. 26. The character input program according to claim 25, wherein 27. The recognition data synthesizing function relates to the character recognition data obtained from a plurality of images of the same document,
The overlapping portion between the character recognition data obtained from the latest image and the character recognition data obtained from the previous image is detected, and when the overlapping portion is not detected, the character obtained from the latest image 24. The character input program according to claim 23, wherein the recognition data is stored as reservation data, and the character recognition data obtained from an image input next is also detected for an overlapping portion with the reservation data. 28. An amount of movement of the photographing device is detected from a difference between images that are continuously input, and a still state of the photographing device is determined based on the movement amount to automatically photograph an image. 22. The character input device program according to claim 21, having a function of: 29. A function of temporarily storing an image captured by the image capturing device, a difference between the temporarily stored image and an image input from the image capturing device is detected, and the detected difference is predetermined. 24. The character input program according to claim 23, further comprising a function of determining that the photographing device is in a stationary state and sending a photographing command to the photographing device when the threshold value is equal to or less than the threshold value. 30. A mark drawing means for drawing a mark indicating the position of a specified specific character string in a photographed image by the photographing device, and the photographed image is divided into row images row by row, and the divided rows are divided. The character input device according to any one of claims 2 to 10, further comprising line extracting means for sending an image to the character extracting means in order from a line image closer to the mark. 31. The recognition data synthesizing means detects an overlapping portion between the character recognition data obtained from the latest image and the character recognition data obtained from the previous image stored in the recognition data storage means. 31. A forcible connection unit for forcibly connecting the character recognition data in accordance with an instruction from the user when not detected, the method according to any one of claims 2 to 10 or 30. Character input device. 32. Based on the character recognition data stored in the recognition data storage means, it is determined whether or not the character recognition data is a character recognition data by a first shot image. If not, it is provided with position control means for clearly indicating the position of the end of the recognition character string, and deletion means for deleting the character of the character recognition data based on the explicit position of the beginning or end of the recognition character string. The character input device according to any one of claims 2 to 10 or claim 30 to 31.