KR101577824B1 - Apparatus and method for character recognition using camera - Google Patents

Abstract

An apparatus for enhancing character recognition performance in a terminal equipped with a camera, the apparatus comprising: a controller for controlling entry or termination of an image conversion mode, transmitting a high resolution still image received through the camera, A user operation and system control unit for transmitting a preview low resolution image and a low resolution still screen image received through a camera and a global motion vector receiving unit for receiving the preview low resolution image from the user operation and system control unit at predetermined time intervals, A global motion vector detecting unit for detecting a global motion vector of the still picture and a global motion vector detecting unit for detecting the global motion vector;

Character recognition, perspective transformation, global motion vector

Description

[0001] APPARATUS AND METHOD FOR CHARACTER RECOGNITION USING CAMERA [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to character recognition, and more particularly, to an apparatus and method for correcting a distorted image at the time of photographing using a camera.

Recently, due to the development of camera technology, it has become possible to embed camera functions in mobile communication terminals such as PDA (Personal Digital Assistants) and mobile phones as a means of acquiring information. If a camera mounted on the mobile communication terminal is used, any type of character information can be easily obtained from an image input in real time, and character recognition results can be easily obtained by recognizing and processing characters from the input image. Unlike a conventional method in which a paper document is scanned and then recognized as a text file or an electronic document, characters that can not be input by a scanner as well as a paper document can be easily acquired. It differs from character recognition.

As described above, since the character recognition method using the camera can acquire and recognize the character regardless of the medium on which the input target characters are recorded and can be utilized in the mobile communication terminal which is easy to carry, Lt; / RTI > A Snapshot-based image character recognition method using a camera mounted on a character recognition device such as a mobile communication terminal is a method of capturing a desired image of a user and recognizing one or more characters contained in the stored image after storing the captured image. The recognized one or more characters are combined into words, and the corresponding meaning of the word combined through the dictionary DB is detected and displayed on the screen. At this time, the user can directly point or click on each word included in the still image photographed to check the meaning thereof.

Hereinafter, a general snapshot-based character recognition process will be described with reference to FIG. FIG. 1 is a flowchart illustrating an existing character recognition process. 1, in step 101, a user views a preview image received from a camera mounted on a character recognition apparatus to determine whether a recognition target is displayed properly on a screen before capturing an image including a character, . The preview image is displayed on the screen of the character recognition device in real time. In step 103, the user captures an image including a desired character. The image photographed in step 105 is acquired as a still image and stored in the memory unit of the character recognition apparatus. In step 107, the character recognition apparatus recognizes the character included in the still image stored in the memory unit. In step 109, the result of the character recognition is displayed on the screen of the character recognition apparatus.

In the case of the snapshot-based image character recognition using the camera as described above, since it is easy to input the character image to be recognized and the result of the character recognition performed in real time can be displayed on the screen of the character recognition device, There is an appreciable benefit. However, in the case of the snapshot-based image character recognition, if the direction of the camera and the direction of the recognition object are tilted at the time of shooting, That is, if the direction of the camera and the direction of the image including the characters are perpendicular to each other, a proper recognition result can be obtained. If you take a picture with a tilted direction, the characters included in the image may be distorted, which is a major cause of misreading.

2 is an exemplary view showing a slope for photographing an existing image. As shown in (a) of FIG. 2, when the user shoots an image in a posture that is convenient for shooting, the angle of the camera to be recognized and the angle of the camera become obliquely distorted. However, if the photographing is performed in a state where the angle of the camera to be recognized and the angle of the camera to be recognized are perpendicular to each other as shown in FIG. 2 (b), distortion of the image may be remarkably reduced, but the user may feel uncomfortable.

Accordingly, there is a need for a method that can reduce the distortion of the photographed image by simple correction even if the user shoots the image in a posture that is easy to photograph.

It is an object of the present invention to provide an improved apparatus and method for communication, and another object of the present invention is to provide an apparatus and method capable of generating a personalized voice mail greeting message.

According to an aspect of the present invention, there is provided an apparatus for enhancing character recognition performance in a terminal equipped with a camera, the apparatus comprising: a controller for controlling entry or termination of an image conversion mode, transmitting a high resolution still image received through the camera, A user operation and system control unit for transmitting a preview low-resolution image and a low-resolution still image received through the camera when the image input mode is entered; and a display unit for displaying the preview low- A global motion vector detection unit that receives the global motion vector at intervals and detects a global motion vector and an image conversion unit that corrects distortion of the still image by transforming the low resolution still image using the detected global motion vector, do.

According to another aspect of the present invention, there is provided a method for enhancing character recognition performance in a terminal equipped with a camera, the method comprising: storing a low resolution still image and a high resolution still image by user input; Extracting a global motion vector using a pre-view low-resolution image received from the camera when there is a distortion; extracting a low-resolution and high-resolution still image using the extracted global motion vector; And outputting the recognition result after recognizing the characters included in the high-resolution still picture image in which the distortion is corrected.

The present invention uses a method of additionally selecting an image conversion mode before performing shooting-based character recognition. Unlike the method of requesting a limited photographing posture, the present invention corrects a received image before recognizing a character, There is an effect of improving the recognition rate.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It will be appreciated that those skilled in the art will readily observe that certain changes in form and detail may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims. To those of ordinary skill in the art.

The present invention proposes a method for improving the performance of character recognition using a camera mounted on a mobile communication terminal. To this end, the present invention comprises a process of performing recognition of an existing snapshot-based image character and a process of converting the input image. In particular, by proposing a method of minimizing distortion by converting an input image in the present invention, it is possible to expect a character recognition performance with an improved recognition rate without being affected by the photographing posture of a user.

The components of the character recognition apparatus having the functions as described above and their operation will be described with reference to FIG. 3 is an internal block diagram of a character recognition apparatus according to an embodiment of the present invention. Referring to FIG. 3, the character recognition apparatus performs a function of controlling a distorted image only by the movement of the apparatus itself using a gesture control UI through a camera. The character recognition apparatus for providing such functions includes a camera unit 301, a user operation and system control unit 303, a motion vector detection unit 305, a memory 307, a character recognition unit 309, an image conversion unit 311 ).

First, the camera unit 301 receives an image in every frame in real time and displays the preview image on the screen of the character recognition apparatus. At this time, the preview image displayed on the image in real time is displayed on the screen with a low resolution, and when the user sends a signal for capturing a specific still image, the preview image is shot with a preset high resolution.

The memory unit 307 stores the high-resolution still image acquired by the user at the start of shooting. It also saves the preview low resolution still image displayed on the screen immediately before shooting. The preview low resolution still image displayed on the screen immediately before shooting is used for interpolation of high resolution still image in image conversion mode. Although the memory 307 is not separately shown in the drawing, the memory 307 may be separately located outside the character recognition apparatus, and the preview low resolution image may be stored in a storage space such as a temporary memory.

The user operation and system control unit 303 converts the image according to the movement of the character recognition apparatus when the user enters the image conversion mode. If it is determined that the acquired image needs to be converted, the user can determine the entry into the image conversion mode according to the user selection, and can determine whether to enter the input with the specific key button located in the character recognition device. When the user enters the image entry mode and moves the character recognition apparatus, the camera unit 301 delivers the preview low resolution image received in real time to the motion vector detection unit 305. [

The motion vector detection unit 305 receives a preview low-resolution image received from the camera unit 301 every frame and detects a global motion vector.

The image transforming unit 311 transforms the image according to a predetermined method according to the direction of the detected motion vector. Resolution still picture image stored in the memory unit 307 or the temporary memory in accordance with the finally-converted preview low-resolution image. The user can confirm the converted low resolution still image on the screen of the character recognition device and perform image conversion again if it is determined that the image conversion is further needed. If it is judged that the image conversion is not necessary, the coefficients used for the conversion of the low resolution still image are applied to the high resolution still image to convert the high resolution still image.

The character recognition unit 309 recognizes characters included in the high-resolution still picture image stored in the memory unit 307 by a predetermined method and displays the recognition result on the screen of the character recognition apparatus.

The character recognition apparatus having the above configuration can improve the character recognition performance while performing the image conversion according to the input of the user. In particular, when real-time character recognition is performed in a portable device such as a mobile communication terminal, Image conversion technology plays a very important role.

The main technical features of the present invention, which will be described below, for the above-mentioned image conversion can be classified as follows. The first step is to acquire the global motion vector using the preview low resolution image received in real time when the image conversion is started by the user. The second step is to perform the image transformation according to the perspective method using the obtained motion vector to correct the distorted part. Through these steps, higher character recognition performance can be obtained in the present invention.

Hereinafter, a character recognition process according to an embodiment of the present invention will be described with reference to FIG. 4 is a flowchart illustrating a character recognition process according to an exemplary embodiment of the present invention.

Referring to FIG. 4, in step 401, the character recognition apparatus obtains a preview low-resolution image in real time using a camera unit. This preview low-resolution image is displayed on the screen of the character recognition apparatus. In step 403, the still image desired by the user is photographed through the camera unit of the character recognition apparatus. The captured still image is divided into a high resolution still image for character recognition and a low resolution still image for use in the image conversion mode. The high resolution still image captured at step 405 is stored in the memory of the character recognition device. Is stored in the memory unit or in the temporary memory. A low-resolution still picture image is displayed on the screen of the character recognition apparatus.

In step 407, the user determines whether there is distortion in the low-resolution still picture image displayed on the screen, and determines whether the still picture image needs to be transformed. If the conversion of the still image is not required, the character recognition apparatus recognizes the character included in the high resolution still image by proceeding to step 409 for performing character recognition. When the character recognition is completed, the character recognition result is output to the screen of the character recognition apparatus in step 411.

If it is necessary to convert the still image, the flow advances to step 413 to receive the preview low resolution image. The character recognition apparatus receives the preview low-resolution image using the camera unit. The received preview low resolution image is not intended to be displayed to the user but to detect the global motion vector and convert the low resolution still image, so that it is not necessary to output the image on the screen of the character recognition device and the converted low resolution still image is output. In step 415, the preview low-resolution image is acquired as a frame image input at predetermined time intervals according to a predetermined frame per second (FPS). This frame image can be inputted variously according to the movement change of the character recognition device.

In step 417, the character recognition apparatus compares at least one frame image, that is, an object and features existing in the current frame image with the previous frame image to obtain a global motion vector. Here, an object existing in a frame image means objects (desk, flower, building, etc.) existing in the image, and a characteristic existing in a frame image means a degree of change of such objects. If the image input through the camera section is a solid background or a white wall, the global motion vector can not be found because no change can be found between the previous frame image and the current frame image. This global motion vector is a measure of the direction in which the frame image moves in its entirety, that is, the movement of the character recognition device. The detection process of the global motion vector is not output to the screen of the character recognition apparatus, and a method of obtaining the global motion vector will be described in more detail with reference to FIGS. 5 and 6. FIG.

In step 419, the character recognition apparatus performs conversion of the low resolution still image according to the global motion vector. For example, if the character recognition apparatus obtains a frame image after moving in the upward direction, the current frame image is compared with the previous frame image obtained before moving in the upward direction to detect a global motion vector. The detected motion vector is applied to the low resolution still image and displayed on the screen of the character recognition device in real time.

In step 421, the character recognition apparatus outputs the finally converted low resolution still image on the screen of the character recognition apparatus. In step 423, the user determines whether the converted low resolution still picture image needs further conversion. If additional conversion is required, the process of steps 413 and 421 is repeatedly performed. If additional conversion is not required, the process proceeds to step 425 where the coefficient of the final low-resolution still image is applied to the high- And stores it in the memory unit of the recognition apparatus. Thereafter, in step 409, recognition of the characters included in the converted high-resolution still image is performed, and in step 411, the character recognition result is output to the screen of the character recognition apparatus.

5 is a diagram illustrating an image conversion according to a direction according to an embodiment of the present invention. This figure shows a process of converting a still image stored in a memory unit by applying a perspective transformation according to a perspective method according to the direction of a character recognition apparatus.

In FIG. 5, when the character recognition apparatus is moved upward, the lower part of the still image is reduced and the upper part is expanded. Up-scaling interpolation is performed in the upper part of the still image and up-scaling is performed in the lower part of the still image because the still image photographed obliquely at this time is converted into the vertically photographed image of the camera additional image. Down scaling interpolation that is narrowed on both sides is performed as shown in 501 of FIG. In the still image, the direction opposite to the direction of the global motion vector moves away from the screen. On the other hand, the direction of the global motion vector has the effect of approaching the screen, thereby balancing the entire screen.

Thus, the remaining 503, 505, and 507 and perspective transformation are performed along the respective directions. At this time, the magnitude of the transformation or the degree of transformation can be changed according to the size of the global motion vector, and the direction can be expressed as an angle with the right side as the reference (0 degree). The direction specified by the reference angle can be changed.

The converted still image can be displayed on the screen in real time, and the user can see the screen to determine whether the distortion of the image is sufficiently corrected. If the image has been sufficiently corrected, the image conversion mode is terminated. If the image is not sufficiently corrected, the perspective transformation is performed again in the direction requiring correction to convert the still image. The end of image conversion mode can be done by inputting touch screen or button. A detailed description of the perspective transformation by the global motion vector will be described below.

FIG. 6 is a flowchart illustrating a still image conversion process using perspective transformation using a global motion vector according to an exemplary embodiment of the present invention. Referring to FIG.

Referring to FIG. 6, in step 601, the camera unit of the character recognition apparatus receives a preview low-resolution image for conversion of a stored still image. The preview low-resolution image is obtained as a frame image input with a predetermined time interval according to a predetermined frame per second (FPS), and the frame image is not displayed for the user but for detecting a global motion vector and converting the still image It is not output to the screen of the character recognition apparatus. In step 603, the character recognition apparatus detects a global motion vector from received frame images. The global motion vector may be detected by comparing a previous frame image and a current frame image among the received frame images to detect a global motion vector from objects and features existing in the image.

In step 605, the size and angle of the detected global motion vector are calculated. The global motion vector can represent all two-dimensional positions according to size and angle, and can represent all directions between 0 and 360 degrees. For example, assuming that the right direction is 0 degree, the upward direction in the counterclockwise direction may be 90 degrees, the left direction 180 degrees, and the downward direction 270 degrees. Since the size of the global motion vector detected in step 607 is very variable, a predetermined threshold value is compared with the size of the detected global motion vector. If the global motion vector is detected to exceed the predetermined threshold value, the magnitude of the global motion vector increases or decreases in proportion to the transform coefficient. Here, the transform coefficient means the degree of transform of the perspective transform, and the still image is transformed according to the transform degree of the perspective transform. For example, when interpolation is required in the upward direction as shown in 501 of FIG. 5, if the size of the global motion vector increases or decreases according to the detected transformation coefficient by converting the character recognition apparatus in the upward direction, Based on this, still image conversion is performed. If the global motion vector is not detected above the predetermined threshold value, steps 601 to 605 are repeated.

When the image conversion mode ends, the high resolution still image stored in the memory unit is converted. At this time, the coefficients used for the low resolution still image conversion are applied to the high resolution still image. The high-resolution still picture image whose distortion is corrected is received by the character recognizing unit, recognizes the characters included in the high-resolution still picture image by a predetermined method, and displays the recognition result on the screen of the character recognizing apparatus.

As described above, the configuration and operation of the apparatus and method for recognizing a character using a camera according to an embodiment of the present invention can be performed. In the meantime, although the embodiments of the present invention have been described in detail, Can be carried out without departing from the scope. Accordingly, the scope of the present invention should not be limited by the illustrated embodiments, but should be determined by equivalents of the claims and the claims.

1 is a flowchart showing an existing character recognition process.

FIG. 2 is a view showing an example of a slope for photographing an existing image

3 is an internal block diagram of a character recognition apparatus according to an embodiment of the present invention.

4 is a flowchart illustrating a character recognition process according to an embodiment of the present invention.

5 is a diagram illustrating an image conversion according to a direction according to an embodiment of the present invention.

6 is a flowchart illustrating a process of converting a still image using perspective transformation using a global motion vector according to an exemplary embodiment of the present invention.

Claims (13)

An apparatus for improving character recognition performance in a terminal equipped with a camera, A preview resolution image and a low resolution still image, which are received through the camera when the image conversion mode is entered, are transmitted through the camera, A user operation and a system control unit, A global motion vector detection unit receiving the preview low resolution image from the user operation and system control unit at predetermined time intervals in units of a predetermined frame to detect a global motion vector, And an image transform unit for transforming the low-resolution still image by using the detected global motion vector to correct the distortion of the still image, Wherein the conversion of the low resolution still image is repeatedly performed and when the conversion of the low resolution still image is completed, the high resolution still image is converted using the global motion vector applied to the conversion of the low resolution still image, A device for improving the character recognition performance. The method according to claim 1, A memory unit receiving the preview low resolution image and the low resolution high resolution still image from the user operation and system control unit, And a character recognition unit for recognizing a character included in the converted high-resolution still picture image. The method according to claim 1, Wherein the low-resolution still picture image can be stored in a storage space different from the high-resolution still picture image. delete The system according to claim 1, wherein the user operation and system control unit Wherein the terminal has a touch screen or a key button in order to enter or exit the image conversion mode. The apparatus of claim 1, wherein the global motion vector detection unit And comparing the current preview low-resolution image and the previous preview low-resolution image among the one or more preview low-resolution images received on a frame-by-frame basis to extract a global motion vector. The method according to claim 6, Wherein the comparison of the preview low resolution images is performed by comparing the moving direction of the current preview low resolution image and the degree of movement of the current preview low resolution image based on the previous preview low resolution image. The image processing apparatus according to claim 1, Wherein the low resolution still image is transformed using perspective transformation according to the detected global motion vector. A method for improving character recognition performance in a terminal equipped with a camera, Storing a low resolution still image and a high resolution still image by user input; Determining whether the low-resolution still image is distorted; Extracting a global motion vector using a preview low-resolution image received from the camera when there is distortion; Transforming the low-resolution and high-resolution still picture images using the extracted global motion vectors to correct distortions; And outputting a recognition result after recognizing the characters included in the high-resolution still picture image in which the distortion is corrected, Wherein the conversion of the low resolution still image is repeatedly performed and when the conversion of the low resolution still image is completed, the high resolution still image is converted using the global motion vector applied to the conversion of the low resolution still image, A method for improving the character recognition performance. delete The method of claim 9, wherein the step of extracting the global motion vector comprises: Receiving the preview low-resolution image at a predetermined time interval according to a predetermined frame per second (FPS) Comparing the current preview low resolution image with the previous preview low resolution image as a center; And extracting a global motion vector according to the comparison result. 12. The method of claim 11, wherein comparing the preview low resolution images comprises: And comparing the movement direction and the movement degree of the current preview low resolution image with reference to the previous preview low resolution image. 10. The method of claim 9, wherein the step of correcting the distortion comprises: Detecting the magnitude and angle of the extracted global motion vector, Determining whether the size of the extracted global motion vector is greater than or equal to a preset threshold value, Applying a magnitude and an angle of the extracted global motion vector to the low resolution still image if the threshold value is greater than the threshold, And applying the size and angle of the global motion vector applied to the finally converted still picture image to the high resolution still picture image to correct the distortion when the conversion of the low resolution still picture image is completed A method for improving character recognition performance.

Images