JP2009088944A - Character recognition apparatus, imaging apparatus and video reproducing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily perform highly accurate character recognition from moving images. <P>SOLUTION: The evaluation value (eval) of a plurality of frame images before and after the frame image to be the object of the instruction of character recognition is calculated. Then, the frame image whose evaluation value (eval) exceeds a threshold A or the frame images of the highest evaluation value among the frame images within a range to a threshold B is selected, the time and labor of searching an image suitable for the character recognition by a user are omitted, and the highly accurate character recognition is easily performed from the moving images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a character recognition device, an imaging device, a video reproduction device, a character recognition method, a program, and a recording medium, and in particular, detects a character or character string present in a frame image indicated in a moving image and encodes it. The present invention relates to a technique suitable for use.

  A character recognition device that reads a character string from a printed matter such as a book or a document and converts it into an electronic text is still widely used. Further, not only books and documents but also a method of extracting character information from a natural image and converting it into a character code is described in Patent Document 1 and Patent Document 2, for example. Patent Document 1 and Patent Document 2 also describe a method for controlling an imaging device and a still image processing method for obtaining a still image that is easier to recognize characters.

JP-A-9-83789 JP 2006-81087 A

  However, the techniques described in Patent Document 1 and Patent Document 2 are both premised on performing character recognition from a still image. For this reason, when the original still image is an image that is not suitable for character recognition due to reasons such as blurring, blurring, and low contrast, accurate character recognition cannot be performed. Further, when character recognition is performed from a moving image, there is a problem in that a user has to search for a frame image that does not have the above-described problems, which takes a lot of time and effort.

  An object of the present invention is to make it possible to easily perform character recognition with high accuracy from a moving image in view of the above-described problems.

  The character recognition device of the present invention comprises character recognition means for recognizing characters from frame images constituting moving image data, and instruction means for instructing character recognition by the character recognition means in response to a user operation, The character recognition means calculates an evaluation value of a plurality of frame images in a predetermined section including a frame image that is a target of the character recognition instruction by the instruction means, selects a frame image with the highest evaluation value, and selects the selection Characters are recognized from the frame image.

  An image pickup apparatus according to the present invention includes the character recognition device described above and an image pickup unit.

  A video reproduction apparatus according to the present invention includes the character recognition device described above, and reproduction means for reproducing the moving image data from a storage medium storing the moving image data and imaging information of the moving image data. Features.

  The character recognition method of the present invention comprises a character recognition step for recognizing characters from a frame image constituting moving image data, and an instruction step for instructing character recognition in the character recognition step according to a user operation, In the character recognition step, select a frame image with the highest evaluation value by calculating an evaluation value of a plurality of frame images in a predetermined section including the frame image that is the target of the character recognition instruction in the instruction step, Characters are recognized from the selected frame image.

  The program of the present invention causes a computer to execute a character recognition step for recognizing characters from a frame image constituting moving image data, and an instruction step for instructing character recognition in the character recognition step according to a user operation, In the character recognition step, select a frame image having the highest evaluation value by calculating an evaluation value of a plurality of frame images in a predetermined section including a frame image that is a target of the character recognition instruction in the instruction step, A computer is executed to recognize characters from the selected frame image.

  The recording medium of the present invention is characterized by recording the program described above.

  According to the present invention, the evaluation value of a plurality of frame images in a predetermined section including the frame image that is the target of the character recognition instruction is calculated, and the frame image having the highest evaluation value is selected. This saves the user from having to search for an image suitable for character recognition. Therefore, it is possible to easily perform character recognition with high accuracy from a moving image.

(First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a functional configuration example of an imaging device 100 in which a camera unit is provided in the character recognition device of the present embodiment. First, an operation when character recognition is performed during shooting and recording of moving image data will be described.

  Video signals captured by the camera unit 101 functioning as an imaging unit are sequentially transmitted to the video processing unit 102. The video processing unit 102 performs appropriate video signal processing on the video signal and sends it to the codec 104. The codec 104 compresses the video signal into a data format that can be stored in the storage medium 109 and transmits the compressed video signal to the recorder unit 108. Examples of compression methods include MPEG-2, MOTION JPEG, H.264, and so on. H.264 AVC method or the like is used. In addition, in order to display the captured video on the display unit 106, the video processing unit 102 simultaneously sends an uncompressed video signal to the recorder unit 108.

  When the operation unit 110 functions as an instruction unit by a user operation and a character recognition process is instructed from the operation unit 110, the video processing unit 102 sends a video signal before compression to the character recognition unit 105. Then, the character recognition unit 105 recognizes the character code of the character or character string and sends it to the recorder unit 108. The recorder unit 108 adds the character code recognized by the character recognition unit 105 to the compressed video signal as metadata, and saves it in the storage medium 109.

  At this time, the recorder unit 108 may stand by without storing the compressed video signal in the storage medium 109 until the character recognition process is completed. On the other hand, the video signal may be stored in the storage medium 109 first, and the character code may be stored later in association with the time code of the frame at the time when the character recognition processing is instructed from the operation unit 110. The address mapping of the storage medium 109 at this time is as shown in FIG.

  If the character recognition process is successful, the obtained character code is converted into a bitmap character by the display control unit 107 and superimposed on the video signal and displayed on the display unit 106. An example of the screen configuration displayed at this time is shown in FIG. Specifically, the display unit 106 is a built-in device such as an LCD panel. On the other hand, a configuration may be employed in which a signal is output to an external video display device as composite video output or HDMI output.

  In order to capture an appropriate video, the video processing unit 102 holds control parameters such as a shutter speed, an aperture value, or a focus position, and sequentially transmits the control parameters to the camera unit 101. At the same time, the video processing unit 102 evaluates the video signal received from the camera unit 101. If the video is not appropriate, such as overexposure or defocusing, the control parameter is changed so that more appropriate shooting can be performed. .

  Further, the video processing unit 102 determines the state of camera shake from the signal output from the camera shake amount detection unit 103, drives the lens of the camera unit 101 so as to cancel the camera shake, and corrects the video itself. The character recognition unit 105 functions as an acquisition unit, and can acquire the imaging information such as the shutter speed, the aperture value, the focus position, and the amount of camera shake from the video processing unit 102 and can be used for the character recognition process. These data can be recorded together with the video signal from the video processing unit 102 via the recorder unit 108 to the storage medium 109.

  The operation unit 110 includes a recording start key and an end key. When recording an image, a series of recordings from when the recording start key is pressed until the end key is pressed is stored in the storage medium 109 as one scene. To do.

  Next, an operation of performing character recognition during playback of a moving image as a video playback device will be described. First, the compressed video signal stored in the storage medium 109 is read out, sent from the recorder unit 108 to the codec 104, and developed into an uncompressed video signal. Then, the video processing unit 102 sends this uncompressed video signal again to the recorder unit 108 and displays it on the display unit 106 under the control of the display control unit 107.

  When the character recognition process is instructed from the operation unit 110, the uncompressed video signal after decompression is sent from the video processing unit 102 to the character recognition unit 105. Then, the character recognition unit 105 sends the recognized character or character code of the character string to the recorder unit 108. The recorder unit 108 writes the character code back to the storage medium 109 as metadata in association with the time code of the frame at the time when the character recognition process is instructed. The address mapping of the storage medium 109 at this time is as shown in FIG. 2 as in the case of performing character recognition processing at the time of shooting.

  If the character recognition process is successful, the obtained character code is converted into a bitmap character by the display control unit 107 and superimposed on the video signal and displayed on the display unit 106. An example of the screen configuration displayed at this time is shown in FIG.

  When shooting information such as shutter speed, aperture value, focus position, and camera shake amount is stored in the storage medium 109 together with the video signal at the time of shooting, the data is read together with the video signal by the recorder unit 108 and displayed on the display unit 106. can do. Further, by sending these data to the character recognition unit 105, it can be used for character recognition processing.

FIG. 2 is a diagram illustrating an example of address mapping when a recognized character code is given as metadata to the moving image data and stored in the storage medium 109 in the present embodiment.
As shown in FIG. 2, the moving image data is composed of frames from frame (0) 201 to frame (z) 206, and character recognition processing is performed in frame (m) 204 and frame (n) 205. Has been done. As a result, the character code data 207 “Hello World” and the character code data 208 “Good Bye!” Are acquired.

  These character code data are stored at an arbitrary address in the storage medium 109, and each character-coded frame number is held as a pointer. When the moving image data is reproduced, the frame pointed to by the character code data is detected in advance. When the frame is reproduced, the stored character code data can also be reproduced and displayed on the display unit 106.

Next, a procedure for character recognition with high accuracy in the present embodiment will be described. FIG. 5 is a flowchart illustrating an example of a character recognition processing procedure performed by the character recognition unit 105 of the present embodiment.
In step S501 in FIG. 5, the processing is started by starting shooting or reproduction. First, in step S502, the character recognition unit 105 determines whether a recognition start key included in the operation unit 110 has been pressed. If the result of this determination is that the recognition start key has not been pressed, the system waits.

  On the other hand, as a result of the determination in step S502, if the recognition start key is pressed, the process proceeds to step S503, and first, the frame number (frame_num), variable (max), and variable (max_frame) are initialized. Next, in step S504, the evaluation value (eval) of the frame image of the target frame number (frame_num) is calculated.

  Here, the evaluation value (eval) of the frame image is a value that becomes larger as the frame image can be recognized with higher accuracy. A method of calculating the evaluation value (eval) will be described later.

  Next, in step S505, it is determined whether or not the calculated evaluation value (eval) is larger than the maximum value (max) among the evaluation values calculated so far. If the result of this determination is not greater than the maximum value (max), processing proceeds to step S508. On the other hand, if the result of determination in step S505 is greater than the maximum value (max), the process proceeds to step S506, where the frame number (frame_num) and the evaluation value (eval) are changed to variable (max_frame) and variable (max), respectively. Update to

  Next, in step S507, it is determined whether or not the maximum value (max) of the evaluation value (eval) exceeds a threshold A at which it can be determined that the frame image is in a state sufficient for character recognition. If the result of this determination is that the threshold value A is exceeded, processing proceeds to step S513, character recognition processing is performed from the frame image corresponding to the frame number (max_frame), and processing is terminated in step S514. On the other hand, if the threshold value A is not exceeded as a result of the determination in step S507, the process proceeds to step S508. In step S508, it is determined whether the evaluated frame number (frame_num) is a value greater than zero.

  If the frame number (frame_num) is greater than 0 as a result of this determination, the process proceeds to step S510, and the next frame number (frame_num) to be evaluated is set to a negative frame number (frame_num) having the same absolute value. Here, the frame number (frame_num) is changed so that the number of the frame being reproduced or photographed is 0 at the moment when the recognition start key is pressed, and gradually moves back and forth.

  Therefore, if the frame number (frame_num) is 0 or less as a result of the determination in step S508, the process proceeds to step S509. Then, the frame number (frame_num) to be evaluated next is set to a value obtained by adding 1 to the positive frame number (frame_num) having the same absolute value. In the present embodiment, 1 is added in order to make all frames subject to evaluation in step S509. However, in order to reduce the processing load, for example, a value of 2 or more may be added.

  Next, in step S511, it is determined whether or not the frame number (frame_num) changed in step S509 exceeds the threshold value B. If it is determined that the threshold value B is exceeded, the process proceeds to step S513. Note that the threshold B indicates the maximum interval for calculating the evaluation value. On the other hand, if the threshold value B is not exceeded as a result of the determination in step S511, the process proceeds to step S512, where the character recognition unit 105 functions as a determination unit, and the subject is compared with the first frame whose frame number (frame_num) is 0. Are the same.

  As a result of this determination, if the subject is not the same, the frame is not evaluated, and the process returns to step S508. A method for determining the identity of the subject will be described later. On the other hand, if it is determined in step S512 that the subject is the same, the process returns to step S504, and the evaluation value (eval) is calculated again.

  As described above, a frame suitable for character recognition is searched by repeating a series of processes in steps S504 to S512. Then, a frame having an evaluation value (eval) exceeding the threshold A is found, or the search is interrupted when the frame is separated from the threshold B, and character recognition is performed from the frame having the maximum evaluation value (eval) until then. Process. Note that a method for performing character region detection, character recognition, and encoding from a natural image is the same as the method described in Patent Document 1.

Next, a method for selecting a frame with high accuracy of character recognition processing will be described. FIG. 6 is a diagram illustrating an example of a frame image with high accuracy of character recognition processing and a frame image with low accuracy.
FIG. 6A shows a frame image with high accuracy of character recognition processing, and characters are clearly photographed.

  On the other hand, FIG. 6B shows a frame image with low accuracy of the character recognition process, and it is difficult to detect the character shape correctly due to the subject blurring. FIG. 6C also shows a frame image with low accuracy of the character recognition process, and it is difficult to detect the outline of the character because the subject is blurred due to a focus shift. Further, FIG. 6D also shows a frame image with low accuracy of character recognition processing, and there is no blurring or blurring. However, since the contrast between the character area and the background is low, it is still difficult to detect the character outline. As described above, the image shown in FIG. 6A can be recognized with high accuracy, and the images shown in FIGS. 6B to 6D can be accurately recognized. Low.

  An evaluation value (eval) for determining whether or not a certain frame image is an image suitable for character recognition as shown in FIG. 6A is calculated using the following equation (1).

  T is the shutter speed, and A is the aperture value (f value). C is the ratio of the brightness between the brightest place and the darkest place in the character area of the image, and S is the shake amount obtained from the shake amount detection unit 103. W, x, y, and z are coefficients for normalizing the shutter speed, f value, brightness ratio, and blur amount, respectively. For the brightness ratio, the video processing unit 102 functions as an evaluation unit, calculates the brightness ratio of each frame image, and sends the calculation result to the character recognition unit 105.

  Thus, as the evaluation value (eval) is larger, the frame image becomes as shown in FIG. 6A, and as the evaluation value (eval) is smaller, one of FIGS. 6B to 6D. It will be as shown in Further, the evaluation value (eval) can be calculated at a calculation cost smaller than that of the character recognition process itself. Therefore, by searching for a frame image having a large evaluation value (eval) according to the procedure shown in FIG. 5 and performing character recognition from the frame, character recognition with higher accuracy can be performed with a small calculation cost.

Next, a method for determining the identity of the subject in step S512 in FIG. 5 will be described. FIG. 7 is a diagram illustrating an example in which a subject of a moving image changes.
In the example illustrated in FIG. 7A, the frame image 702 is switched to a frame image 702 in which a different subject in a completely different scene from the first frame image 701 is captured. In step S512 of FIG. 5, a scene break in the moving image data is detected or a difference between two frame images is calculated by image processing, and it is determined that the scene has been changed when the difference exceeds a threshold value. Thereby, in step S512 of FIG. 5, it can be determined that the subject is not the same in the example shown in FIG. 7A.

  In the example shown in FIG. 7B, the focus of the camera unit 101 is moved from the front subject to the back subject, so that the first frame image 703 is switched to the frame image 704 in which the main subject is changed. In step S512 in FIG. 5, it is detected from the control parameters at the time of shooting that the focus position has changed significantly, or it is calculated by image processing that a portion with high sharpness has moved to another position in the frame image. Thereby, in step S512 of FIG. 5, it can be determined that the subject is not the same in the example shown in FIG. 7B. That is, the evaluation value (eval) is not calculated for the subject whose focus position is shifted.

  In the example shown in FIG. 7C, when the camera unit 101 pans to the right, the proportion of the subject displayed from the first frame image 705 changes, and the frame image 706 changes to the main subject. Yes. In step S512 of FIG. 5, it is determined that the camera unit 101 has been panned from the blur amount data output from the blur amount detector 103, or the movement vectors of the two frame images are calculated by image processing. Thereby, in step S512 of FIG. 5, it can be determined that the subject is not the same in the example shown in FIG. 7C.

  As described above, in this embodiment, the evaluation values (eval) of a plurality of frame images before and after the frame image that is the target of the character recognition instruction are calculated. Then, the frame image having the highest evaluation value is selected from the frame images in which the evaluation value (eval) exceeds the threshold value A or the frame images in the range up to the threshold value B. This saves the user from having to search for an image suitable for character recognition. Therefore, it is possible to easily perform character recognition with high accuracy from a moving image.

(Other embodiments according to the present invention)
Each means constituting the character recognition apparatus and each step of the character recognition method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  In addition, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  Note that the present invention includes a case where a software program for realizing the functions of the above-described embodiments (in the embodiment, a program corresponding to the flowchart shown in FIG. 5) is directly or remotely supplied to a system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a recording medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, the program read from the recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a block diagram which shows the function structural example of the imaging device which provided the camera part in the character recognition apparatus of embodiment of this invention. It is a figure which shows an example of the address mapping of the recognized character code data in embodiment of this invention. In an embodiment of the present invention, it is a figure showing an example of a display when character recognition is carried out during photography. In an embodiment of the present invention, it is a figure showing an example of a display when character recognition is carried out during reproduction. It is a flowchart which shows an example of the process sequence of the character recognition by the character recognition part of embodiment of this invention. In an embodiment of the present invention, it is a figure showing an example of a frame image with high accuracy of character recognition processing, and a frame image with low accuracy. In an embodiment of the present invention, it is a figure showing an example in which a subject of a moving picture changes.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 101 Camera part 102 Image | video process part 103 Blur amount detection part 104 Codec 105 Character recognition part 106 Display part 107 Display control part 108 Recorder part 109 Storage medium 110 Operation part

Claims (16)

Character recognition means for recognizing characters from frame images constituting moving image data;
An instruction means for instructing character recognition by the character recognition means in response to a user operation,
The character recognition means calculates an evaluation value of a plurality of frame images in a predetermined section including a frame image that is a target of the character recognition instruction by the instruction means, and selects a frame image with the highest evaluation value, A character recognition device, wherein a character is recognized from a selected frame image. An acquisition means for acquiring imaging information when imaging the moving image data,
The character recognition device according to claim 1, wherein the character recognition unit calculates an evaluation value based on the imaging information acquired by the acquisition unit. The acquisition means acquires information on a shutter speed value in the imaging information,
The character recognition device according to claim 2, wherein the character recognition means calculates an evaluation value based on the shutter speed value. The acquisition means acquires aperture value information in the imaging information,
The character recognition apparatus according to claim 2, wherein the character recognition unit calculates an evaluation value based on the aperture value. The acquisition unit acquires information on a shake amount in the imaging information,
The character recognition device according to claim 2, wherein the character recognition unit calculates an evaluation value based on the blur amount. An evaluation means for evaluating a brightness ratio from a frame image constituting the moving image data;
The character recognition device according to claim 1, wherein the character recognition unit calculates an evaluation value based on a brightness ratio evaluated by the evaluation unit. An acquisition means for acquiring information on a shutter speed, an aperture value, and a blur amount when the moving image data is captured;
Evaluation means for evaluating a ratio of brightness from frame images constituting the moving image data,
The character recognition apparatus according to claim 1, wherein the character recognition unit calculates an evaluation value based on a ratio of the shutter speed, aperture value, blur amount, and brightness. The character recognition means has a determination means for determining whether or not the subject of the frame image for calculating the evaluation value is the same as the subject of the frame image that is the target of the character recognition instruction by the instruction means;
The character recognition apparatus according to claim 1, wherein if the subject is not the same as a result of determination by the determination unit, the evaluation value is not calculated.   The character recognition apparatus according to claim 8, wherein the determination unit determines whether or not they are the same based on a focus position of the subject.   The character recognition apparatus according to claim 8, wherein the determination unit determines whether or not they are the same according to a ratio of a subject included in the frame image.   The character recognizing means interrupts the calculation of the evaluation value in the plurality of frame images when the calculated evaluation value exceeds a predetermined value, and selects a frame image having an evaluation value exceeding the predetermined value. The character recognition device according to any one of claims 1 to 10.   An imaging apparatus comprising: the character recognition apparatus according to claim 1; and an imaging unit.   The character recognition device according to claim 1, and a reproducing unit that reproduces the moving image data from a storage medium storing the moving image data and imaging information of the moving image data. A video playback device characterized by the above. A character recognition process for recognizing characters from frame images constituting moving image data;
An instruction step for instructing character recognition in the character recognition step according to a user operation,
In the character recognition step, select a frame image having the highest evaluation value by calculating an evaluation value of a plurality of frame images in a predetermined section including a frame image that is a target of the character recognition instruction in the instruction step, A character recognition method, wherein a character is recognized from the selected frame image. A character recognition process for recognizing characters from frame images constituting moving image data;
In response to a user operation, the computer is caused to execute an instruction step for instructing character recognition in the character recognition step,
In the character recognition step, select a frame image having the highest evaluation value by calculating an evaluation value of a plurality of frame images in a predetermined section including a frame image that is a target of the character recognition instruction in the instruction step, A program for causing a computer to recognize characters from the selected frame image.   A computer-readable recording medium on which the program according to claim 15 is recorded.

Images