JP5835035B2 - Character recognition program and character recognition device - Google Patents

Description

  The present invention relates to a technique for processing a moving image generated by photographing a character string and recognizing information indicated by characters in the image.

  In general character recognition processing, characters in an image are individually extracted by projection processing, and then each character is collated with a plurality of types of character image models (hereinafter referred to as “character models”) to obtain a predetermined value or more. It is determined that the character indicated by the character model from which the similarity is obtained is the corresponding character. Further, the reliability of the recognition result is calculated based on the similarity to the character model used for this determination.

  In order to obtain a more reliable recognition result, it has also been proposed to perform a recognition process using a moving image. For example, in Patent Document 1, character region extraction and matching processing between a character region and a dictionary image are executed on a plurality of frame images generated under different imaging conditions, and matching among the recognition results obtained by these processing is performed. It is described that a result with a high degree is output (see claim 1, paragraphs 0012 to 0031 and the like).

  In Patent Document 2, when a user performs a character recognition operation in a state where shooting is performed while changing control parameters so that more appropriate shooting can be performed, a frame image at the time of the operation and its front and back It is described that an evaluation value for determining whether or not each of the plurality of frame images is suitable for character recognition is obtained, and recognition processing is performed using an image having the highest evaluation value (claim). (Refer item | item 1, paragraphs 0018-0042 etc.).

JP 2003-242440 A JP 2009-88944 A

  OCR applications using moving images have been introduced into portable information devices (such as cellular phones) with a photographing function in recent years, but portable information devices are carried and used in various places. The OCR application is likely to be used under various brightness environments. In addition, since the user himself / herself holds the machine body with the camera and aligns the camera with the character string to be recognized, the state of the image greatly varies depending on the alignment state. In order to ensure character recognition accuracy under these circumstances, it is necessary to automatically adjust various parameters for photographing so that an image having an image quality suitable for recognition processing can be obtained.

  Patent Documents 1 and 2 describe that the most accurate recognition result is derived by using a plurality of images generated by moving image shooting while changing parameters such as illumination, shutter speed, and aperture. Therefore, it seems that the means to solve the above problem is suggested.

  However, in the invention described in Patent Document 1, the camera is fixed and shooting is performed, and the above processing is performed after detecting that a character area is included in the image. No circumstances are taken into account when photographing the target character string. In the invention described in Patent Document 1, the recognition process under each condition is simply performed once, and the result with the best matching degree is selected. Even if the matching degree is the best result, there is a possibility that an error has occurred, and it cannot be said that the recognition accuracy is sufficient.

  In the invention described in Patent Document 2, character recognition processing is performed by selecting an image having an image quality most suitable for character recognition from a plurality of continuously generated frame images. Since shooting control is performed regardless of the image, there is a possibility that an image suitable for character recognition cannot be acquired. In the invention described in Patent Document 2, the maximum interval for calculating the evaluation value is set. However, an image whose subject is different from the frame at the time of operation is excluded from the target for calculating the evaluation value. (See paragraphs 0034 to 0035, FIG. 5, etc.), there is a risk that the number of frames used to calculate the evaluation value will be insufficient. Moreover, nothing is considered about evaluating the result of character recognition just by evaluating the image before recognition processing, and it is assumed that accurate recognition results can be obtained by such a processing method. That ’s a serious question.

The present invention pays attention to the above problems, and when a user performs a reading operation while holding a camera, the present invention is not limited to a complicated operation and includes a plurality of setting modes prepared for character recognition processing. It is an object of the present invention to select a mode suitable for recognition automatically, and to perform character recognition processing with high accuracy by the selected setting mode.
In addition, an object of the present invention is to enable a user to easily confirm whether or not the accuracy of a recognition result is ensured.

  A program according to the present invention inputs a moving image generated by shooting a character string from a computer connected to a camera having a moving image shooting function and a display unit that displays a moving image generated by the camera. It functions as a character reading device that reads characters in an image.

  In this character reading device, while a character string is being photographed, a process for recognizing characters in the image by inputting the latest frame image obtained by the photographing and a process for calculating the reliability of the recognition result Recognition processing means for repeating the above; mode control means for selecting one of a plurality of setting modes that define at least one of the image quality of the frame image to be subjected to character recognition processing and the operation of the recognition processing means; recognition processing Each means of the display control means which displays the character information based on the recognition result of each time by the means on the same screen as the display of the moving image of the display unit is provided.

  The mode control means also includes a first step of analyzing a recognition result each time while sequentially switching a plurality of setting modes in accordance with a processing cycle of the recognition processing means, and a recognition result satisfying a predetermined consistency condition. In accordance with the fact that the switching of the setting mode is circulated a predetermined number of times under the condition that is obtained every cycle, the setting mode with the highest reliability of the recognition result is selected based on the analysis result, and a plurality of subsequent recognition processing means The second step of maintaining the selection in the processing for the cycle is executed.

  According to the above configuration, while the camera is aligned with the character string to be recognized and character recognition is being executed, a plurality of setting modes are sequentially switched in accordance with the recognition processing cycle. The reliability of the recognition result during this period varies depending on the setting mode, but there should be some degree of consistency between the recognition results every time as long as the same character string is being photographed. . Focusing on this point, in the present invention, the recognition result with the highest reliability is obtained in response to a predetermined number of cycles of switching between setting modes while a recognition result satisfying a certain consistency condition is obtained. Since the obtained setting mode is selected and recognition processing is further performed a plurality of times according to the setting mode, the accuracy of the recognition result can be recognized depending on whether or not the recognition results obtained by the plurality of recognition processing match. .

  While the above processing is being performed, the recognition result is displayed on the display unit together with the moving image being shot. While the setting mode selection is switched, the recognition result display is likely to fluctuate in various ways, but the recognition result display may fluctuate when moving to recognition processing that maintains the appropriate setting mode selection. The nature becomes low. Since the recognition result that is fixedly displayed without fluctuation is obtained in common in each of a plurality of consecutive recognition processes, the user can recognize that the recognition result has changed from unstable display to fixed display. Can be determined. If unstable display continues even after an appropriate setting mode is selected, it can be recognized that an image suitable for recognition processing has not been obtained.

  Under conditions where the camera has not been aligned with the character string to be recognized, such as immediately after the reading operation is started, each recognition result does not satisfy the consistency condition. There is no transition to two steps. Accordingly, since an appropriate setting mode can be selected after the image to be recognized is acquired, the accuracy of recognition can be ensured also by this.

  In the first embodiment of the character recognition device described above, the mode control means returns to the first process in the first step in response to the recognition means having processed a predetermined number of frame images in the second step. That is, the processing by the first step and the processing by the second step are periodically switched.

  In this way, for example, when the setting mode selected based on the result of the first first step is not suitable for the recognition process after that, it is automatically changed to another setting mode. . In addition, since the recognition process with the first alignment did not provide a highly accurate recognition result, the camera alignment state for the recognition target character string or the recognition target character string itself may be changed. Even if shooting is not stopped or scanning is not performed again, the process automatically returns to the first process of the first step, undergoes the mode switching process in the first step, and is set according to the changed conditions. The mode can be set.

In the second embodiment of the character recognition device described above, the mode control means performs the first process of the first step when the latest recognition result by the recognition processing means becomes inconsistent with the previous recognition result. Return.
According to this configuration, when the character string to be recognized is changed, it is possible to quickly start the process of searching for the optimum setting mode for the new character string. be able to. In addition, the user does not need to perform a cumbersome operation such as stopping or resuming to change the character string to be recognized, and is suitable for recognizing the changed character string by simply changing the object to be photographed by the camera. Reading operation can be performed under the setting mode.

  In the third embodiment of the character recognition device described above, the mode control means selects a plurality of setting modes having different setting contents of the camera shooting parameters. As a result, it is possible to automatically set a shooting mode suitable for the environment of the imaging location and ensure the image quality of the image to be recognized.

  In the fourth embodiment of the character recognition device described above, the mode control means selects a plurality of setting modes with different types of character models used when the recognition processing means collates the characters detected from the images for recognition. The target of. Thereby, it becomes possible to automatically select a character model suitable for the font and language type of the character to be recognized.

  According to the present invention, when a user aligns the camera with a character string to be recognized while referring to the display of a moving image, a setting mode suitable for recognition of the character string is automatically selected, and the selection is performed. Multiple recognition processes are performed in the state. During this time, the display of the recognition result displayed together with the moving image is unstable when the setting modes are sequentially switched, but the result of the recognition processing multiple times after the appropriate setting mode is selected. If they match, the display changes to a fixed display. Therefore, the user can easily grasp that the recognition result has been confirmed by changing from unstable display to fixed display. As a result, the operability can be improved and a recognized character string with high accuracy can be displayed, thereby improving convenience.

It is a functional block diagram of an OCR application to which the present invention is applied. It is a figure which shows the example of transition of the display screen of a recognition result. It is the table which matched the kind of imaging | photography mode, and the setting content. It is the table which matched the imaging | photography mode set to each recognition process, and the reliability of the recognition result obtained by the recognition process. A table that associates the shooting mode set for each recognition process when the recognition process while switching the shooting mode and the recognition process in the optimum mode are repeated, and the reliability of the recognition result obtained by the recognition process. It is. FIG. 6 is a flowchart showing a schematic procedure of processing in an OCR application when a shooting mode switching process shown in the table of FIG. 5 is applied. It is a flowchart which shows the detailed procedure of the analysis process of a recognition result. It is the table which matched the setting mode of the character model set to each recognition process, and the reliability of the recognition result obtained by the recognition process.

FIG. 1 is a functional block diagram illustrating a configuration example of an OCR application 1 incorporated in a portable information processing apparatus.
A specific form of the information processing apparatus of this embodiment is a smartphone, and includes a camera 2 having a video shooting function and a touch panel 3 in which a display unit and an operation unit are integrated. In addition to the camera interface 13 and the input / output interface 14, the OCR application 1 includes a library 10 (a program group having a function for character recognition) for causing the control unit of the smartphone to function as a character reading device.

  The library 10 of this embodiment includes the functions of the character recognition processing unit 11 and the control unit 12.

  The camera interface 13 causes the camera 2 to start moving image shooting in response to the activation of the OCR application 1 and captures a frame image every time. The captured image is provided to the input / output interface 14 and the character recognition processing unit 11.

  The character recognition processing unit 11 includes a dictionary (not shown) in which a plurality of types of character models are registered. The frame image provided from the camera interface 13 is stored in the image by a method of image projection processing. Character extraction corresponding to each character is recognized and recognized by performing character segmentation processing that extracts individual characters of the character and collation processing that compares the extracted character images with various character models in the dictionary. Calculate the reliability of the result.

  The above recognition processing and reliability calculation processing are repeatedly executed for each latest frame image captured by the camera interface 13 at that time. The recognition result and reliability of each time are stored in a buffer memory (not shown) and processed by the control unit 12.

  The control unit 12 has a function of setting a shooting mode for recognition processing, a function of analyzing a recognition result and reliability every time, and a function of generating display information including character information indicating the recognition result. It is done. The display information is transferred to the input / output interface 14 and displayed on the touch panel 3 together with the moving image provided from the camera interface 13.

FIG. 2 shows an example of a screen displayed on the touch panel 3.
In this example, character recognition is performed by photographing a character string “OMRON”. In the figure, the character string to be recognized is shown largely in the central frame 100, and the character strings M1 to M4 that are displayed below the character string are the recognition results.

  In this embodiment, three types of photographing modes (see FIG. 3) of “standard mode”, “outdoor mode 1”, and “outdoor mode 2” are provided. The standard mode is a mode suitable mainly for indoor shooting, the outdoor mode 1 is a mode suitable for outdoor shooting when the weather is fine, and the outdoor mode 2 is outdoor when the weather is cloudy. This mode is suitable for shooting with the camera.

  In the screen shown in FIG. 2, the latest frame image input from the camera 2 is displayed, and information indicating the shooting mode applied to the shooting of the displayed image is displayed in the upper left corner of the screen. . In this embodiment, the above three types of shooting modes are selected in order, and each time the recognition process is performed, a process for switching the shooting mode is executed for a predetermined cycle, and then the shooting mode with the most reliable recognition result is obtained. After that, the optimum mode is maintained. 2 (1), (2), and (3) are screens during the last switching of the shooting mode, and FIG. 2 (4) is a screen after the optimum mode is selected.

Each shooting mode is switched at a cycle corresponding to the character recognition processing cycle. For each shooting mode, character recognition processing for one frame image obtained in the mode is executed and the mode is switched to the next mode. The recognition result is displayed before. Each of the character strings M1 to M4 indicating the recognition results on each screen is a result obtained during the shooting mode displayed on this screen. For example, the character string M1 in FIG. 2 (1) is a recognition result for one frame image within the period in which the standard mode is set, and the outdoor mode 1 is set in the character string M2 in FIG. 2 (2). The character string M3 in FIG. 2 (3) is a recognition result for one frame image within the period for which the outdoor mode 2 is set.
The image in FIG. 2 (4) is a frame image when the standard mode is selected as the optimum mode based on the reliability for each shooting mode and the recognition process is performed several times under the optimum mode. Column M4 is a recognition result for an image generated before the image being displayed.

  In the display of the recognition result, among the recognized characters, characters whose similarity (reliability in character units) with respect to the corresponding character model is below a predetermined reference value are displayed in a high-luminance color (for example, red). It is trying to display. In FIG. 2, instead of the color, a character with low reliability is represented by a background halftone pattern.

  According to the example of FIG. 2, in the character string M1 recognized in the standard mode among the three types of shooting modes, all characters are recognized with a reliability equal to or higher than the reference value. On the other hand, in the character string M3 recognized in the outdoor mode 2, “ro” among the characters “o”, “m”, “ro”, and “n” is indicated by “[” and “]” as the bracket characters. Separated and misrecognized. Further, in the character string M2 recognized in the outdoor mode 1, “Ro” is erroneously recognized as the “Kuchi” character. For this reason, in the example of FIG. 2, the reliability for each mode is the highest in the standard mode, and the standard mode is set as the optimum mode as shown in FIG.

FIG. 3 shows specific setting contents of the three types of shooting modes.
Each shooting mode has its own mode number (No. column in the figure). When the shooting mode is switched, each shooting mode is selected in ascending order of the mode number.

  Each shooting mode has a different combination of the degree of brightness adjustment at the time of shooting and the selection of whether or not to perform contrast enhancement processing. In FIG. 2, the brightness adjusted in the standard mode is “standard”, and the adjustment stronger than the standard is “strong”. However, the actual settings indicated by these are combinations of specific values of each control parameter, Registered in memory.

Parameters for adjusting the brightness include a numerical value indicating the shutter speed of the camera 2, a numerical value indicating the aperture of the diaphragm, a flag indicating whether or not the strobe light source emits light, and the like. These parameters are given from the control unit 12 to the camera interface 13, and the operations of the camera 2 and the strobe light source are controlled based on the respective parameters.
The contrast enhancement process is a function that is provided as a standard in the signal processing unit of the camera 2. The parameter relating to this function is a flag indicating whether or not the function is to be enabled. Like the brightness parameter, the parameter is given from the control unit 12 to the camera interface 13 so that the function is turned on or off. Is done.

  In the standard mode, shooting is performed with standard brightness, and contrast enhancement is not performed. In the outdoor mode 1, contrast enhancement processing is performed using the brightness at the time of shooting as a standard. In the outdoor mode 2, photographing with brightness higher than the standard is performed, and further contrast enhancement processing is performed.

FIG. 4 is a table in which the shooting modes set for each recognition process and the reliability of the recognition result obtained by the recognition process are associated in a table format.
In this embodiment, the photographing mode is switched in the order of standard mode → outdoor mode 1 → outdoor mode 2. Further, the average value of the reliability of the recognition result is obtained for each mode, and when the sixth recognition process in which the switching is performed twice is completed, the mode having the highest average reliability is selected as the optimum mode.

  In the example of FIG. 4, in the standard mode, recognition results with reliability of “85” and “90” are obtained in two recognition processes, respectively. The reliability of the recognition result in the outdoor mode 2 is “75” both times. In the recognition processing twice in the outdoor mode 3, recognition results with reliability of “70” and “60” are obtained. Therefore, as described in the margin of the table, since the average reliability in the standard mode is the highest, the standard mode is maintained in the seventh and subsequent recognition processes.

  However, the process of selecting the optimum mode by comparing the average reliability for each operation mode is performed only when a matching degree satisfying a predetermined criterion is recognized among the recognition results in the six recognition processes. The If the recognition result does not satisfy the matching degree criterion, the processing count is reset and the processing is restarted from the first time.

In this embodiment, the recognition result is displayed according to each recognition process, but as illustrated in FIG. 2, there is a high possibility that an error has occurred in the recognition result with low reliability.
In particular, while the shooting mode is switched, there is a high possibility that the display will fluctuate due to variations in reliability every hour. However, when the optimal shooting mode is selected, a highly accurate recognition process is performed each time. As a result, the recognition results at each time match. Therefore, the recognition result on the screen also shifts from an unstable display to a fixed display, so that the user can determine that the recognition result has been confirmed when the fixed display state is reached.

  Therefore, the user thinks that it is necessary to continue shooting while the display is changing, and maintains the state in which the body of the smartphone faces the character string. As a result, the image quality of the image after the optimum shooting mode is selected is further improved, and a recognition result with higher accuracy can be obtained.

  In the case where the camera is not properly aligned with the character string, there is a possibility that the reliability is lowered even in the recognition processing under the optimum photographing mode, erroneous recognition occurs, and unstable display continues. However, by continuing the unstable display, the user can recognize that the recognition accuracy is not ensured and take measures such as changing the position and posture of the camera.

FIG. 5 shows another embodiment relating to selection of the shooting mode.
Also in this embodiment, in the order of standard mode → outdoor mode 1 → outdoor mode 2, the shooting mode is switched every time the recognition processing is performed, and the optimum shooting mode is selected based on the average reliability of each mode when the switching is repeated twice. Thereafter, the recognition process in the optimum mode is continued as in the embodiment of FIG. However, in this embodiment, when the recognition process in the selected optimum mode continues 24 times, the state again switches to the shooting mode. Then, based on the average reliability of each mode when this switching is performed twice, the optimum shooting mode is selected again.

  In the specific example shown in FIG. 5, when the shooting modes are sequentially switched in the first to sixth recognition processes, the average reliability of the standard mode is the highest, so the seventh to 30th times. In the recognition process, the standard mode is adopted. However, when the shooting mode is switched again in the next recognition process from the 31st to the 36th, the average reliability of the outdoor mode 1 is the highest instead of the standard mode. In response, outdoor mode 1 is selected in the 37th to 60th recognition processes.

Similarly, when the recognition process under the shooting mode selected as the optimum mode is executed 24 times, the process returns to the process of switching the shooting mode, and each mode in the 6 recognition processes performed under this switching is performed. Based on the average reliability, the optimum shooting mode is selected again. In other words, in the first six out of 30 recognition processes, the imaging mode is switched for each recognition process, and the imaging mode having the highest average reliability by these processes is set as the optimum mode, and the remaining 24 times. The algorithm of executing the recognition process is repeated.
However, this repetition is also a condition that the recognition result of each time satisfies the consistency standard. If the latest recognition result and the previous recognition result do not meet the consistency standard, the process returns to the first process again.

Hereinafter, a specific procedure of processing executed by the OCR application 1 will be described on the assumption that the shooting mode shown in FIG. 5 is selected.
FIG. 6 shows the flow of the entire process, and FIG. 7 shows the detailed procedure of step S8 (recognition result analysis process) in FIG.

  First, the procedure of FIG. 6 will be described. This process is started in response to an operation for instructing to start photographing for reading. After the counter n indicating the number of processes is set to an initial value 1 in the first step S1, n reaches 30. Until this time, the loop of steps S2 to S9 is repeated.

  In this loop, while the value of n is 6 or less, step S2 is “YES”, and the photographing mode corresponding to the value of n is selected (step S3). Specifically, when n takes a value from 1 to 3, the shooting mode of the mode number corresponding to that value (see FIG. 2) is selected, and when n takes a value from 4 to 6. The shooting mode of the mode number corresponding to the value obtained by subtracting 3 from n is selected. Further, after this selection, in step S4, the operation of the camera 2 or the like is controlled by the selected shooting mode parameter, and a frame image generated under the control is acquired.

  On the other hand, when the value of n exceeds 6, step S2 becomes “NO” and the process proceeds to step S5. In step S5, the operation of the camera 2 or the like is controlled based on the parameters of the optimum mode, and a frame image generated under the control is acquired.

  As described above, by executing steps S3 and S4 or step S5 according to the value of n, the imaging control is performed based on the setting of the parameter of the selected imaging mode, and the frame image generated by the imaging is acquired. To do.

  In step S6, a character recognition process using the frame image acquired through the above process is executed. Briefly, each character image is detected by binarizing the image to be processed and projecting it in the x and y axis directions. Then, for each detected character image, the similarity is calculated by sequentially comparing with various character models registered in the dictionary, and the character indicated by the character model having the highest similarity is applied to the character image. This maximum similarity is the reliability in character units. The recognized characters and their reliability are stored in the buffer memory in the order that the characters follow.

  When the character recognition process is completed, in step S7, the similarity stored for each character in the current (n-th) character recognition process is read, and the reliability of this recognition result (average value of similarity in character units) is obtained. calculate. The calculated reliability is stored in the buffer memory in association with the previous recognition result.

  Thereafter, a recognition result analysis process (step S8) is performed, and then the recognition result is displayed (step S9). If n at this time is less than 30 ("YES" in step S10), the value of n is incremented by 1 in step S11, and then the process returns to step S2. If n = 30 (“NO” in step S10), the process returns to step S1, and the value of n returns to 1.

Next, with reference to FIG. 7, the details of the recognition result analysis processing in step S8 will be described.
In this process, first, the value of n is checked. If n> 1 (“YES” in step S12), the current (nth) recognition result and the previous (n−1) th recognition result are collated. Then, it is checked whether or not both satisfy the consistency standard (steps S13 and S14).

  For example, the number of characters recognized at each time is compared, and if the difference in the number of characters is less than or equal to a predetermined value, the recognition results for each character are compared along the character sequence to obtain the number of matches of the recognition results. If the number of matches or the ratio of the number of matches to the larger number of recognized characters is equal to or greater than a predetermined reference value, it is determined that the current recognition result and the previous recognition result satisfy the consistency standard. . If it is determined that the consistency criterion is not satisfied (step S14 is “NO”), n is set to 0 in step S19, and the analysis process is terminated.

  When it is determined that the consistency criterion is satisfied (step S14 is “YES”), the value of n is maintained. If n is 3 or less or 7 or more, step S15 is “NO”, and the analysis process is terminated with this determination.

  When n takes any value of 4, 5, and 6, step S15 becomes “YES”, the process proceeds to step S16, and the average reliability of the photographing mode corresponding to the value of n is calculated. Specifically, an average value of the reliability calculated in the n-3th process and the reliability calculated in the nth process is obtained, and this average value is obtained as a shooting mode whose mode number is (n-3). Will be set to the average reliability. The calculated average reliability is stored in the buffer memory in combination with the corresponding mode number.

If the value of n is 4 or 5, step S17 becomes “NO”, and the analysis process is terminated with this determination. If n = 6, step S17 becomes “YES” and the process proceeds to step S18.
In step S18, the average reliability calculated for each mode is read from the buffer memory and compared, and the shooting mode with the highest average reliability is selected as the optimum mode. This completes the analysis process.

  According to the procedure shown in FIG. 6 and FIG. 7, while the shooting for the same character string continues, the recognition process while switching between the three shooting modes and the shooting mode selected as the optimum mode are performed. The recognition process below is repeated periodically. Each recognition process is performed independently at each time using the latest frame image. Therefore, the image to be processed is changed by changing the alignment state of the camera 2 with respect to the character string to be photographed or by changing the photographing mode. Even if the state changes slightly, as long as the condition for consistency of recognition results is satisfied, periodic repetition continues.

  On the other hand, when a user who has finished reading a certain character string continues to start reading another character string, and the recognition result becomes inconsistent with the previous one, in the process of analyzing the recognition result , N is set to 0 (see steps S14 and S19 in FIG. 7). Since the value of n is further updated to 1 in step S11 after the analysis process (step S8) and the display of the recognition result (step S9) are completed, the next recognition process is performed as the first process. The

Here, the actions and effects generated by the processing of FIGS. 6 and 7 will be organized.
First, in this embodiment, the recognition process by switching the photographing mode or the recognition process by the optimum mode is executed only when the recognition result of each time satisfies the consistency condition, so that the photographing position is not determined at the start of reading. Therefore, the switching of the shooting mode does not proceed and an inappropriate shooting mode is not selected as the optimum mode. While the alignment of the camera 2 to the character string to be recognized is not completed, the recognition result of each time does not match even if the recognition process is performed while the image appropriately including the character string is not generated. Therefore, the value of n goes back and forth between 0 and 1, and the first recognition process is performed each time substantially. When the position of the camera 2 is determined, switching of the shooting mode is started from that point, so that the optimum mode can be selected at an early stage.

  On the other hand, even if the state of the image changes slightly, if the condition for the consistency of the recognition result is satisfied, periodic repetition can be continued, so that processing delay can be prevented. For example, even if the hand holding the body of the smartphone moves slightly while the shooting mode switching process is being performed, the process can proceed to selection of the optimum mode without clearing the number of processes.

  Also, if the surrounding environment changes during the character string reading process, such as moving from a bright place to a dark place while reading a document in a moving car, the environment Since the optimum mode is updated by the photographing mode corresponding to the change, the accuracy of the recognition process can be ensured.

  On the display screen, the recognition result of each time is displayed on the same screen as the moving image being shot. Therefore, when the alignment of the camera 2 to the character string is completed, the recognition result is quickly confirmed while checking the current shooting state. Appropriateness can be judged. As described above, the recognition result display is likely to be unstable while the shooting mode is switched, but if the recognition result display is fixed after the optimum mode is selected, the user Can recognize that the recognition result has been confirmed.

If the recognition result is unstablely displayed for a long time or there is an error in the fixedly displayed recognition result, the user can change the position and orientation of the camera 2 and read it without performing another reading start operation. You can continue working. As described above, when the unstable display of the recognition result is changed to the fixed display, it can be interpreted that the recognition result is confirmed, so that the user can change the display according to the change of the display or the content of the fixed display. It is possible to determine whether or not to move 2 and the timing of movement, and adjust the shooting position and shooting direction while referring to the real-time moving image being displayed. If the adjustment of the optimum mode so far becomes inadequate due to the adjustment, the optimum mode is updated after the switching of the photographing mode again. If the adjustment of the position and orientation of the camera 2 by the user is suitable for recognition, there is a high possibility that the image quality of the image is improved by the optimum mode set following the adjustment.
Therefore, the possibility that a misrecognized location can be correctly recognized is increased.

  When the character string to be recognized is changed, the process returns to the first of 30 times when the consistency of the recognition result cannot be obtained, so that the shooting mode suitable for the new character string to be recognized Can be selected promptly. Therefore, even when the camera 2 is pointed in a direction different from that of the character string to be processed last time and the ambient brightness has changed greatly, the optimum shooting mode is quickly selected and accurate recognition processing is performed. It can be performed.

  As described above, according to the processing procedure illustrated in FIGS. 6 and 7, the user can refer to the moving image displayed on the touch panel 3 and the recognition result until the highly accurate recognition result is displayed. It is possible to obtain a desired result by taking a picture while changing the position and posture. Therefore, it is possible to perform a recognition process with high accuracy without complicating the user's operation procedure, and convenience is improved.

  Although not shown in FIG. 6, in this process, the function of the input / output interface 14 can appropriately accept an operation for outputting the recognized character string being displayed to another application. When the operation is accepted, the periodic repetitive processing is terminated or stopped, and text data indicating the recognized character string is output to the designated application. As a result, the read character string can be registered in the address book, used in a memo pad, a translation application, or the like.

  In the above embodiment, the shooting mode is switched every time the character recognition process is performed. However, the switching timing is not limited to each time. For example, the recognition process in the same shooting mode is performed twice. The next shooting mode may be selected after the end of the process. In addition, the timing for selecting the optimum mode when switching the shooting mode every time is not limited to the case where the switching is repeated twice, and may be performed after a greater number of circulations.

  In the specific examples of FIGS. 3 to 5, there are three shooting modes. However, more shooting modes may be set. Also, the types of parameters to be adjusted in each shooting mode are not limited to those described above. In addition, the recognition processing means may have a function of executing preprocessing such as brightness adjustment and deviation correction, and a shooting mode may be set in which selection of whether or not to perform this preprocessing is added.

  In the above description, it can be interpreted that the recognition result is confirmed by the fixed display of the recognition result. However, the confirmation result may be displayed more clearly. For example, a method of counting the number of times that the recognition result in the optimum mode completely matches the previous recognition result, and changing the display form of the recognition result when the counted value reaches a predetermined value (for example, 10 times). Can be considered.

  Up to this point, an embodiment has been described in which an optimum mode is selected from a plurality of shooting modes, and character recognition with high accuracy is performed by recognition processing in the optimum mode. Elements that can be selected for character recognition processing are as follows. Other than shooting mode. As an example, an example will be described in which the setting of the type of character model used for collation with a character image is switched.

FIG. 8 shows an example in which the setting of the character model used in the character recognition process is divided into three types of modes, and these setting modes are sequentially switched.
Specifically, in this embodiment, a first setting mode that uses Mincho and Gothic font data, a second setting mode that uses round character pop-type font data, and line-of-book font data. And a third setting mode that uses. Then, these three types of setting modes are sequentially switched every time recognition processing is performed, and the average reliability is obtained for each mode. When the switching is repeated twice, the setting mode having the highest average reliability is selected as the optimum mode. In the seventh and subsequent recognition processes, the selection of the optimum mode is maintained and the recognition process is continued.

  In the illustrated example, the average reliability in the second setting mode (round character pop type) among the three types of setting modes is the highest. In response to this, the second setting mode is selected as the optimum mode, and the seventh and subsequent recognition processes are advanced.

  According to the above processing, even if the user is not particularly conscious of selecting font data suitable for the character string to be recognized, appropriate font data is automatically selected, and the character model based on the font data is used. Since the character recognition process is performed, the accuracy of the recognition result can be greatly increased. The classification of character models is not limited to font data, and a plurality of types of character models may be registered for each language, and the same processing as in FIG. 8 may be performed using the setting mode for each language.

  Further, the processing procedure shown in FIGS. 6 and 7 is applied to the example of FIG. 8 as well, depending on the recognition process while switching the setting mode and the optimum mode selected by the analysis of the recognition result in this switching period. The recognition process may be repeated periodically. Also in this case, when the consistency with the previous recognition result cannot be obtained, the processing returns to the first time, so that the recognition processing for one character string is finished, and the character string of another different font has a recognition target. Even if it is changed, the recognition process using the optimum font data can be performed in response to the change promptly.

  Furthermore, a combination of the setting mode related to the character model and the shooting mode is set as one setting mode, and a plurality of setting modes are prepared, and the optimum mode for recognition processing is selected from these setting modes by the same method. May be.

  In each embodiment, it is assumed that the result of reading a captured character string is displayed as a recognition result as it is. Instead, the word indicated by the read character string is translated into another language. Then, a character string indicating the translation result may be displayed.

  The OCR application 1 to which each of the above embodiments is applied is not limited to a smartphone. It can also be incorporated into an information processing apparatus (for example, a digital video camera) having no communication function, such as a mobile phone other than a smartphone, or an information processing apparatus (for example, a personal computer) connected to a general-purpose digital video camera.

1 OCR Application 2 Camera 3 Touch Panel 10 Library (Program Group for Character Reader)
11 Character recognition processing unit 12 Control unit 13 Camera interface 14 Input / output interface M1 to M4 Character string of recognition result

Claims (6)

A computer connected to a camera having a video shooting function and a display unit for displaying a moving image generated by the camera is inputted with a moving image generated by shooting a character string, and characters in the moving image are input. A program for functioning as a character recognition device for recognizing information indicated by
While the character string is photographed, a recognition processing unit that repeats a process of inputting the latest frame image by the photographing and recognizing characters in the image and a process of calculating the reliability of the recognition result. ,
Mode control means for selecting one of a plurality of setting modes that define at least one of the image quality of the frame image to be subjected to character recognition processing and the operation of the recognition processing means;
Causing the computer to function as each means of display control means for displaying the character information based on the recognition result of each time by the recognition processing means on the same screen as the moving image display of the display unit;
The mode control means performs a first step of analyzing each recognition result while sequentially switching the plurality of setting modes in accordance with a processing cycle of the recognition processing means, and a recognition satisfying a predetermined consistency condition. Based on the result of the analysis, the setting mode with the highest reliability of the recognition result is selected based on the predetermined number of times of switching of the setting mode in a state where the result is obtained every cycle, and the recognition thereafter Executing the second step of maintaining the selection in the processing for a plurality of cycles of the processing means,
A character recognition program characterized by the above.   2. The character recognition program according to claim 1, wherein the mode control means returns to the first process of the first step in response to the recognition processing means having processed a predetermined number of frame images in the second step. 3. .   3. The mode control unit according to claim 1, wherein the mode control unit returns to the first process of the first step when the latest recognition result by the recognition processing unit becomes inconsistent with the previous recognition result. A program for character recognition.   The program for character recognition according to any one of claims 1 to 3, wherein the mode control means selects a plurality of setting modes having different setting contents of shooting parameters of the camera.   4. The mode control unit according to claim 1, wherein the mode control unit selects a plurality of setting modes having different types of character models used when the recognition processing unit collates characters detected from an image for recognition. The program for character recognition described in either. A device that inputs a moving image of a character string generated by a camera having a video shooting function and recognizes information indicated by characters in the moving image,
A display unit for displaying a moving image generated by the camera;
A recognition processing unit that repeats a process of inputting the latest frame image by the shooting and recognizing the character in the image and a process of obtaining the reliability of the recognition result while the character string is being shot. ,
Mode control means for selecting one of a plurality of setting modes that define at least one of the image quality of the frame image to be subjected to character recognition processing and the operation of the recognition processing means;
Display control means for displaying the character information based on the recognition result of each time by the recognition processing means on the same screen as the moving image display of the display unit,
The mode control means includes a first step of analyzing reliability of each recognition result while sequentially switching the plurality of setting modes in accordance with a processing cycle of the recognition processing means, and a predetermined consistency condition Under the condition that the recognition result satisfying each cycle is obtained every cycle, the setting mode with the highest reliability of the recognition result is selected based on the result of the analysis in response to the switching of the setting mode being circulated a predetermined number of times. Performing the second step of maintaining the selection in the processing for a plurality of cycles of the recognition processing means,
A character recognition device.

Images