JP2023046687A - Information processing device, information processing method and program - Google Patents

Abstract

To improve specification accuracy of a character region.SOLUTION: An information processing device extracts a character string region from document image data, determines whether characters included in a unit character region are handwritten characters or printed characters for each unit character region, executes OCR (handwritten character OCR) using a handwritten character recognition dictionary or OCR (printed character OCR) using a printed character recognition dictionary according to a determination result, specifies a character region (hereinafter, a correction object region) being the object of character region re-generation, selects a reference character region being a unit character region of the printed characters necessary for re-generation of the unit character region, determines the size of search from the selected reference character region, and searches for the printed characters in the correction object region on the basis of the search size determined by the reference character region to re-generate the unit character region.SELECTED DRAWING: Figure 7

Description

The present invention relates to character recognition technology for extracting characters from images.

2. Description of the Related Art Conventionally, OCR (Optical Character Recognition) processing, which is a technology for performing character recognition processing on a scanned image obtained by scanning a document containing characters and converting it into a computer-usable character code, is widely known. there is By using OCR processing, it becomes possible to automate the work of converting paper forms, which are typified by expense reimbursement work performed in general offices, into digital data, improving productivity in data entry work. Improvement is expected.

2. Description of the Related Art Receipts handled in expense reimbursement operations often include handwritten characters in a standard printed format, and many include a mixture of printed characters and handwritten characters. For example, in a receipt issued at a store, characters are written by hand in pre-printed entry fields such as amount, date, and proviso.

In general, the character recognition process differs between printed characters and handwritten characters. Therefore, for a document image containing both printed characters and handwritten characters, it correctly distinguishes between printed characters and handwritten characters, and performs appropriate character recognition processing for each. Need to implement.

In Japanese Patent Application Laid-Open No. 2002-200003, handwritten type determination is first performed based on the shape of the character, type recognition is performed for the typed characters, and handwritten character recognition is performed for the characters determined as handwritten characters. If the reliability of each character recognition is higher than a predetermined threshold, the result is adopted, and if the reliability is lower than the predetermined threshold, character recognition is performed using the other character recognition means. Select the one with the highest confidence among the two character recognition results. This increases the reliability of the character recognition processing result for the document image including printed characters and handwritten characters.

Japanese Patent Application Laid-Open No. 2001-143020 Japanese Patent Application Laid-Open No. 2010-218106

However, the method of Patent Document 1 has a problem that if character area specification, that is, character segmentation fails, handwritten type determination cannot be made correctly for each character, and character recognition accuracy decreases. For example, in the character string shown in FIG. 8, in which the typed characters "issue date year month day" and the handwritten characters "2024", "6", and "21" are described, the handwritten character "4" and the handwritten character "4" which are close to each other In some cases, the characters are erroneously cut with the "year" of the printed character as one character. If the identification of the character area fails in this way, the method of Patent Document 1 will perform handwritten and printed character determination for the character area corresponding to "4 years". However, subsequent character recognition processing will fail.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to improve the accuracy of specifying a character area.

The present invention is an information processing apparatus for extracting unit character area candidates corresponding to each of a plurality of characters from a character string area composed of a plurality of characters contained in a read image obtained by reading a document. means for performing character recognition processing for handwritten characters or printed characters on said unit character region candidates to obtain character recognition results and their reliability; correction means for correcting unit character area candidates excluding unit character area candidates whose reliability is equal to or higher than a predetermined threshold, using the reference unit character area candidates, and The candidate is characterized in that it is a candidate of a unit character area for which the reliability of the character recognition result obtained by the character recognition processing for printed characters performed by the recognition means is equal to or higher than a predetermined threshold.

According to the present invention, it is possible to improve the accuracy of specifying a character area.

1 is a diagram showing a system configuration including a character recognition device; FIG. FIG. 3 is a diagram showing an example of an operation panel that implements the UI of the information processing apparatus; 2 is a diagram showing a software configuration for realizing the character recognition device of the first embodiment; FIG. It is a figure explaining the processing process of a system containing character recognition processing. FIG. 4 is a diagram showing an example of a document to be processed; FIG. 10 is a diagram showing an example of a screen for realizing item extraction from a document image; It is a figure explaining the process of character recognition. FIG. 10 is a diagram showing the result of character segmentation in a character string mixed with handwritten and printed characters; FIG. 10 is a diagram illustrating a process of character area regeneration; (a) is a diagram showing how a character area is searched for in a target area for character area regeneration, and (b) is a diagram showing the result of character area regeneration. (a) is a diagram showing the result of handwritten type determination in the second embodiment, and (b) is a diagram showing how a target area for character area regeneration is generated in the second embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. The embodiments do not limit the present invention, and not all configurations described in the embodiments are essential means for solving the problems of the present invention.

<First embodiment>
[System configuration]
FIG. 1 is a diagram showing an information processing system according to the first embodiment. The information processing system has a reading device 100 and an information processing device 110 . The reading device 100 has a scanner 101 and a reading device side communication section 102 . A scanner 101 reads a document and generates scanned document image data. The reading device side communication unit 102 communicates with an external device via a network.

The information processing apparatus 110 has a system control section 111 , a ROM 112 , a RAM 113 , an HDD 114 , a display section 115 , an input section 116 and an information processing apparatus side communication section 117 . The system control unit 111 reads control programs stored in the ROM 112 and executes various processes. The RAM 113 is used as a main memory of the system control unit 111 and a temporary storage area such as a work area. The HDD 114 stores various data, various programs, and the like. Functions and processes of the information processing apparatus 110, which will be described later, are realized by the system control unit 111 reading a program stored in the ROM 112 or the HDD 114 and executing the program.

The information processing device side communication unit 117 performs communication processing with an external device via a network.

The display unit 115 displays various information processed on the information processing device 110 . The display unit 115 may be an operation panel 201 as shown in FIG. 2 installed in an MFP (Multi Function Peripheral), a display of a PC, or the like. Note that the display unit 115 may not be built in the information processing device 110 and may be an external display connected to the information processing device 110 . The input unit 116 has a keyboard and a mouse, and receives various operations by the user. Note that the display unit 115 and the input unit 116 may be provided integrally like a touch panel. Further, the display unit 115 may perform projection using a projector, and the input unit 116 may recognize the position of the fingertip with respect to the projected image using a camera.

In this embodiment, the scanner 101 of the reading device 100 reads a paper document such as a form and generates scan image data. The scanned image data is transmitted to the information processing device 110 by the reader-side communication unit 102 . The information processing apparatus 110 receives the document image data scanned by the information processing apparatus side communication unit 117 and stores the image in a storage device such as the HDD 114 . Part of the functions of the display unit 115 and the input unit 116 may be provided in the reading device 100 .

[UI]
FIG. 2 is a diagram showing a UI (User Interface) displayed on the display unit 115 of the information processing apparatus 110 according to this embodiment. The operation panel 201 is a configuration example that implements the display unit 115 in an MFP or the like. The operation panel 201 includes a touch panel 202 and a numeric keypad 203 composed of physical keys. In FIG. 2, the logged-in user ID, the main menu, and the like are displayed on the upper left of the touch panel 202 .

The UI in this embodiment also functions as a means for providing the user with information extraction results from the document image data to be processed as shown in FIG. 6, and is provided on the touch panel 202 . The display of the information extraction result from the document image data is not limited to the touch panel 202, and may be executed using the display of the PC.

[Software configuration]
FIG. 3 is a diagram showing a software configuration for realizing the character recognition device 300 on the information processing device 110. As shown in FIG. The character recognition apparatus 300 comprises means (303) of the processing result providing unit 301 and means (304 to 312) of the character recognition result generating unit 302. FIG.

The processing result providing unit 301 is a display control unit that presents the processing result of the character recognition result generating unit 302 to the user, and controls the user interface displayed on the above-described touch panel 202 or PC display, for example. The processing result providing unit 301 includes a recognition result display unit 303 that causes the display unit 115 to display the character recognition result obtained by the item extraction unit 307 of the character recognition result generation unit 302 .

The character recognition result generating unit 302 executes character recognition on the document image data input to the information processing apparatus 110, generates a character recognition result, and converts the item value corresponding to the item name to be extracted to the object to be processed. It is a device for extracting from document image data. Character recognition result generator 302 includes image processing means 304 , handwritten character recognition means 305 , printed character recognition means 306 , item extraction means 307 , character string area extraction means 308 and character area extraction means 309 . Further, the character recognition result generation unit 302 includes handwritten type determination means 310, reference character selection means 311, and character area reproduction means 312 as means for improving the accuracy of character recognition.

The image processing means 304 pre-processes the input document image data so that character recognition processing can be executed. A character string region extraction unit 308 extracts a character string region from the document image data. A character area clipping means 309 clips a unit character area for each character to be recognized from the extracted character string area.

The handwritten character recognition means 305 converts the extracted unit character region into a character code using a handwritten character recognition dictionary (character recognition processing). The type recognition means 306 converts the extracted unit character area into a character code using a recognition dictionary for type (character recognition processing). The item extracting means 307 identifies items required by the user from the character string obtained as the result of character recognition.

The handwritten character discrimination means 310 is a means for discriminating whether characters existing in each unit character area extracted by the character area extraction means 309 are handwritten characters or printed characters. The reference character selection means 311 is a means for selecting a reference character area from the cut out unit character areas in each character area. This reference character area is used to specify a printed character area from unit character areas that could not be definitively determined to be a handwritten character or printed character by the handwritten and printed character discrimination means 310 . The detailed processing steps for specifying the type area will be described later with reference to FIGS. 7 to 10. FIG. The character area regenerating means 312 is a means for regenerating a unit character area based on a reference character area for a unit character area that could not be definitively determined to be a handwritten character or a printed character by the handwritten character determining means 310 . Specific steps for this regeneration will be described later with reference to FIG.

[Processing flow]
In FIG. 4, document image data obtained by reading a document with the reading device 100 is input to the information processing device 110, the user checks and corrects the result of character recognition processing on the document image data, and confirms and corrects the character string. It is the flow chart which showed the processing which registers.

First, in S<b>400 , the information processing apparatus 110 acquires document image data obtained by reading a document with the reading apparatus 100 . A document to be read is, for example, a receipt 500 as shown in FIG.

In S401, the image processing unit 304 corrects the inclination of the character area by detecting the description direction of the character string included in the document image data.

In S402, the image processing unit 304 binarizes the grayscale document image data with a certain threshold value.

In S403, the image processing unit 304 removes ruled lines unnecessary for character recognition from the document image data. The processing performed in steps S401 to S403 is preprocessing for accurately executing character recognition in S404.

In S404, character recognition processing is performed on the document image data to which these preprocessings have been applied. First, character string area extraction means 308 extracts a character string area, character area extraction means 309 extracts an area for each unit character, and handwritten character determination means 310 performs handwritten character determination for each character area. Character recognition processing by handwritten character recognition means 305 and printed character recognition means 306, selection of a reference character area by reference character selection means 311, and regeneration of unit character areas by character area regeneration means 312 are included. Finally, a character code is obtained as a character recognition result for the character area included in the document image data.

In S405, the item extraction unit 307 extracts the item value required by the user from the character string obtained as the character recognition result. Taking the receipt shown in FIG. 5 as an example, item values refer to items such as "telephone number" and "money amount" that are registered in the system in step S407 after being set in advance. FIG. 6 shows an example of a UI screen presented to the user when extracting the item. A UI screen 600 includes a preview display area 601 of document image data to be processed, item names 602 to be extracted, and item values 603 extracted from the document image data displayed in the preview display area 601 . The item value extraction process is realized, for example, by the user specifying the position where the item value to be extracted is described in the document image data 601 displayed in the preview display area 601 . Alternatively, a character string corresponding to an item value related to the item set in the item name 602 is searched within the character string extracted from the document image data, and based on a predefined positional relationship between the item name and the item value, Extract the required item value from the position corresponding to the retrieved item name. This makes it possible to extract automatically without a user's instruction. For example, when extracting the item value "¥11,286" whose item name 602 is "amount", if the type of document to be processed (a receipt in this case) is "receipt amount", A character string such as "amount of money" is searched in the document image data. As a result, when a character area recognized as "amount" is found in the document image data, a character area of "amount" defined in advance in the information processing apparatus 110 and a character area in which the item value corresponding thereto is described. Search for a string that corresponds to the item value based on the positional relationship. In this case, the character string “¥11,286” is extracted as the item value based on the rule that “the item value of the receipt amount exists on the right side of the “money amount” character string”. The method of extracting the item value shown above is merely an example, and other methods may be used as long as the method extracts the information desired by the user.

In S405, the character string that is the character recognition result extracted as the item value 603 is displayed on the display unit 115 by the recognition result display means 303. FIG. As shown in FIG. 6, one character string corresponding to each item name 602 is displayed as an item value 603 . After that, the user checks and corrects the character string extracted as the extracted item value 603, and checks a check box 604 indicating that the confirmation and correction are completed.

In S406, when the recognition result display unit 303 detects that all the items are checked, the UI screen with the "Next" button 605 enabled is displayed. When the user presses the "next" button, that is, when all items have been confirmed and corrected (S406 is true), the process proceeds to S407.

In S407, data is registered in the system, and all processing ends.

[Character recognition processing]
Details of the character recognition processing in S404 of FIG. 4 will be described with reference to the processing steps shown in FIG. 7 and the specific example of FIG.

First, in S700, the character string region extracting means 308 extracts a character string region from within the document image data. Extraction of the character string region is realized using a known technique. For example, a projection histogram in a direction perpendicular to the writing direction of the character string of black pixels forming the character portion can be obtained, and division positions can be determined based on the shape and amount of change to form the character string region. As a result, a character string area of "June 24, 2024" as shown in FIG. 8 is obtained from the document as shown in FIG.

In S701, the character region clipping unit 309 clips a candidate unit character region (hereinafter simply referred to as a unit character region). This is based on a known technique. For example, a projection histogram of black pixels forming a character portion in the horizontal direction or the vertical direction is obtained, and the character string pattern is linearly divided based on the shape and amount of change. After that, the extracted pattern is surrounded by a rectangle, and the area and the aspect ratio of the rectangle are used to judge and extract the pattern. Another method is to pay attention to the contact portions (connected components) where characters are connected to each other, and divide them based on their shapes and connection states. Using the method described above, each area determined to be one character is divided into rectangles, and each area is defined as a unit character area. Rectangles drawn with dashed lines shown in FIG. 8 are examples of the result of cutting out the unit character area. This character string is a character string indicating the item value corresponding to the "issuance date" written on the upper right of the receipt shown in FIG. In 801, the handwritten character "4" and the printed character "year" are close to each other, and the extraction of the unit character area in S701 fails, resulting in two characters being included in one character area. showing.

In S702, the handwritten and printed character determination means 310 determines whether the characters included in each unit character area are handwritten characters or printed characters. Known techniques may also be used for this handwritten character determination. For example, there is a method in which scoring is performed from the character image features and geometric features, and whether each unit character region cut out in S701 is determined as a handwritten character or a printed character. . There is a known technique for determining whether a character line is composed of handwritten characters or printed characters for each unit character line. However, in the present invention, since the object is to determine whether each unit character area is a handwritten character or a printed character, this method cannot be used in S702. However, as the method for determining handwritten and printed characters here, other methods may be used as long as they are methods for determining whether each unit character area is a handwritten character or a printed character. In the example shown in FIG. 8, as a result of the handwritten character determination, "2", "0", "2", "6", "2", and "1" from the left side of the character string are determined to be handwritten characters. ” and “日” are determined to be printed characters. "4 years" in a rectangle 801 containing handwritten characters and printed characters is determined to be either handwritten characters, printed characters, or unknown.

In S703, the handwritten character recognition unit 305 performs OCR (handwritten character OCR) using a handwritten character recognition dictionary on the character area determined to be a handwritten character in S702.

In S704, the printed character recognition unit 306 executes OCR (printed character OCR) using a recognition dictionary for printed characters on the character area determined to be printed in S702.

As a result of S703 and S704, the character code for each unit character area and the reliability of the recognition result are acquired as the recognition result for the character image of each unit character area. In the example shown in FIG. 8, handwritten character OCR is performed for "2", "0", "2", "6", "2" and "1", and for "month" and "day" Type OCR is performed. For the rectangle 801 containing "4 years", handwritten character OCR is performed if it is determined to be handwritten characters in S702, and printed character OCR is performed if it is determined to be printed characters. On the other hand, if it is determined to be "unknown", the OCR processing for this area is skipped, or both handwritten character OCR and printed character OCR are performed.

In S705, the character area regenerating unit 312 identifies a character area (hereinafter referred to as a correction target area) to be regenerated as a unit character area. Here, the character area determined as neither the handwritten character nor the printed character in S702, that is, the character area in which the handwritten character determination is "unknown", or the character area in which the reliability of the OCR result in S703 and S704 is lower than a predetermined threshold value. is specified as a correction target region. That is, the character area identified here is a character area that does not look like handwritten characters or printed characters. In the example shown in FIG. 8, the character area of the rectangle 801 is determined to be "unknown" in S702, or the reliability of the recognition result is lower than the predetermined threshold in S703 or S704, so it is specified as the correction target area. . Since handwritten character OCR and printed character OCR have different reliability trends, different thresholds are set for each.

In S706, if the character area regenerating unit 312 did not specify the correction target area, that is, if it is determined that the reliability of the recognition result of all the unit character areas is equal to or higher than the predetermined threshold, there is no error in character cutting. , and the process proceeds to S711.

On the other hand, if the character area regenerating means 312 specifies the correction target area in S706, the process proceeds to S707 in order to perform unit character area regeneration processing for the correction target area.

In S707, the reference character selection means 311 selects a reference character area necessary for regenerating the character area. Here, the reference character area is a character area that serves as a reference when setting a size for searching for characters in the correction target area (hereinafter referred to as a search size). (hereinafter referred to as reference character area candidates). The reference character area may be selected, for example, from reference character area candidates existing in the same character string including the correction target area. In the example shown in FIG. 8, the reference character area candidates are "month" and "day". The character string including the correction target area is the character string area extracted by the character string area extracting means 308, and is the character string area extracted in S700 in FIG.

At S708, the character area regenerator 312 determines the size of the search from the selected reference character area. The search size is determined, for example, based on the maximum and minimum width and height of the reference character area. Considering that the width and height of characters vary depending on the character code, multiple search sizes are determined by varying the width and height between the maximum and minimum values. When the difference between the maximum and minimum values is large, for example, when the character height level varies greatly depending on the character code, such as lowercase alphabet, the width and height are changed around the maximum and minimum values respectively. Determine multiple search sizes.

In S709, the character area regenerating unit 312 searches for characters in the correction target area based on the search size determined by the reference character area, and regenerates the unit character area. The details of the type search processing in this correction target area will be described later.

In S710, the handwritten character recognition unit 305 and the printed character recognition unit 306 perform OCR on the unit character area regenerated in S709. However, for the character area 1002 determined as the character area in S709, the character OCR is performed, or the character OCR result performed on the character area in S709 is reused. On the other hand, the character area 1003 other than the printed character area in the correction target area is highly likely to be handwritten characters, and handwritten character OCR is performed. As a result of S709, a character code, which is an OCR result for each unit character area regenerated in the correction target area, can be obtained.

In S711, the item extracting unit 307 integrates the handwritten character OCR results and the printed character OCR results in S703, S704, and S710 based on the order of the coordinates of the respective unit character regions, and ends the character recognition processing in S404. This integration of OCR results is performed based on the recognition results of handwritten character OCR and printed character OCR and the coordinates of the character area.

Here, details of the type search processing in the correction target area in S709 will be described. FIG. 9 shows a flowchart for explaining the type search processing in this embodiment. FIG. 10(a) shows a state during the type search process in the example shown in FIG. 8, and FIG. 10(b) shows the result of the type search process. The initial position of the search range is the same height as the surrounding reference character area, and is the right end or left end of the correction target area for regenerating the unit character area. An example in which the right end is the initial search range is shown.

First, in S900, type OCR is performed on the search range set in the correction target area with the search size determined in S708, and the character code and recognition reliability are obtained.

In S901, it is determined whether or not the reliability of the printed character OCR recognition is equal to or higher than a predetermined threshold. If the reliability of the printed character OCR is lower than the predetermined threshold, it is determined that the search range does not include printed characters, and the process proceeds to S902.

In S902, it is determined whether there is room for moving (shifting) the search range in the horizontal direction within the correction target area. If there is room to shift the search range within the correction target area, the process proceeds to S903, and if there is no room to shift, the type search process ends.

In S903, the search range is shifted. The search range is shifted by a predetermined fixed width (for example, 5 pixels). A search range 1001 shown in FIG. 10(a) shows a state after the search range 1000 is horizontally moved to the left by several pixels. After moving the position of the search range, the process returns to S900.

Until there is no more room to shift the search range into the correction target area (NO in S902), or until the reliability of the printed character OCR reaches or exceeds a predetermined threshold (YES in S901), the search range is shifted (S903), The type OCR (S900) for the search range is repeated.

Also, since this search process takes time, the search range may be limited to the right and left ends of the correction target area without shifting the search range in units of pixels.

In step S904, the character area surrounded by the search range determined to have high OCR reliability is set as the new character unit character area, and the correction target area is the new character unit character area and other areas. split into FIG. 10(b) shows the result of the type search division. A search range 1002 in which the reliability of the type OCR is equal to or higher than a predetermined threshold is cut out as a unit character area of the type, and the remaining character area 1003 is set as another unit character area while the pixels in that area are masked. That is, by dividing the area to be corrected into another unit character area consisting of a character unit area and the remaining area, two new unit character areas are regenerated, and the character search processing is terminated.

In the present embodiment, the search range is shifted in the cycle of S900 to S903, and printed OCR is performed on the search range each time, and this is repeated until the reliability of the printed OCR result reaches or exceeds a predetermined threshold. However, it is not limited to this method. For example, candidate regions in the search range are collectively acquired from within the correction target region, type OCR is executed collectively for these candidate regions, and the region with the highest reliability is set as a unit character region of new type. may

As described above, the character recognition processing in this embodiment is realized. As described above, a unit character area for which the reliability of handwritten character determination, handwritten character OCR, and printed character OCR is less than a predetermined threshold is specified as a correction target area for regenerating a new unit character area. Character OCR is performed for each search range whose size is determined from a reference character area, which is a unit character area of a character whose reliability is equal to or higher than a predetermined threshold, and the correction target area is subjected to the character OCR result. A unit character area of a new type is searched while referring to the reliability of . Based on the result of the character search processing, another character unit area is regenerated from the area to be corrected, which consists of a new character unit character area and other remaining areas. As a result, it is possible to improve the accuracy of specifying the character area and also improve the accuracy of character recognition.

<Second embodiment>
In the first embodiment, a character string region extracted from document image data is cut out for each unit character, and whether it is a handwritten character or a printed character is determined for each cut out unit character region. On the other hand, in the present embodiment, a method of performing handwritten character determination for each thin line portion is used. It extracts thin lines from an image containing both handwritten and printed characters, decomposes the extracted thin lines into character strokes, and determines whether the character strokes are part of handwritten characters or printed characters based on a histogram of pixel values within the character strokes. It is determined whether it is part or not (see Patent Document 2). When handwritten character determination for each character stroke is used, character cut is performed after handwritten character determination, handwritten character OCR is performed on a unit character area including the character stroke of the handwritten character, and a unit including the character stroke of the printed character is performed. Type OCR is performed on the character area. That is, in the present embodiment, the order of S701 and S702 in the first embodiment is reversed, and the flow is such that the unit character region is cut out after handwritten type determination.

FIG. 11(a) shows a state in which part of printed characters is erroneously determined to be part of handwritten characters. there is Here, a rectangle 1100 includes pixels determined to be part of a handwritten character, and a rectangle 1101 includes pixels determined to be part of a printed character. In this case, both the handwritten character OCR result for the rectangle 1100 and the printed character OCR result for the rectangle 1101 have low reliability. In the first embodiment, in S705, handwritten character determination is performed on the unit character region after character cutting. This is the area to be corrected. In the present embodiment, among unit character regions with low reliability of handwritten character OCR results or printed character OCR results, unit character regions where the distance between unit character regions is equal to or less than a threshold value or where the unit character regions overlap each other are specified, and these unit character regions are identified. A rectangle surrounding existing pixels is set as a correction target area. FIG. 11B shows a rectangle 1103 as a correction target area. Rectangles 1100 and 1101 shown in FIG. 11(a), that is, unit character areas including pixels determined to be part of handwritten characters and unit character areas including pixels determined to be part of printed characters are both reliable for OCR results. The degree is low and overlaps as shown in FIG. 11(b). Therefore, a rectangle 1103 circumscribing these rectangles 1100 and 1101 is set as a correction target area.

The above is the identification of the correction target area for regenerating the unit character area in this embodiment. Thereafter, as in the first embodiment, a reference character area is determined, characters are searched from the correction target area based on the search size calculated from the reference character area, and the unit character area is regenerated.

By the above-described processing, even when handwritten character determination is performed in units of character strokes, the accuracy of identifying character regions can be improved, and the character recognition accuracy can be improved.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

110 Information processing device 300 Character recognition device 301 Processing result providing device 302 Character recognition result generating device

Claims (14)

extracting means for extracting unit character region candidates corresponding to each of the plurality of characters from a character string region composed of a plurality of characters included in a read image obtained by reading a document;
Recognition means for performing character recognition processing for handwritten characters or printed characters on the candidates of the unit character regions, and obtaining character recognition results and reliability thereof;
correction means for correcting unit character area candidates, excluding unit character area candidates whose reliability of the character recognition result is equal to or higher than a predetermined threshold, from among the unit character area candidates, using reference unit character area candidates; ,
with
The reference unit character region candidate is a unit character region candidate for which the reliability of the character recognition result obtained by the character recognition processing for printed characters performed by the recognition means is equal to or higher than a predetermined threshold.
An information processing device characterized by: The correction means performs the character recognition processing for printed characters on the candidates of the unit character regions to be corrected, using the candidates of the unit character regions serving as the reference, and the reliability of the character recognition result obtained by performing the character recognition processing. correcting the unit character area to be corrected to a plurality of new unit character area candidates including a new type unit character area candidate,
The information processing apparatus according to claim 1, characterized by: The correction means sets a search range according to the size of the reference unit character area candidate, and sets the character for printing for each search range with respect to the correction target unit character area candidate. Determining a candidate for a unit character region of the new type based on the reliability of the character recognition result obtained by performing the recognition process;
3. The information processing apparatus according to claim 2, characterized by: The correction means performs character recognition for printed characters while moving the search range in the horizontal direction at the same height as the reference unit character area candidates for the candidate unit character areas to be corrected. process,
4. The information processing apparatus according to claim 3, characterized by: The correction means determines the search range based on the maximum and minimum values of the width and height of the candidates for the reference unit character area.
5. The information processing apparatus according to claim 3, wherein: The correction means sets a search range having the highest reliability of a character recognition result obtained by performing character recognition processing for printed characters for each of the search ranges as a candidate for the unit character area of the new printed character.
6. The information processing apparatus according to any one of claims 3 to 5, characterized in that: The correction means determines a search range in which the reliability of the result of character recognition processing for printed characters performed for each search range is equal to or higher than a predetermined threshold value as a candidate for the unit character area of the new printed character.
6. The information processing apparatus according to any one of claims 3 to 5, characterized in that: the candidate for the unit character region to be corrected and the candidate for the reference unit character region are included in the same character string region;
8. The information processing apparatus according to any one of claims 1 to 7, characterized by: The unit character region candidate to be corrected is one unit character region candidate,
9. The information processing apparatus according to any one of claims 1 to 8, characterized by: The extraction means is
character cutting means for cutting out candidates for the unit character region from the character string region;
determination means for determining whether a character represented by the candidate for the unit character region is a handwritten character or a printed character;
including
The recognition means performs character recognition processing for handwritten characters on candidate unit character regions determined to be handwritten characters, and character for printed characters on candidates for unit character regions determined to be printed characters. perform recognition processing,
10. The information processing apparatus according to claim 9, characterized by: The candidates for the unit character region to be corrected are the candidates for the plurality of unit character regions out of the candidates for the unit character region excluding the candidates for the unit character region whose reliability of the character recognition result is equal to or higher than a predetermined threshold. A plurality of unit character region candidates whose distance between is equal to or less than a predetermined threshold,
9. The information processing apparatus according to any one of claims 1 to 8, characterized by: The extraction means is
determining means for determining whether each part constituting a character in the character string area is a handwritten character or a printed character;
character cutting means for cutting out candidates for the unit character region from the character string region based on the determination result of the portion;
including
The recognition means performs character recognition processing for handwritten characters on candidate unit character regions determined to be handwritten characters, and character for printed characters on candidates for unit character regions determined to be printed characters. perform recognition processing,
12. The information processing apparatus according to claim 11, characterized by: a step of extracting unit character region candidates corresponding to each of the plurality of characters from a character string region composed of a plurality of characters included in a read image obtained by reading a document;
a step of performing character recognition processing for handwritten characters or printed characters on the candidates for the unit character region, and obtaining a character recognition result and its reliability;
a step of correcting unit character region candidates, excluding unit character region candidates whose reliability of the character recognition result is equal to or higher than a predetermined threshold, from among the unit character region candidates, using a reference unit character region candidate;
with
The reference unit character region candidate is a unit character region candidate for which the reliability of the character recognition result obtained by the character recognition processing for printed characters performed by the recognition means is equal to or higher than a predetermined threshold.
An information processing method characterized by: A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 11.

Images