JP7054662B2 - Image recognition device, image recognition method, and image recognition program - Google Patents

Description

The present invention relates to an image recognition device, an image recognition method, and an image recognition program.

An image recognition device that recognizes character information in an image is applied to automatic recognition of characters in a form and automatic recognition of characters in an identity verification document in a financial institution. Until now, many image recognition devices of this type specify the position in an image of an area containing character information for character recognition based on the layout of ruled lines, but in recent years, manuscripts without ruled lines ( A technique has been proposed that enables character recognition for an image of a document) (see, for example, Patent Document 1).

JP-A-2017-134525

At financial institutions, etc., the format is publicly unified, such as driver's licenses, and only the identity verification documents whose position in the image of the character information to be recognized is specified are automatically recognized by the image recognition device. Often targeted at. On the other hand, in recent years, the number of people who are presented with identity verification documents other than driver's licenses is increasing, and financial institutions want to automatically recognize characters other than driver's licenses using image recognition devices. There is a request.

However, some identity verification documents, such as health insurance cards, have no ruled lines and have a unique layout of various information (for example, name, date of birth, address, etc.) that can be used for identity verification. .. Furthermore, on the card-type health insurance card, in addition to the information necessary for using it as a health insurance card, the name of the company to which the insured belongs, the name of the organization, the logo are printed, and a generic drug is desired. There may be a sticker that indicates your intention. It was difficult to automatically recognize the characters in the image of the identity verification document with these atypical layouts.

In one aspect, it is an object of the present invention to provide a technique capable of automatically recognizing characters in images of various identity verification documents.

The image recognition device according to the first aspect of the present invention extracts a character string from the image of the identity verification document, and is included in the extracted character string and the identity verification document predetermined for each type of the identity verification document. A document type identification unit that identifies the type of the identity verification document in the image based on the position in the image and the degree of matching of the character with the item keyword indicating the item, and the type of the identified identity verification document. A heading character string that matches the name of the item for character recognition from the character strings extracted from the image based on the information of the name of the item for character recognition predetermined for each type of the identity verification document. And the item position that specifies the position in the image where the character recognition is performed based on the positional relationship between the heading character string in the identity verification document and the information that can be used for identity verification corresponding to the heading character string. The character is based on the characteristics of a specific unit, a character recognition unit that recognizes a character at a specified position in the image, and a character sequence of a character string containing information that can be used for identity verification in the identity verification document. It is an image recognition device including a recognition result correction unit that corrects the recognition result.

In the image recognition method according to the second aspect of the present invention, the computer extracts a character string from the image of the identity verification document, and the extracted character string and the identity verification document predetermined for each type of the identity verification document. The type of the identity verification document in the image is identified based on the position in the image and the degree of matching of the character with the item keyword indicating the item included in the image, and the type of the identified identity verification document and the identity are identified. Based on the information of the name of the item to be recognized characters predetermined for each type of confirmation document, the heading character string that matches the name of the item to be recognized is specified from the character strings extracted from the image. Then, the position where the character recognition is performed in the image is specified and specified based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string. An image that recognizes characters at a position in the image and corrects the result of the character recognition based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document. It is a recognition method.

The image recognition program, which is the third aspect of the present invention, extracts a character string from the image of the identity verification document, and is included in the extracted character string and the identity verification document predetermined for each type of the identity verification document. The type of the identity verification document in the image is identified based on the position in the image and the degree of matching of the character with the item keyword indicating the item, and the type of the identified identity verification document and the identity verification document are identified. Based on the information of the name of the item for character recognition predetermined for each type, a heading character string matching the name of the item for character recognition is specified from the character strings extracted from the image, and the above-mentioned Based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string, the position where the character recognition is performed in the image is specified, and the specified position in the image is used. Image recognition that recognizes the character of the position and causes the computer to execute a process of correcting the result of the character recognition based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document. It is a program.

According to the above aspect, it is possible to automatically recognize characters in images of various identity verification documents.

It is a figure which shows the structure of an image recognition system. It is a figure which shows the functional structure of the image recognition apparatus which concerns on one Embodiment. It is a figure which shows the contents of a document identification table. It is a figure which shows the contents of a document type table. It is a figure which shows the contents of the recognition method table. It is a figure which shows the content of the recognition item keyword table. It is a figure which shows the contents of the recognition item table. It is a figure which shows the contents of the recognition category table. It is a flowchart explaining the process performed by an image recognition apparatus. It is a flowchart explaining the content of application processing which concerns on one Embodiment. It is a flowchart (the 1) explaining the content of the identity verification document identification process. It is a flowchart (2) explaining the content of the identity verification document identification process. It is a flowchart explaining the content of the character recognition processing by the ruled line keyword recognition method. It is a flowchart explaining the content of the right-hand side search process. It is a flowchart explaining the content of the downward search process. It is a flowchart (the 1) explaining the content of the data character string correction processing. It is a flowchart (2) explaining the content of a data character string correction process. It is a figure which shows an example of the identity verification document. It is a figure (the 1) explaining the example of the character string extracted from the binarized image. It is a figure (the 2) explaining the example of the character string extracted from the binarized image. It is a figure explaining an example of the determination method of whether or not it is a data candidate character string in a right-handed search process. It is a figure explaining the determination method of on / off of another heading flag in the right side search process. It is a figure explaining the example of the determination method of whether or not it is a data candidate character string in the lower search process. It is a figure explaining the correction method of a data character string. It is a figure which shows the hardware configuration of a computer.

Hereinafter, the image recognition system according to the embodiment will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of an image recognition system.

As shown in FIG. 1, the image recognition system 1 according to the present embodiment includes an image recognition device 2, a scanner device 3, an input device 4, a display device 5, and a server device 6.

The image recognition device 2 automatically recognizes characters of items (for example, name, address, date of birth, etc.) that can be used for identity verification included in the image of the identity verification document (for example, driver's license, health insurance card, etc.). It is a device performed in. The image of the identity verification document is acquired by, for example, the scanner device 3 connected to the image recognition device 2. The scanner device 3 is an example of a device that generates image data (digital data) of an identity verification document such as a paper medium or a plastic card.

The input device 4 is a device for inputting various information to the image recognition device 2. The input device 4 is, for example, a keyboard device. A plurality of types of input devices 4 (for example, a keyboard device and a mouse device) may be connected to one image recognition device 2. The display device 5 is used for displaying an image of an identity verification document, a result of recognition processing for the image, various data held by the image recognition device 2, and the like.

The server device 6 is a device that stores various data referred to in the recognition process in the image recognition device 2, the result of the recognition process, and the like. The image recognition device 2 and the server device 6 are each connected to the communication network 7. The image recognition device 2 acquires various data referred to in the recognition process from the server device 6 via the communication network 7. Further, the image recognition device 2 transmits the image of the identity verification document, the result of the recognition process, and the like to the server device 6 via the communication network 7.

The image recognition system 1 may include a plurality of sets of the image recognition device 2, the scanner device 3, the input device 4, and the display device 5. Further, the image recognition system 1 may include a server device different from the server device 6.

The image recognition device 2 according to the present embodiment is used for recognizing identity verification documents when conducting various transactions such as opening an account at a financial institution such as a bank. In many cases, this type of image recognition device 2 can only recognize identity verification documents having a publicly unified format (position of each item) such as a driver's license. However, financial institutions accept identity verification documents other than driver's licenses (for example, health insurance cards, residence cards, special permanent resident certificates, My Number cards, etc.) as identity verification documents. For this reason, in recent years, financial institutions and the like have requested that the image recognition device 2 also recognize identity verification documents other than a driver's license.

FIG. 2 is a diagram showing a functional configuration of an image recognition device according to an embodiment.
As shown in FIG. 2, the image recognition device 2 of the present embodiment includes a scanner control unit 201, a document type identification unit 202, a recognition method selection unit 203, an item position identification unit 204, and a recognition result correction unit 205. It includes a character recognition unit 206, a recognition result notification unit 207, a communication unit 208, a main control unit 209, and a storage unit 290.

The scanner control unit 201 controls the operation of the scanner device 3 and acquires an image such as an identity verification document.

The document type identification unit 202 identifies the type of the identity verification document in the image based on the image of the identity verification document and the document identification table 291 and the document type table 292 stored in the storage unit 290.

The recognition method selection unit 203 selects a recognition method for the identity verification document in the image based on the identification result of the document type identification unit 202 and the recognition method table 293 stored in the storage unit 290. In the image recognition device 2 of the present embodiment, the recognition method is selected from the fixed position recognition method, the keyword recognition method, and the non-ruled line keyword recognition method. The fixed position recognition method is a method of recognizing characters of each item in an image based on the position in the image defined for each item to be recognized. The keyword recognition method is a method of recognizing characters of each item in an image based on the headword of the item to be recognized and the ruled line in the image. The non-ruled line keyword recognition method is based on the headword of the item to be recognized and the position (for example, right or lower) of the information used for identity verification for the headword in the image of each item in the image. This is a method for character recognition.

The item position specifying unit 204 specifies the position (range) of the character string of each item to be recognized based on the recognition method selected by the recognition method selection unit 203. Here, the position of the character string of each item to be recognized is specified by the information indicating the position and the dimension in the image of the rectangular area including the character string. For example, the position of the character string of each item to be recognized is specified by the coordinates of the upper left corner portion and the coordinates of the lower right corner portion of the rectangular area including the character string. The item position specifying unit 204 includes a fixed position recognition unit 211, a keyword recognition unit 212, and a non-ruled line keyword recognition unit 213.

The fixed position recognition unit 211 specifies the position in the image of the character string of each item to be recognized when character recognition is performed by the fixed position recognition method. The position fixing recognition unit 211 specifies the position of the character string of each item to be recognized based on the position fixing recognition table 294 stored in the storage unit 290. The position-fixed recognition table 294 includes information indicating the position of each item used for identity verification in each image in the identity verification document that performs character recognition by the position-fixed recognition method.

The keyword recognition unit 212 specifies the position of each item to be recognized in the image when character recognition is performed by the keyword recognition method. The keyword recognition unit 212 specifies the position of the character string of each item to be recognized based on, for example, the recognition item keyword table 295 stored in the storage unit 290 and the ruled line in the image. The recognition item keyword table 295 includes a headword (keyword) of each item used for identity verification in the identity verification document that performs character recognition by the keyword recognition method and the non-ruled line keyword recognition method.

The non-ruled line keyword recognition unit 213 specifies the position of each item to be recognized in the image when character recognition is performed by the non-ruled line keyword recognition method. The non-ruled line keyword recognition unit 213 has, for example, the recognition item keyword table 295 stored in the storage unit 290 and the position of the headword of each item to be recognized in the image, and the character string of each item to be recognized. Identify the location.

The character recognition unit 206 performs character recognition of each item based on the position of the character string of each item to be recognized specified by the item position identification unit 204 and the type character recognition dictionary 298 stored in the storage unit 290. do.

The recognition result correction unit 205 corrects the recognition result of the character recognition unit 206 based on the recognition item table 296 and the recognition category table 297 stored in the storage unit 290. The recognition item table 296 includes information indicating a category (character type) for each item to be recognized, the number of digits of the character string, the threshold value of the distance between the character strings, and the final character. The recognition category table 297 includes the correspondence between the information indicating the category in the recognition item table 296 and the included character type.

The recognition result notification unit 207 outputs the recognition result to the display device 5 or the like. The communication unit 208 communicates with an external device such as the server device 6. The main control unit 209 includes a scanner control unit 201, a document type identification unit 202, a recognition method selection unit 203, an item position identification unit 204, a recognition result correction unit 205, a character recognition unit 206, a recognition result notification unit 207, and a communication unit 208. It controls the operation of the entire image recognition device 2 including the control of the operation of each part of the above.

The storage unit 290 of the image recognition device 2 according to the present embodiment has a document identification table 291, a document type table 292, a recognition method table 293, a fixed position recognition table 294, a recognition item keyword table 295, a recognition item table 296, and a recognition category table. Stores various information including 297 and the print character recognition dictionary 298. Of these, the position-fixed recognition table 294 includes, as described above, information indicating the position of each item used for identity verification in the image in the identity verification document. The fixed position recognition table 294 contains, for example, information indicating the position of each item of a driver's license such as a number, a name, a date of birth, an address, and an expiration date in an image. The type character recognition dictionary 298 is a dictionary referred to in known character recognition. Therefore, in the present specification, the description of the contents of the fixed position recognition table 294 and the type character recognition dictionary 298 will be omitted. Hereinafter, with reference to FIGS. 3 to 8, the document identification table 291, the document type table 292, the recognition method table 293, the recognition item keyword table 295, the recognition item table 296, and the recognition item table 291 referred to by the image recognition device 2 of the present embodiment are referred to. The contents and usage of the category table 297 will be described.

FIG. 3 is a diagram showing the contents of the document identification table. FIG. 4 is a diagram showing the contents of the document type table. FIG. 5 is a diagram showing the contents of the recognition method table. FIG. 6 is a diagram showing the contents of the recognition item keyword table. FIG. 7 is a diagram showing the contents of the recognition item table. FIG. 8 is a diagram showing the contents of the recognition category table.

The document identification table 291 and the document type table 292 are information to be referred to when identifying the type of the identity verification document in the image. As shown in FIG. 3, the document identification table 291 includes a document No., an identification keyword, a score, an upper left abscissa, an upper left ordinate, a lower right ordinate, and a lower right ordinate. Further, the document type table 292 includes the document No., the document name, and the recognition method No., as shown in FIG.

The document No. of the document identification table 291 is a number for identifying the type of the identification document. The correspondence between the document number and the type of the identification document is defined in the document type table 292. For example, in the document type table 292 of FIG. 4, the document name of the identity verification document of document No. 1 is a driver's license. Therefore, the row of document No. = 1 in the document identification table 291 shows information about the driver's license. Further, for example, in the document type table 292 of FIG. 4, the document name of the identity verification document of document No. = 2 is a health insurance card (card). Therefore, the row of document No. = 2 in the document identification table 291 shows information about the card type health insurance card.

Here, the explanation returns to the description of the document identification table 291. The identification keyword of the document identification table 291 is a character string described in the identity verification document that can be used to identify the identity verification document specified by the document No. The upper left horizontal coordinate, upper left vertical coordinate, lower right horizontal coordinate, and lower right vertical coordinate of the document identification table 291 are information indicating the position (range) of the identification keyword in the image of the identity verification document. For example, the character string "driver's license" is written on the license of document No. = 1. Then, the upper left horizontal coordinate X1, the upper left vertical coordinate Y1, the lower right horizontal coordinate X2, and the lower right vertical coordinate Y2 in the row where the identification keyword in the document identification table 291 is "driver's license" are "driver's license" in the image of the license. Indicates the position (range) in which the character string "Proof" is written. Here, the horizontal coordinates are information indicating the horizontal position when the image of the identity verification document is viewed in the normal position (for example, the number of pixels from the left end of the image), and the ordinates are the information indicating the position of the identity verification document in the normal position. Information indicating the position in the vertical direction of the case (for example, the number of pixels from the upper end of the image). That is, in the document identification table 291 of FIG. 3, the character string "driver's license" has (upper left horizontal coordinate X1, upper left ordinate Y1) as the upper left corner in the image, and (lower right horizontal coordinate X2, lower right). It is shown that it is described in the rectangular area whose lower right corner is the ordinate Y2).

Further, in the document identification table 291 of FIG. 3, for example, in the case of an image of a card type insurance card with document No. = 2, (upper left horizontal coordinate X21, upper left vertical coordinate Y21) is set as the upper left corner, and ( It shows that the character string "health insurance" is described in the rectangular area having the lower right horizontal coordinate X22 and the lower right vertical coordinate Y22) as the lower right corner.

Next, the score of the document identification table 291 will be described. The score of the document identification table 291 is information indicating the degree of matching between the character string extracted from the image of the identity verification document and the character string of the identification keyword. In the present embodiment, among the identification keywords, the score indicating the degree of matching of the character strings is calculated for the identification keywords whose positions in the image and the character strings extracted from the image of the identity verification document substantially match. At this time, the score when the character string extracted from the image of the identity verification document and the character string of the identification keyword match is set to 100 points. In addition, the score when the character string extracted from the image of the identity verification document and the character string of the identification keyword partially match is {(the number of matching characters / the number of characters of the identification keyword) × 100} points. For example, when the score indicating the degree of matching between the character string extracted from the image of the driver's license and the character string of each identification keyword in the document identification table 291 is calculated, the score of each identification keyword of document No. = 1 becomes 100 points. The values are close to each other, and the scores of each identification keyword of other document numbers are low. Therefore, in the image recognition device 2 of the present embodiment, the document No. having the highest total score calculated for each document No. in the document identification table 291 is determined as the document No. indicating the identity verification document in the image.

After determining the document No., the image recognition device 2 refers to the document type table 292 of FIG. 4 and reads out the recognition method No. corresponding to the determined document No. The recognition method No. in the document type table 292 of FIG. 4 corresponds to the recognition method No. in the recognition method table 293 of FIG. That is, the image recognition device 2 selects the recognition method of the identity verification document in the image based on the recognition method No. determined with reference to the document type table 292 and the recognition method table 293. For example, in the recognition method table 293 of FIG. 5, the recognition method of the recognition method No. 1 is non-ruled line keyword recognition. Therefore, when the identity verification document in the image is identified as the health insurance card of document No. 2, the image recognition device 2 selects the non-ruled line keyword recognition method as the recognition method of the identity verification document in the image. Further, for example, when the identity verification document in the image is identified as a driver's license of document No. 1, the image recognition device 2 selects the fixed position recognition method as the recognition method of the identity verification document in the image.

Next, the contents and usage of the recognition item keyword table 295 will be described with reference to FIG. The recognition item keyword table 295 is used, for example, to specify the position of the headword of the item to be recognized in the image when character recognition is performed by the ruled line keyword recognition method. As shown in FIG. 6, the recognition item keyword table 295 includes a document No., a recognition item No., and an item keyword. The document No. corresponds to the document No. of the document type table 292. The recognition item No. is a number that identifies an item for character recognition in the identity verification document specified by the document No. The item keyword is a headword of an item for character recognition in a character string extracted from an identity verification document. The insurance card with document No. 2 is issued by each municipality in Japan, for example, and there may be multiple headwords for one item. In addition, since the extraction accuracy when extracting a character string from an identity verification document is not 100%, the extracted character string may contain another character similar to the character described in the identity verification document. .. Therefore, a plurality of item keywords for one recognition item No. are registered in the recognition item keyword table 295.

For example, the recognition item No. = 1 in the recognition item keyword table 295 in FIG. 6 is an item indicating a number assigned to the insured person. The heading of the item indicating the number assigned to the insured person in the insurance card may be simply described as "number" or may be described as "insured person number", for example. Further, when extracting the character string "number" from the image of the health insurance card, for example, it may be mistakenly extracted as "banyu". Therefore, in the recognition item keyword table 295 of FIG. 6, three keywords of "number", "insured person number", and "banyu" are registered as item keywords of recognition item No. = 1. There is. As a result, for example, even if the insurance card in the image contains "insured person number" or the character string described as "number" is mistakenly extracted as "banyu", the insurance will be insured. It is possible to specify the position of the headword indicating the number assigned to the person.

Further, in the recognition item keyword table 295 of FIG. 6, for example, "date of birth", "birthday", "day of the horse", "day of the horse" are set as the item keywords of the recognition item No. = 3. , "Date of birth", and "Date of birth" are registered. Of the six item keywords, the four keywords "date of birth", "date of birth", "date of birth", and "date of birth" were incorrectly extracted as "date of birth". An example of the result. As a result, for example, even when the headword of the date of birth in the insurance card in the image is described as "birthday", the position of the headword indicating the date of birth can be specified. In addition, for example, even if the character string described as "date of birth" in the image is mistakenly extracted as "date of birth" or "date of birth", the headword indicating the date of birth may be used. It is possible to specify the position.

Further, in the recognition item keyword table 295 of FIG. 6, for example, three keywords of "gender", "sex I", and "sex 嗣" are registered as item keywords of recognition item No. = 5. .. Of the three item keywords, the two keywords "gender I" and "gender" are examples of incorrect extraction results of "gender". This makes it possible to specify the position of the headword indicating gender even when the character string described as "gender" in the image is mistakenly extracted as "gender I" or "gender". It becomes.

Next, the contents and usage of the recognition item table 296 and the recognition category table 297 will be described with reference to FIGS. 7 and 8. The recognition item table 296 and the recognition category table 297 are used, for example, to correct the recognition result after character recognition is performed by the ruled line keyword recognition method. As shown in FIG. 7, the recognition item table 296 includes a recognition item No., a recognition item name, a recognition category No., a number of digits, a distance threshold between character strings, and a final character. The recognition item No. corresponds to the recognition item No. in the recognition item keyword table 295 of FIG. The recognition item name is information (name) that identifies the headword of the recognition item indicated by the recognition item No.

The recognition category No. in the recognition item table 296 of FIG. 7 identifies a character string category (character type) that is located to the right or below the recognition item name and the corresponding headword and indicates information that can be used for identity verification. It is a number and corresponds to the recognition category No. of the recognition category table 297 in FIG. The number of digits indicates the maximum number of characters in a character string indicating information that can be used for identity verification. The distance threshold (mm) between character strings is considered to be a character string containing information about the recognition item when there are multiple character strings to the right or below the headword of the recognition item. It is the maximum value of the distance between the character strings. The final character is the last character in the information (character string) used for identity verification, which is written to the right or below the headword of the recognition item.

For example, in the recognition item table 296 of FIG. 7, the recognition category No. of the recognition item (number) of the recognition item No. = 1 is 2. Then, in the recognition category table 297 of FIG. 8, the recognition category (character type) of the recognition category No. 2 is all categories. Further, the distance threshold between the character strings of the recognition item (number) of the recognition item No. = 1 is 10 mm. Therefore, in the image recognition device 2, when a plurality of character strings are lined up to the right or below the heading word indicating a number and the distance between the character strings is within a threshold value (within 10 mm), the plurality of characters. Combine columns and regard them as one character string. However, the recognition item (number) of the recognition item No. = 1 has 32 digits and the final character is undefined. Therefore, when the character string located to the right or below the headword indicating the number has 33 digits or more, the image recognition device 2 makes a correction to delete the 33rd and subsequent digits.

Further, for example, the recognition category No. of the recognition item (date of birth) of the recognition item No. = 4 is 1. Then, in the recognition category table 297 of FIG. 8, the recognition category (character type) of the recognition category No. 1 is a number. Further, the distance threshold between the character strings of the recognition item (number) of the recognition item No. = 4 is 10 mm. Therefore, in the image recognition device 2, when a plurality of character strings containing only numbers are lined up to the right or below the headword indicating the date of birth, and the distance between the character strings is within the threshold value (within 10 mm). , The plurality of character strings are combined and regarded as one character string. However, the number of digits of the recognition item (date of birth) of the recognition item No. = 4 is 8. Therefore, when the character string located to the right or below the headword indicating the date of birth has 9 or more digits, the image recognition device 2 makes a correction to delete the 9th and subsequent digits. In addition, the final character of the recognition item (date of birth) of recognition item No. = 4 is "day". Therefore, even if the character string located to the right or below the headword indicating the date of birth is 8 digits or less, the image recognition device 2 includes "day" in the character string. Make a correction to delete the digit to the right of the "day" of the character string.

As described above, the image recognition device 2 according to the present embodiment recognizes characters by the ruled line keyword recognition method, and then confirms the identity of the recognized character strings based on the recognition item table 296 and the recognition category table 297. Make corrections to remove parts that show information that is different from the information available in. Therefore, even if the combination of the heading of the item to be recognized and the information that can be used for identity verification is not surrounded by a ruled line, it is possible to appropriately recognize the information that can be used for identity verification. Become.

Next, the process performed by the image recognition device 2 according to the present embodiment will be described with reference to FIGS. 9, 10, 11A and 11B, 12, 13, 14, and 15A and 15B. ..

FIG. 9 is a flowchart illustrating a process performed by the image recognition device.
After the image recognition device 2 of the present embodiment is started, as shown in FIG. 9, it waits until a reading start command for starting reading an image of an identity verification document is input (step S1). The reading start command is input from, for example, either the input device 4 or the scanner device 3 connected to the image recognition device 2.

Upon receiving the input of the reading start command, the image recognition device 2 reads (acquires) the image of the identity verification document, and performs application processing (step S2) for character recognition of the item used for identity verification in the image. .. The content of the application process in step S2 will be described later.

When the application processing is completed, the image recognition device 2 causes the display device 5 to display the recognition result (step S3). In step S3, the image recognition device 2 creates screen data including, for example, an image of an identity verification document, a type, and a character string (recognition result) of an item used for identity verification, and displays the screen data on the display device 5. Further, in step S3, the image recognition device 2 is in a state where, for example, the image recognition device 2 can accept corrections to the recognition result by the operator.

After displaying the recognition result, the image recognition device 2 determines whether or not to correct the recognition result (step S4). Step S4 is, for example, a process of accepting the operator's selection of whether or not to modify the recognition result. When the operator chooses to modify (step S4; YES), the image recognition device 2 accepts the input of the modification information and modifies the recognition result (step S5). When the correction of the recognition result is completed, the image recognition device 2 transmits the recognition result to the server device 6 (step S6). If the operator chooses not to modify (step S4; NO), the image recognition device 2 omits the process of step S5 and transmits the recognition result to the server device 6.

When the process of step S6 is completed, the image recognition device 2 ends the recognition process for the image of one identity verification document. After that, the image recognition device 2 waits until the reading start command is input (step S1).

Next, the contents of the application processing in step S2 will be described. The image recognition device 2 of the present embodiment performs, for example, a process according to the flowchart of FIG. 10 as an application process.

FIG. 10 is a flowchart illustrating the contents of the application process according to the embodiment.

When the application process is started, the image recognition device 2 first acquires an image of the identity verification document as shown in FIG. 10 (step S11). The processing of step S11 is performed by, for example, the scanner control unit 201. The scanner control unit 201 acquires an image (digital data) of an identity verification document from the scanner device 3.

Next, the image recognition device 2 removes the background pattern of the acquired image and then binarizes the image (step S12). The process of step S12 is performed by, for example, the main control unit 209. The main control unit 209 removes the background pattern from the image of the identity verification document according to a known method in the field of image processing, and binarizes the image from which the background pattern has been removed. When binarizing an image from which the background pattern has been removed, the main control unit 209 separates the pixels in the image into pixels showing characters and ruled lines and pixels other than the pixels.

Next, the image recognition device 2 performs character recognition for the entire binarized image (step S13). The process of step S13 is performed by, for example, the main control unit 209. The main control unit 209 recognizes characters in the image binarized according to a method known in the field of image processing, for example.

Next, the image recognition device 2 extracts a character string from the entire binarized image based on the result of character recognition in step S13 (step S14). The process of step S14 is performed by, for example, the main control unit 209. The main control unit 209 extracts, for example, a set of a plurality of characters arranged in the horizontal direction and having a distance between the characters within the threshold value as one character string among a plurality of characters in the binarized image. do.

Next, the image recognition device 2 identifies the type of the identity verification document in the image based on the character string extracted from the binarized image and the document identification table 291 (step S15). I do. The processing of step S15 is performed by the document type identification unit 202. The document type identification unit 202 compares the position of the character string with the position of the identification keyword in the document identification table 291 for each character string extracted from the image, and the degree of matching between the identification keyword and the character string whose positions substantially match. Calculate the score indicating. After that, the document type identification unit 202 identifies the type of the identity verification document in the image based on the calculated score. The detailed contents of the process of step S15 will be described later.

After completing the identity verification document identification process, the image recognition device 2 next determines whether or not the type of the identity verification document in the image can be identified (step S16). The determination in step S16 is performed by the document type identification unit 202. If the identification cannot be performed (step S16; NO), the document type identification unit 202 omits the processing of steps S17 to S21 and returns the character recognition result to the main control unit 209 (step S22). If the identification could not be made (step S16; NO), the document type identification unit 202 indicates that the type of the identity verification document in the image could not be identified in the process of step S22, for example. Is returned to the main control unit 209 as a result of character recognition. When the process of step S22 is completed, the image recognition device 2 ends the application process and causes the display device 5 to display the recognition result (step S3 in FIG. 9).

On the other hand, if it can be identified (step S16; YES), the image recognition device 2 then determines the recognition method for character recognition based on the document type table 292 and the recognition method table 293 (step S17). .. For example, the document type identification unit 202 and the recognition method selection unit 203 cooperate with each other in the process of step S17. The document type identification unit 202 identifies the recognition method No. based on the identification result (value of the document No.) of the identity verification document and the document type table 292, and the recognition method No. is passed through the main control unit 209. Is notified to the recognition method selection unit 203. The recognition method selection unit 203 refers to the recognition method table 293 and determines the recognition method associated with the notified recognition method No. as the recognition method for character recognition for the image currently being processed.

Next, the image recognition device 2 determines whether the recognition method determined in step S17 is a fixed position recognition method, a keyword recognition method, or a non-ruled line keyword recognition method (step S18). The determination in step S18 is performed by, for example, the item position specifying unit 204. When the recognition method determined in step S17 is a fixed position recognition method (step S18; fixed position recognition), the image recognition device 2 performs character recognition processing (step S19) by the fixed position recognition method. The character recognition process in step S19 is performed by, for example, the position fixing recognition unit 211 of the item position specifying unit 204 and the character recognition unit 206 in cooperation with each other. The fixed position recognition unit 211 specifies the position of an item (character string) for character recognition in the image with reference to the fixed position recognition table 294. The character recognition unit 206 refers to the print character recognition dictionary 298 and performs character recognition of an item (character string) whose position is specified. The method of specifying the position of the item for character recognition and the method of character recognition may be known methods in the field of image recognition, respectively. When the process of step S19 is completed, the character recognition unit 206 returns the result of character recognition to the main control unit 209 (step S22), and ends the application process.

When the recognition method determined in step S17 is the keyword recognition method (step S18; keyword recognition), the image recognition device 2 performs character recognition processing (step S20) by the keyword recognition method. The character recognition process in step S20 is performed, for example, by the keyword recognition unit 212 of the item position specifying unit 204 and the character recognition unit 206 in cooperation with each other. The keyword recognition unit 212 specifies, for example, the position of the headword of the item to be recognized in the image by referring to the recognition item keyword table 295, and the range in which the character recognition is performed based on the position of the headword and the ruled line. To identify. The character recognition unit 206 refers to the type character recognition dictionary 298 and recognizes characters within the specified range. The method for specifying the range for character recognition and the method for character recognition may be known methods in the field of image recognition, respectively. When the process of step S20 is completed, the character recognition unit 206 returns the result of character recognition to the main control unit 209 (step S22), and ends the application process.

When the recognition method determined in step S17 is the ruled line keyword recognition method (step S18; ruled line keyword recognition), the image recognition device 2 performs character recognition processing (step S21) by the ruled line keyword recognition method. The character recognition process in step S21 is performed by, for example, the non-ruled line keyword recognition unit 213 of the item position specifying unit 204, the character recognition unit 206, and the recognition result correction unit 205 in cooperation with each other. The non-ruled line keyword recognition unit 213 specifies, for example, the position of the headword of the item to be recognized in the image with reference to the recognition item keyword table 295, and performs character recognition on the right side or the lower side of the headword. The range. The character recognition unit 206 refers to the type character recognition dictionary 298 and performs character recognition in a specified range. The recognition result correction unit 205 corrects the information included in the character recognition result that is not used for identity verification. The details of the character recognition process (step S21) by the non-ruled line keyword recognition method will be described later. When the process of step S21 is completed, the character recognition unit 206 returns the result of character recognition to the main control unit 209 (step S22), and ends the application process.

As described above, in the application processing, the image recognition device 2 identifies the type of the identity verification document based on the character string extracted from the image of the identity verification document and the document identification table 291 and performs character recognition based on the identification result. Determine the recognition method. The image recognition device 2 performs, for example, a process according to the flowcharts of FIGS. 11A and 11B as the identity verification document identification process (step S15) for identifying the type of the identity verification document.

FIG. 11A is a flowchart (No. 1) for explaining the content of the identity verification document identification process. FIG. 11B is a flowchart (No. 2) for explaining the content of the identity verification document identification process.

When the identity verification document identification process is started, the image recognition device 2 first performs the first loop process (steps S31 to S39) as shown in FIG. 11A. The first loop process is performed by the document type identification unit 202. In the first loop processing, each of the plurality of character strings extracted from the binarized image is identified by comparing the position and the character sequence of the character string with each of the identification keywords in the document identification table 291 for each character string. It is a process to set a score for each of the keywords.

At the beginning of the first loop process (step S31), the document type identification unit 202 selects one of a plurality of character strings extracted from the binarized image.

Next, the document type identification unit 202 compares the selected character string with the position of each of the identification keywords in the document identification table 291 and the sequence of characters in the character string, and sets a score for each of the identification keywords. Loop processing (steps S32 to S38) is performed.

At the beginning of the second loop process (step S32), the document type identification unit 202 selects and reads one record (one line) in the document identification table 291.

Next, the document type identification unit 202 indicates whether the coordinates indicating the position (range) of the currently selected character string in the image are within the coordinates of the currently selected record in the document identification table 291. It is determined whether or not (step S33). For example, when the record in the first row in the document identification table 291 of FIG. 3 is selected, the document type identification unit 202 has coordinates indicating the position (range) of the currently selected character string in the image. , (X1, Y1) is the upper left corner portion, and (X2, Y2) is the lower right corner portion. If it does not fit (step S33; NO), the document type identification unit 202 omits the processing of steps S34 to S37, and determines whether or not there is an unselected record at the end of the second loop processing (step S38). judge. If there is an unselected record, the document type identification unit 202 repeats the processes after step S32. If all the records have been selected, the document type identification unit 202 ends the second loop process.

Here, the process returns to the description of the determination in step S33. If the coordinates of the currently selected character string are within the coordinates of the currently selected record (step S33; YES), the document type identification unit 202 is then currently selected in the document identification table 291. It is determined whether or not the character sequence of the identification keyword of the existing record and the currently selected character string match (step S34). When the character arrangements match (step S34; YES), the document type identification unit 202 sets 100 points for the score of the currently selected record in the document identification table 291 (step S35). When the process of step S35 is completed, the document type identification unit 202 determines whether or not there is an unselected record at the end of the second loop process (step S38).

On the other hand, when the character arrangements do not match (step S34; NO), the document type identification unit 202 is next selected as the identification keyword of the currently selected record in the document identification table 291. It is determined whether or not the arrangement of characters with the character string partially matches (step S36). When the character arrangements are partially matched (step S36; YES), the document type identification unit 202 sets the score of the currently selected record in the document identification table 291 to {(number of matching characters / number of characters of identification keyword) ×. The 100} point is set (step S37). When the process of step S37 is completed, the document type identification unit 202 determines whether or not there is an unselected record at the end of the second loop process (step S38). Further, when the character arrangement does not partially match (step S36; NO), the document type identification unit 202 omits the process of step S37, and the unselected record at the end of the second loop process (step S38). Determine if there is.

At the end of the second loop processing (step S38), the document type identification unit 202 indicates, for example, whether or not there is an unselected record (row) in the document identification table 291 in the second loop processing currently being performed. Is determined. If there is an unselected record, the document type identification unit 202 repeats the processes after step S32. On the other hand, when all the records have been selected, the document type identification unit 202 ends the second loop process.

When the second loop processing (steps S32 to S38) is completed, the document type identification unit 202 is then selected from the character strings extracted from the image at the end of the first loop processing (step S39). Determine if there is a string. If there is an unselected character string, the document type identification unit 202 repeats the processes after step S31. On the other hand, when all the character strings have been selected, the document type identification unit 202 ends the first loop process (steps S31 to S39).

When the first loop process is completed, the document type identification unit 202 then initializes the maximum score (step S40) and initializes the document No. indicating the identity verification document with the maximum score, as shown in FIG. 11B. (Step S41). The document No. indicating the maximum score and the identity verification document with the maximum score is stored in, for example, a buffer (not shown) included in the storage unit 290. The document type identification unit 202 sets the maximum score stored in the buffer and the value of the document No. to predetermined values (for example, 0).

Next, the document type identification unit 202 performs a third loop process (steps S42 to S51). The third loop process is a process of calculating the total score (total score) of the document identification table 291 for each identity verification document registered in the document type table 292 and determining the identity verification document having the maximum total score. Is.

At the beginning of the third loop process (step S42), the document type identification unit 202 selects and reads one record (one row) in the document type table 292.

Next, the document type identification unit 202 initializes the total score associated with the document No. included in the selected record (step S43). The total score associated with the document No. is stored in, for example, a buffer (not shown) included in the storage unit 290. The document type identification unit 202 sets the total score stored in the buffer to 0.

Next, the document type identification unit 202 performs a fourth loop process (steps S44 to S47) for calculating the total score (total score) of the records of the currently selected document No. in the document identification table 291. ..

At the beginning of the fourth loop process (step S44), the document type identification unit 202 selects and reads one record (one line) in the document identification table 291.

Next, the document type identification unit 202 determines the document No. included in the record in the currently selected document type table 292 and the document No. included in the record in the currently selected document identification table 291. It is determined whether or not they match (step S45). If they match (step S45; YES), the document type identification unit 202 adds the score of the record in the currently selected document identification table 291 to the total score (step S46). After the processing of step S46, the document type identification unit 202 determines whether or not there is an unselected record in the document identification table 291 at the end of the fourth loop processing (step S47). If the document numbers do not match (step S45; NO), the document type identification unit 202 omits the process of step S46 and enters the document identification table 291 at the end of the fourth loop process (step S47). Determine if there are unselected records. If there is an unselected record in the document identification table 291, the document type identification unit 202 repeats the processes after step S44. Then, when all the records in the document identification table 291 have been selected, the document type identification unit 202 ends the fourth loop process (steps S44 to S47).

After finishing the fourth loop process, the document type identification unit 202 then determines whether or not the total score calculated in the fourth loop process is larger than the current maximum score (step S48). When the total score> the maximum score (step S48; YES), the document type identification unit 202 sets the document No. indicating the identity verification document with the maximum score to the document included in the currently selected record in the document type table 292. No. is set (step S49). After that, the document type identification unit 202 sets the total score calculated in the fourth loop process as the maximum score (step S50).

After completing the processes of steps S49 and S50, the document type identification unit 202 determines whether or not there is an unselected record (row) in the document type table 292 at the end of the third loop process (step S51). do. If there is an unselected record, the document type identification unit 202 repeats the processes after step S42. Then, when all the records have been selected, the document type identification unit 202 ends the third loop process (steps S42 to S51). When the third loop process is completed, the document type identification unit 202 returns the document No. indicating the identity verification document with the maximum score to the main control unit 209 (step S52), and ends the identity verification document identification process.

In this way, in the identity verification document identification process, the score indicating the position and the similarity of the character arrangement between the character string extracted from the identity verification document and the identification keyword that can identify the identity verification document specified for each identity verification document. And identify the identity verification document that maximizes the score.

After completing the identity verification document identification process, the image recognition device 2 determines the character recognition recognition method based on the identification result of the identity verification document (step S17), and the characters according to the determined recognition method. The recognition process (one of steps S19, S20, and S21) is performed. The character recognition process by the fixed position recognition method (step S19) and the character recognition process by the keyword recognition method (step S20) may be performed according to known character recognition processes, respectively. Therefore, in the present specification, the description of the contents of the character recognition processing in steps S19 and S20 will be omitted. On the other hand, as the character recognition process (step S21) by the ruled line keyword recognition method, the image recognition device 2 performs the process according to the flowchart of FIG. 12, for example.

FIG. 12 is a flowchart illustrating the content of the character recognition process by the ruled line keyword recognition method.

When the character recognition process by the ruled line keyword recognition method is started, the image recognition device 2 performs the first loop process (steps S61 to S69) as shown in FIG. In the first loop processing, the position of the character string (heading character string) in the image corresponding to the item keyword of the record is set for each record of the document No. indicating the identity verification document in the image in the recognition item keyword table 295. It is a process of specifying and recognizing the character of the character string.

At the beginning of the first loop process (step S61), the image recognition device 2 selects and reads one record (one row) in the recognition item keyword table 295. The process of step S61 is performed by the non-ruled line keyword recognition unit 213 of the item position specifying unit 204.

Next, the image recognition device 2 performs a second loop process (steps S62 to S68) of specifying the position of the item for character recognition in the image, performing character recognition, and correcting the recognition result.

At the beginning of the second loop process (step S62), the image recognition device 2 selects one of the character strings extracted from the image of the identity verification document. The process of step S62 is performed by the non-ruled line keyword recognition unit 213.

Next, the non-ruled line keyword recognition unit 213 determines whether or not the item keyword in the currently selected recognition item keyword table 295 matches the character string selected in step S62 (step S63). When the item keyword and the character string do not match (step S63; NO), the non-ruled line keyword recognition unit 213 omits the processing of steps S64 to S67 and is not selected at the end of the second loop processing (step S68). Determine if there is a character string. If there is an unselected character string, the non-ruled line keyword recognition unit 213 repeats the processes after step S62. On the other hand, when all the character strings have been selected, the ruled line keyword recognition unit 213 ends the second loop process (steps S62 to S68).

Here, the process returns to the description of the determination in step S63. When the item keyword and the character string match (step S63; YES), the non-ruled line keyword recognition unit 213 is on the right side of the character string (heading character string) that matches the item keyword in the image selected in step S62. , Performs a right-hand search process (step S64) for searching for a character string indicating information that can be used for identity verification corresponding to the character string. The content of the right search process will be described later.

When the right-side search process is completed, the non-ruled line keyword recognition unit 213 determines whether or not there is a character string corresponding to the character string indicating information that can be used for identity verification to the right of the character string matching the item keyword. Is determined (step S65). When there is a character string corresponding to the character string on the right side of the character string matching the item keyword (step S65; YES), the non-ruled line keyword recognition unit 213 cooperates with the character recognition unit 206 and the recognition result correction unit 205. Then, a data character string correction process (step S67) including character recognition of the character string corresponding to the item keyword and correction of the recognition result is performed. The content of the data character string correction process in step S67 will be described later.

On the other hand, when there is no character string corresponding to the character string on the right side of the character string (heading character string) matching the item keyword (step S65; NO), the non-ruled line keyword recognition unit 213 is then in the image. A downward search process (step S66) is performed to search for a character string indicating information that can be used for identity verification corresponding to the character string below the character string that matches the item keyword of. The content of the downward search process will be described later. When the downward search process in step S66 is completed, the non-ruled line keyword recognition unit 213 performs the data character string correction process (step S67) in cooperation with the character recognition unit 206 and the recognition result correction unit 205. The content of the data character string correction process in step S67 will be described later.

When the data character string correction process of step S67 is completed, the ruled line keyword recognition unit 213 determines whether or not there is an unselected character string at the end of the second loop process (step S68). If there is an unselected character string, the non-ruled line keyword recognition unit 213 repeats the processes after step S62. On the other hand, when all the character strings have been selected, the ruled line keyword recognition unit 213 ends the second loop process (steps S62 to S68).

When the second loop processing is completed, the non-ruled line keyword recognition unit 213 determines whether or not there is an unselected record (row) in the recognition item keyword table 295 at the end of the first loop processing (step S69). judge. If there is an unselected record, the non-ruled line keyword recognition unit 213 repeats the processes after step S61. When all the records in the recognition item keyword table 295 have been selected, the non-ruled line keyword recognition unit 213 ends the first loop processing (steps S61 to S69).

When the first loop processing is completed, for example, the non-ruled line keyword recognition unit 213 notifies the main control unit 209 of the recognition results of all the recognition items (step S70), and ends the character recognition processing by the non-ruled line keyword recognition method. do.

Next, the contents of the right side search process (step S64) will be described with reference to FIG. FIG. 13 is a flowchart illustrating the contents of the right side search process.

When the right-side search process is started, the non-ruled line keyword recognition unit 213 first acquires the coordinates of the heading character string as shown in FIG. 13 (step S81). The heading character string is a character string (character string selected in step S62 of FIG. 12) extracted from the image in the right-hand search process currently being performed, and is the recognition item keyword table 295 selected in step S61 of FIG. A character string that matches the item keyword contained in the record.

Next, the non-ruled line keyword recognition unit 213 is the first loop process (steps S82 to S85) of searching for a data character string candidate (data candidate character string) corresponding to the heading character string on the right side of the heading character string. I do.

At the beginning of the first loop process (step S82), the ruled line keyword recognition unit 213 selects, for example, one of the character strings to the right of the heading character string among the character strings extracted from the image.

Next, the non-ruled line keyword recognition unit 213 determines whether or not the coordinate range of the height of the heading character string includes the coordinate range of the height of the character string selected in step S82 (step S83). When the coordinate range of the height of the heading character string includes the coordinate range of the height of the selected character string (step S83; YES), the ruled line keyword recognition unit 213 holds the selected character string as a data candidate character string. (Step S84). After the processing of step S84, the non-ruled line keyword recognition unit 213 determines whether or not there is an unselected character string at the end of the first loop processing (step S85). Further, when the coordinate range of the height of the heading character string does not include the coordinate range of the height of the selected character string (step S83; NO), the ruled line keyword recognition unit 213 omits the process of step S84. At the end of the first loop processing (step S85), it is determined whether or not there is an unselected character string.

If there is an unselected character string, the non-ruled line keyword recognition unit 213 repeats the processes after step S82. When all the character strings have been selected, the non-ruled line keyword recognition unit 213 ends the first loop processing (steps S82 to S85), and then determines whether or not there is a data candidate character string (). Step S86). When there is no data candidate character string (step S86; NO), the non-ruled line keyword recognition unit 213 omits the processing after step S87 and ends the right side search processing.

On the other hand, when there is a data candidate character string (step S86; YES), the non-ruled line keyword recognition unit 213 next, data including information that can be used for identity verification corresponding to the heading character string from the data candidate character strings. The second loop processing (steps S87 to S94) for extracting the character string is performed. In the second loop processing, it is determined that when an item keyword different from the heading character string is detected in the data candidate character string, or all the data candidate character strings are not different item keywords from the heading character string. If you end up.

At the beginning of the second loop process (step S87), the non-ruled line keyword recognition unit 213 selects one of the data candidate character strings. When there are a plurality of data candidate character strings, the non-ruled line keyword recognition unit 213 starts from the data candidate character string whose position is on the left side (in other words, the data candidate character string close to the heading character string) among the plurality of data candidate character strings. Select in order.

Next, the non-ruled line keyword recognition unit 213 performs a third loop process (steps S88 to S91) for determining whether or not the selected data candidate character string is an item keyword different from the heading character string.

At the beginning of the third loop process (step S88), the non-ruled line keyword recognition unit 213 selects and reads one record (one row) in the recognition item keyword table 295.

Next, the non-ruled line keyword recognition unit 213 determines whether or not the item keyword included in the record selected from the recognition item keyword table 295 matches the data candidate character string (step S89). When the item keyword and the data candidate character string match (step S89; YES), the non-ruled line keyword recognition unit 213 turns on the separate heading flag (step S90). After the processing of step S90, the non-ruled line keyword recognition unit 213 determines whether the separate heading flag is ON or all the data candidate character strings are selected at the end of the third loop processing (step S91). judge. If the item keyword and the data candidate character string do not match (step S89; NO), the non-ruled line keyword recognition unit 213 omits the process of step S90 and at the end of the third loop process (step S91). It is determined whether the separate heading flag is ON or whether all the data candidate character strings are selected. When the separate heading flag is not turned ON and there is an unselected data candidate character string, the ruled line keyword recognition unit 213 repeats the processes after step S88. When the separate heading flag is turned ON, or when all the data candidate character strings are selected, the ruled line keyword recognition unit 213 ends the third loop process (steps S88 to S91).

When the third loop process is completed, the non-ruled line keyword recognition unit 213 next determines whether or not the separate heading flag is ON (step S92). When the separate heading flag is not turned on (step S92; NO), the non-ruled line keyword recognition unit 213 uses the currently selected data candidate character string as information that can be used for identity verification corresponding to the heading character string. It is retained as a data character string to be included (step S93). After the process of step S93, the ruled line keyword recognition unit 213 determines whether or not there is an unselected data candidate character string at the end of the second loop process (step S94). If there is an unselected data candidate character string, the non-ruled line keyword recognition unit 213 repeats the processing after step S87. When all the data candidate character strings have been selected, the non-ruled line keyword recognition unit 213 ends the second loop process (steps S87 to S94) and returns the retained data character string (step). S95).

Here, the process returns to the description of the determination in step S92. When the separate heading flag is ON after the end of the third loop processing (step S92; YES), the non-ruled line keyword recognition unit 213 ends the second loop processing (steps S87 to S94) and holds the second loop processing. The data character string is returned (step S95).

In this way, in the right search process, among the character strings located to the right of the heading character string that matches the item keyword, the character string that does not match the item keyword in the recognition item keyword table 295 corresponds to the heading character string. Extract as a data string containing information that can be used for identity verification. Further, in the right search process, there are a plurality of character strings on the right side of the heading character string that matches the item keyword, and a character string that matches the item keyword of the recognition item keyword table 295 in the plurality of character strings (separately). When the heading character string) is included, only the character string located between the character string and the heading character string is extracted as a data character string containing information that can be used for identity verification corresponding to the heading character string.

If the data character string including the information that can be used for identity verification corresponding to the heading character string cannot be extracted in the right-side search process, the ruled line keyword recognition unit 213 moves downward as shown in FIG. The search process (step S66) is performed. The non-ruled line keyword recognition unit 213 performs, for example, a process according to the flowchart of FIG. 14 as a downward search process.

FIG. 14 is a flowchart illustrating the content of the downward search process.
When the downward search process is started, the non-ruled line keyword recognition unit 213 first acquires the coordinates of the heading character string as shown in FIG. 14 (step S101). The heading character string is a character string (character string selected in step S62 of FIG. 12) extracted from the image in the right-hand search process currently being performed, and is the recognition item keyword table 295 selected in step S61 of FIG. A character string that matches the item keyword contained in the record.

Next, the non-ruled line keyword recognition unit 213 performs a first loop process (steps S102 to S105) for searching a candidate for a data character string (data candidate character string) corresponding to the heading character string below the heading character string. conduct.

At the beginning of the first loop process (step S102), the non-ruled line keyword recognition unit 213 selects, for example, one of the character strings below the heading character string among the character strings extracted from the image.

Next, the non-ruled line keyword recognition unit 213 determines whether or not the coordinate range of the width of the heading character string includes the coordinate range of the width of the character string selected in step S102 (step S103). When the coordinate range of the width of the heading character string includes the coordinate range of the width of the selected character string (step S103; YES), the ruled line keyword recognition unit 213 holds the selected character string as a data candidate character string (step S103; YES). Step S104). After the processing of step S104, the ruled line keyword recognition unit 213 determines whether or not there is an unselected character string at the end of the first loop processing (step S105). Further, when the coordinate range of the width of the heading character string does not include the coordinate range of the width of the selected character string (step S103; NO), the ruled line keyword recognition unit 213 omits the process of step S104 and first At the end of the loop processing (step S105), it is determined whether or not there is an unselected character string.

If there is an unselected character string, the non-ruled line keyword recognition unit 213 repeats the processing after step S102. When all the character strings have been selected, the ruled line keyword recognition unit 213 ends the first loop processing (steps S102 to S105) and determines whether or not there is a data candidate character string (step S106). .. When there is no data candidate character string (step S106; NO), the non-ruled line keyword recognition unit 213 omits the processing after step S107 and ends the downward search processing.

On the other hand, when there is a data candidate character string (step S106; YES), the non-ruled line keyword recognition unit 213 next, data including information that can be used for identity verification corresponding to the heading character string from the data candidate character strings. The second loop processing (steps S107 to S114) for extracting the character string is performed. In the second loop processing, it is determined that when an item keyword different from the heading character string is detected in the data candidate character string, or all the data candidate character strings are not different item keywords from the heading character string. If you end up.

At the beginning of the second loop process (step S107), the non-ruled line keyword recognition unit 213 selects one of the data candidate character strings. When there are a plurality of data candidate character strings, the non-ruled line keyword recognition unit 213 sequentially starts with the data candidate character string whose position is higher (in other words, the data candidate character string closer to the heading character string) among the plurality of data candidate character strings. select.

Next, the non-ruled line keyword recognition unit 213 performs a third loop process (steps S108 to S111) for determining whether or not the selected data candidate character string is an item keyword different from the heading character string.

At the beginning of the third loop process (step S108), the non-ruled line keyword recognition unit 213 selects and reads one record (one row) in the recognition item keyword table 295.

Next, the non-ruled line keyword recognition unit 213 determines whether or not the item keyword included in the record selected from the recognition item keyword table 295 matches the data candidate character string (step S109). When the item keyword and the data candidate character string match (step S109; YES), the non-ruled line keyword recognition unit 213 turns on the separate heading flag (step S110). After the processing of step S110, the non-ruled line keyword recognition unit 213 determines whether the separate heading flag is ON or all the data candidate character strings are selected at the end of the third loop processing (step S111). judge. If the item keyword and the data candidate character string do not match (step S109; NO), the non-ruled line keyword recognition unit 213 omits the process of step S110 and at the end of the third loop process (step S111). It is determined whether the separate heading flag is ON or whether all the data candidate character strings are selected. When the separate heading flag is not turned ON and there is an unselected data candidate character string, the ruled line keyword recognition unit 213 repeats the processes after step S108. When the separate heading flag is turned ON, or when all the data candidate character strings are selected, the ruled line keyword recognition unit 213 ends the third loop process (steps S108 to S111).

When the third loop process is completed, the non-ruled line keyword recognition unit 213 next determines whether or not the separate heading flag is ON (step S112), and if the separate heading flag is not ON (step S112). Step S112; NO), the non-ruled line keyword recognition unit 213 holds the currently selected data candidate character string as a data character string including information that can be used for identity verification corresponding to the heading character string (step S113). .. After the processing of step S113, the non-ruled line keyword recognition unit 213 determines whether or not there is an unselected data candidate character string at the end of the second loop processing (step S114). If there is an unselected data candidate character string, the non-ruled line keyword recognition unit 213 repeats the processes after step S107. When all the data candidate character strings have been selected, the non-ruled line keyword recognition unit ends the second loop process (steps S107 to S114) and returns the retained data character string (step S115). ).

Here, the process returns to the description of the determination in step S112. When the separate heading flag is ON after the end of the third loop process (step S112; YES), the non-ruled line keyword recognition unit 213 ends the second loop process (steps S107 to S114) and holds the second loop process. The data character string is returned (step S115).

In this way, in the downward search process, among the character strings located below the heading character string that matches the item keyword, the character string that does not match the item keyword in the recognition item keyword table 295 is the person corresponding to the heading character string. Extract as a data string containing information that can be used for confirmation. Further, in the downward search process, there are a plurality of character strings below the heading character string that matches the item keyword, and a character string that matches the item keyword of the recognition item keyword table 295 in the plurality of character strings (separate heading character). (Column) is included, only the character string located between the character string and the heading character string is extracted as a data character string containing information that can be used for identity verification corresponding to the heading character string.

After extracting the data character string including the information that can be used for identity verification corresponding to the heading character string by the above-mentioned right-side search process or downward search process, the image recognition device 2 performs character recognition of the extracted data character string. Then, a data character string correction process (step S67 in FIG. 12) for correcting the result of the character recognition is performed. The data character string correction process is performed by the recognition result correction unit 205 and the character recognition unit 206 in cooperation with each other. The recognition result correction unit 205 and the character recognition unit 206 perform data character string correction processing according to, for example, the flowcharts of FIGS. 15A and 15B.

FIG. 15A is a flowchart (No. 1) for explaining the content of the data character string correction process. FIG. 15B is a flowchart (No. 2) for explaining the content of the data character string correction process.

When the data character string correction process is started, the image recognition device 2 first extracts the first character string (step S121) and holds the first character string as the correction data character string (step), as shown in FIG. 15A. S122). The processing of steps S121 and S122 is performed by, for example, the recognition result correction unit 205. The recognition result correction unit 205 sets the leftmost data character string (in other words, the data character string closest to the heading character string) among the data character strings corresponding to the heading character string as the first character string.

Next, the recognition result correction unit 205 determines whether or not there is one data character string (step S123). When there are a plurality of data character strings (step S123; NO), the recognition result correction unit 205 then determines, for each data character string, whether or not the distance between adjacent character strings is within the distance threshold. The first loop processing (steps S124 to S129) is performed.

At the beginning of the first loop processing (step S124), the recognition result correction unit 205 sequentially selects the data character string located on the leftmost side among the other data character strings excluding the data character string extracted as the first character string. select.

After selecting the data character string, the recognition result correction unit 205 extracts the selected data character string as the next character string (step S125), and extracts the next character string (data character string) and the data character string. The distance from the data character string to the left of is calculated (step S126).

Next, the recognition result correction unit 205 determines whether or not the calculated distance is within the distance threshold value between the character strings in the recognition item table 296 (step S127). In the process of step S127, the recognition result correction unit 205 sets the distance between the character strings as the distance based on the distance threshold between the character strings included in the record of the recognition item No. corresponding to the heading character string in the recognition item table 296. Determine if it is within the threshold.

When the distance between the character strings is within the distance threshold (step S127; YES), the recognition result correction unit 205 holds the data character string (next character string) extracted in step S125 as the correction data character string (step S127; YES). Step S128). After the processing of step S128, the recognition result correction unit 205 determines whether or not there is an unselected data character string at the end of the first loop processing (step S129). If there is an unselected data character string, the recognition result correction unit 205 repeats the processes after step S124.

On the other hand, when all the data character strings have been selected, the recognition result correction unit 205 ends the first loop processing (steps S124 to S129). When the distance calculated in step S126 is longer than the distance threshold between the character strings in the recognition item table 296 (step S127; NO), the recognition result correction unit 205 performs the first loop processing (steps S124 to S129). finish.

When the first loop processing is completed, the recognition result correction unit 205 determines whether or not the holding correction data character string is two or more (step S130). Further, when it is determined in step S123 that there is only one data character string (step 123; YES), the recognition result correction unit 205 omits the first loop processing itself and performs the determination in step S130.

When the number of the retained correction data character strings is two or more (step S130; YES), the recognition result correction unit 205 combines the two or more correction data character strings into one data character string (step). S131), the combined data character string is notified to the character recognition unit 206. On the other hand, when there is only one correction data character string held (step S130; NO), the recognition result correction unit 205 omits the processing of step S131 and uses one holding correction data character string as a character. Notify the recognition unit 206. The character recognition unit 206 performs character recognition of one notified data character string (step S132), and notifies (replies) the recognition result correction unit 205 of the result of character recognition.

Upon receiving the character recognition result, the recognition result correction unit 205 determines whether or not the character recognition result is within the number of digits included in the currently selected record in the recognition item table 296, as shown in FIG. 15B. Determination (step S133). When the result of character recognition is larger than the number of digits included in the currently selected record (step S133; NO), the recognition result correction unit 205 has the number of digits of the currently selected record in the character recognition result + 1. The character string after the digit is deleted (step S134). After step S134, the recognition result correction unit 205 searches for the final character included in the currently selected record in the recognition item table 296 from the character recognition result (step S135). If the character recognition result is within the number of digits included in the currently selected record (step S133; YES), the recognition result correction unit 205 omits the process of step S134 and performs the search process of step S135. conduct.

After the search process in step S135, the recognition result correction unit 205 determines whether or not the final character has been detected from the character recognition result (step S136). When the final character is detected (step S136; YES), the recognition result correction unit 205 deletes the character string after the number of digits (numerical value representing the digit position) + 1 digit of the final character (step S137). For example, when the digit position of the last character is the fifth digit (fifth from the left end), the recognition result correction unit 205 deletes the character string after the 6th (= 5 + 1) digit. After the process of step S137, the recognition result correction unit 205 returns the character recognition result to the main control unit 209 (step S138), and ends the data character string correction process. If the final character is not included (step S136; NO), the recognition result correction unit 205 skips the process of step S137 and returns the character recognition result to the main control unit 209 (step S138), and the data character. End the column correction process.

As described above, in the image recognition device 2 of the present embodiment, the item keyword and the item keyword are based on the number of digits and the final character of the information (character string) that can be used for identity verification corresponding to the item keyword in the recognition item keyword table 295. Delete information other than the information that can be used for identity verification, which is included in the result of character recognition of the character string located to the right or below the character string in the matching image. For this reason, for example, information other than the information that can be used for identity verification, such as a company name, an organization name, a logo, and other additional information, is printed on the right or bottom of the headword in the identity verification document. Even if this is the case, only the information that can be used for identity verification can be extracted and recognized.

FIG. 16 is a diagram showing an example of an identity verification document.
FIG. 16 shows a card-type health insurance card 10 as an example of identity verification documents. The insurance card 10 has the character string 10A "health insurance" and the character string 10B "insured person's card" printed on the upper left corner when viewed in the normal position. Further, the name of the insured person is printed on the health insurance card 10 as one of the information (items) that can be used for identity verification. However, the insurance certificate 10 does not have a ruled line surrounding the set of the character string 10C (headword) of "name" and the character string 10D (name of the insured person) of "Fujitsu Hanako". Further, the health insurance card 10 is issued by, for example, each municipality, and the format is not unified. For example, some card-type health insurance cards may have a character string (headword) of "name" and the position and interval of the insured's name different from those of the health insurance card 10 of FIG. Therefore, when the health insurance card 10 is recognized as an identity verification document by the image recognition device 2, character recognition is performed by the character recognition process (step S19) by the fixed position recognition method and the character recognition process (step S20) by the keyword recognition method. I can't. Therefore, when the health insurance card 10 is recognized by the image recognition device 2 as an identity verification document, the character recognition is performed by the character recognition process (step S21) by the ruled line keyword recognition method.

FIG. 17A is a diagram (No. 1) illustrating an example of a character string extracted from a binarized image. FIG. 17B is a diagram (No. 2) illustrating an example of a character string extracted from a binarized image.

When the health insurance card 10 is recognized by the image recognition device 2 as an identity verification document, first, the image of the health insurance card 10 is acquired by the scanner device 3 or the like (step S11). After that, the image recognition device 2 removes the background pattern of the image of the health insurance card 10 and binarizes it (step S12), and performs character recognition for the entire binarized image (step S13).

In the process of step S13, as shown in FIG. 17A (a), the image recognition device 2 has "health", "yasu", "ho", and "ho" in the partial region 11A around the upper left corner of the image of the health insurance card 10. It recognizes the four characters "rugged" and the five characters "covered", "ho", "rugged", "person", and "proof". Further, the positions of these nine characters in the image are represented by the coordinates of the upper left corner portion and the coordinates of the lower right corner portion of the rectangular area including one character. For example, the position of the character "Ken" is represented by the coordinates (x1, y1) of the upper left corner portion and the coordinates (x2, y2) of the lower right corner portion of the rectangular area 1201 surrounding the character "Ken". Further, for example, the position of the character "covered" is represented by the coordinates (x1, y3) of the upper left corner portion and the coordinates (x2, y4) of the lower right corner portion of the rectangular area 1210 surrounding the character "covered". ..

Further, in the process of step S13, the image recognition device 2 uses the “name” and “name” of the partial area 11B in which the name and date of birth in the image of the health insurance card 10 are described, as shown in FIG. 17B (a). Recognize the two characters of "tomi", "shi", "tsu", "flower", and the five characters of "child". As described above, the position of each character recognized from the partial region 11B is represented by the coordinates of the upper left corner portion and the coordinates of the lower right corner portion of the rectangular region including one character.

After recognizing the characters in the image in step S13, the image recognition device 2 extracts the character string from the entire binarized image (step S14). In the process of step S14, the image recognition device 2 extracts, for example, a plurality of characters that are adjacent to each other in the horizontal direction (x direction) and the distance between the characters is equal to or less than the threshold value as one character string. For example, as shown in FIG. 17A (b), the image recognition device 2 has "health", "yasu", "ho", and "rugged" in the partial region 11A around the upper left corner of the image of the insurance card 10. The four characters of are extracted as one character string 1301 called "health insurance". Further, the image recognition device 2 extracts five characters of "insured", "insurance", "rugged", "person", and "certificate" as one character string 1310 called "insured person's certificate". The position of the character string 1301 "health insurance" extracted from the binarized image (partial area 11A) in the image is the coordinates (x1, y1) of the upper left corner of the rectangular area including the character string 1301 and the lower right corner. It is represented by the coordinates of the part (x8, y2). Similarly, the position of the character string 1310 in the image of the "insured person's card" is the coordinates of the upper left corner (x1, y3) and the coordinates of the lower right corner (x20, y4) of the rectangular area including the character string 1310. It is represented by.

Further, the image recognition device 2 extracts, for example, the two characters "name" and "name" of the partial area 11B as the character string 1320 "name" as shown in (b-1) of FIG. 17B, and "rich". , "Shi", "tsu", "flower", and "ko" are extracted as the character string 1321 "Fujitsu" and the character string 1322 "Hanako". Similarly, the image recognition device 2 is, for example, as shown in (b-1) of FIG. 17B, from the partial area 11B to the character string 1330 of "date of birth", the character string 1331 of "Showa", and "55 years". The character string 1332, the character string 1333 "November", and the character string 1334 "11th" are extracted. Further, the image recognition device 2 extracts, for example, the character string 1335 of "gender" and the character string 1336 of "female" from the partial region 11B. The positions of these character strings 1320 to 1322 and 1330 to 1336 are represented by the coordinates of the upper left corner portion and the coordinates of the lower right corner portion of the rectangular area surrounding the character string, respectively.

The image recognition device 2 is, for example, as shown in FIG. 17B (b-2), insured persons such as “wealth”, “shi”, “tsu”, “flower”, and “child” in the partial region 11B. You may extract the name of "Fujitsu Hanako" as one character string 1329. Similarly, the image recognition device 2 may extract the date of birth of the insured person in the partial area 11B as one character string 1339 "November 11, 1980".

After binarizing the image of the health insurance card 10 and extracting the character string, the image recognition device 2 performs the identity verification document identification process (step S15). The image recognition device 2 performs, for example, a process according to the flowcharts of FIGS. 11A and 11B as the identity verification document identification process.

When performing the processing according to the flowcharts of FIGS. 11A and 11B, the image recognition device 2 first, for example, first extracts from the partial region 11A in the image of the insurance card 10 at the beginning of the first loop processing (step S31). Select the character string 1301 "health insurance". After that, the image recognition device 2 calculates a score indicating the degree of matching between the identification keyword of each record (each line) of the document identification table 291 (see FIG. 3) and the character string 1301.

The identity verification document of document No. = 1 in the document identification table 291 of FIG. 3 is a driver's license, and the identification keyword matching the character string 1301 of "health insurance" in the identification keyword of each record of document No. = 1. And there is no partial matching identification keyword. Therefore, even if there is an identification keyword whose position in the image substantially matches the position in the image of the character string 1301 among the identification keywords of each record of the document No. = 1, the document No. = 1 The score of each record of is 0 points. Similarly, the scores of each record of document No. = 3, each record of document No. = 4, and each record of document No. 5 in the document identification table 291 of FIG. 3 are all 0 points.

On the other hand, among the records of document No. = 2 in the document identification table 291 of FIG. 3, the score of the record whose identification keyword is health insurance is 100 points or a high score close to 100 points.

Further, when the character string 1310 "insured person's card" extracted from the partial area 11A in the image of the health insurance card 10 is selected at the beginning of the first loop process (step S31), the document in the document identification table 291 of FIG. 3 is selected. Of the records with No. = 2, only the score of the record whose identification keyword is the insured person's card is 100 points or a high score close to 100 points.

After that, for example, when the character string 1320 "name" and the character string 1329 "Fujitsu Hanako" extracted from the partial area 11B at the beginning of the first loop processing (step S31) are selected, the score of each record is calculated. , All scores are 0 points or close to 0 points. Therefore, when the third loop processing (steps S42 to S51) is performed after the first loop processing (steps S31 to S39) is completed, the document number of the identity verification document, which is the maximum score, becomes 2. Therefore, the image recognition device 2 recognizes characters in the image of the health insurance card 10 based on the document No. 2 of the identity verification document in the image of the health insurance card 10, the document type table 292, and the recognition method table 293. Is determined to be the non-ruled line keyword recognition method of the recognition method No. 1 (step S17).

The image recognition device 2 performs, for example, the processing according to the flowcharts of FIGS. 12, 13, 14, and 15A and 15B as the character recognition processing (step S21) of the ruled line keyword recognition method. That is, the image recognition device 2 selects and reads one record (one line) in the recognition item keyword table 295, and the character matching the item keyword of the selected record among the character strings extracted from the image of the insurance certificate 10. The right search process (step S64) is performed using the column as a heading character string. For example, the image recognition device 2 selects the record in the fourth row from the top of the recognition item keyword table 295 in FIG. 6, and has the character string "name" 1320 (FIG. 17B) from the character string extracted from the image of the insurance certificate 10. When) is selected, the right-hand search process is performed with the character string 1320 "name" as the heading character string. At this time, the image recognition device 2 first determines whether or not the coordinate range of the height of the heading character string 1320 includes the coordinate range of the height of the character string located to the right of the heading character string 1320. (Step S83). Then, when the coordinate range of the height of the heading character string 1320 includes the coordinate range of the height of the character string located on the right side of the heading character string 1320, the image recognition device 2 is on the right side of the heading character string 1320. Let the character string of be a data candidate character string.

FIG. 18 is a diagram illustrating an example of a method of determining whether or not to use as a data candidate character string in the right-side search process.

18 (a) to 18 (d) show the heading character string 1320 "name" and the information "Fujitsu Hanako" located to the right of the heading character string 1320, which can be used for identity verification, respectively. The relationship between the position and the dimension of the character string 1329 is shown.

The coordinate range H1 of the height of the heading character string 1320 is calculated by the vertical coordinate (y coordinate) y21 of the upper left corner portion and the vertical coordinate (y coordinate) y22 of the lower right corner portion in the rectangular area surrounding the heading character string 1320. .. Therefore, in the image recognition device 2, the coordinate range of the height of each of the character strings located to the right of the heading character string 1320 including the character string 1329 "Fujitsu Hanako" is the coordinate range of the height of the heading character string 1320. It is determined whether or not it is included in H1 (step S83).

For example, in FIG. 18A, the coordinates (x31, y31) of the upper left corner of the character string 1329 "Fujitsu Hanako" are above the upper end of the coordinate range H1 at the height of the heading character string 1320, and are on the right. The coordinates (x38, y38) of the lower corner portion are below the lower end of the coordinate range H1 at the height of the heading character string 1320. In such a case, the coordinate range H1 at the height of the heading character string 1320 includes the coordinate range H2 at the height of the character string 1329 (step S83; YES). Therefore, the image recognition device 2 holds the character string 1329 "Fujitsu Hanako" as a data candidate character string (step S84).

Further, in FIG. 18B, the coordinates (x38, y38) of the lower right corner of the character string 1329 "Fujitsu Hanako" are located between the upper end and the lower end of the coordinate range H1 at the height of the heading character string 1320. be. Even in such a case, the coordinate range H1 at the height of the heading character string 1320 includes the coordinate range H2 at the height of the character string 1329 (step S83; YES). Therefore, the image recognition device 2 holds the character string 1329 "Fujitsu Hanako" as a data candidate character string (step S84).

On the other hand, in (c) of FIG. 18, the coordinates (x38, y38) of the lower right corner of the character string 1329 "Fujitsu Hanako" are above the upper end of the coordinate range H1 of the height of the heading character string 1320. be. In such a case, the coordinate range H1 at the height of the heading character string 1320 does not include the coordinate range H2 at the height of the character string 1329 (step S83; NO). Therefore, the image recognition device 2 does not use the character string 1329 "Fujitsu Hanako" as a data candidate character string.

The determination in step S83 in the flowchart of FIG. 13 is not limited to whether or not the coordinate range of the height of the heading character string includes the coordinate range of the height of the character string on the right, for example, the heading character string. It may include the relationship between the height (character size) and the height of the character string on the right (character size) as a condition. For example, in FIG. 18D, the height coordinate range H1 of the heading character string 1320 "name" includes the height coordinate range H2 of the character string 1329 "Fujitsu Hanako". However, when the upper limit of the height of the character string on the right side is set to the height corresponding to the coordinate range Th1, the height of the character string 1329 becomes larger than the height corresponding to the coordinate range Th1. In such a case, the space between the character string 1329 and another character string adjacent to each other in the height direction (vertical direction) becomes narrow, and there is a high possibility that an error occurs in character recognition. Further, for example, in the example of (a) and (b) of FIG. 18, when the company name or the logo of the organization is printed on the right side of the character string 1329, the company name or the logo is a part of the name. It may be recognized as. In order to prevent such a situation, if the height of the character string on the right side of the heading character string is larger than the height corresponding to the coordinate range Th1, the character string on the right side should not be used as the data candidate character string. You may. The coordinate range Th1 in this case may be appropriately set, and for example, Th1 = 1.7 × H1.

When the first loop process (steps S82 to S85) in the right search process is completed, the image recognition device 2 determines whether or not the data candidate character string is held (step S86). When the data candidate character string is not held (step S86; NO), the image recognition device 2 ends the right side search process, and then performs the downward search process (step S66). On the other hand, when the data candidate character string is held (step S86; YES), the image recognition device 2 performs a second loop process (steps S87 to S94) for extracting the data character string from the data candidate character string.

For example, as shown in FIG. 18A, the character string 1320 "name" is the heading character string, and the height of the character string 1329 to the right of the heading character string is in the coordinate range H1 of the height of the heading character string. When the coordinate range H2 is included, the image recognition device 2 holds the character string 1329 as a data candidate character string. Further, when there is another character string further to the right of the character string 1329 and the coordinate range H1 of the height of the heading character string includes the coordinate range of the other character string height, the image recognition device 2 , Another character string is held as a data candidate character string on the right side of the character string 1329. In this case, the image recognition device 2 performs the second loop processing (steps S87 to S94) after the first loop processing.

In the second loop processing, the data candidate character string that matches the item keyword of the recognition item keyword table 295 in the data candidate character string and another data candidate character string to the right of the data candidate character string are headed. Exclude from the character string (data character string) indicating the information that can be used for identity verification corresponding to the character string. Specifically, as shown in the flowchart of FIG. 13, when the data candidate character string matches the item keyword of the recognition item keyword table 295, the separate heading flag is turned ON, and it is based on whether or not the separate heading flag is ON. It is determined whether or not the data candidate character string is used as the data character string.

FIG. 19 is a diagram illustrating a method of determining on / off of another heading flag in the right-side search process.

First, with reference to FIG. 19A, an example of the second loop processing in the case where the character string 1320 "name" included in the partial area 11B of the image of the health insurance card 10 is a heading character string will be described. There is one character string 1329 on the right side of the heading character string 1320, and the coordinate range of the height of the heading character string 1320 includes the coordinate range of the height of the character string 1329. Therefore, the image recognition device 2 holds the character string 1329 as a data candidate character string (step S84). In this case, the image recognition device 2 performs the second loop processing (steps S86 to S94) after the first loop processing.

In the second loop process, one data candidate character string is selected (step S87), and it is searched whether or not the data candidate character string matches the item keyword in the recognition item keyword table 295 (steps S88 to S91). ). When the character string 1329 "Fujitsu Hanako" shown in FIG. 19 (a) is held as a data candidate character string, all the item keywords in the recognition item keyword table 295 do not match the character string 1329. Therefore, when the character string 1329 is selected as the data candidate character string in the second loop processing, the separate heading flag is not turned ON. Therefore, the image recognition device 2 holds the character string 1329 "Fujitsu Hanako" as a data character string for the character string (heading character string) 1320 "name" (step S93). After completing the second loop processing, the image recognition device 2 returns the character string 1329 "Fujitsu Hanako" as a data character string for the character string (heading character string) 1320 "name" (step S95), and right. The direction search process is terminated.

Further, when the heading character string in the flowchart of FIG. 13 is the character string 1330 "date of birth" shown in FIG. 19B, the image recognition device 2 performs the heading character string in the first loop processing. The character string 1339 "November 11, 1980" located on the right side of 1330 is retained as a data candidate character string. Further, in the first loop processing, the image recognition device 2 holds the character string 1335 "gender" and the character string 1336 "woman" on the right side of the character string 1339 as data candidate character strings.

When the first loop processing is completed, the image recognition device 2 performs the second loop processing. In the second loop processing, first, the character string 1339 "November 11, 1980", which is the closest to the heading character string 1330 among the three data candidate character strings, is selected, and the items in the recognition item keyword table 295 are selected. Compare with each of the keywords. Since the recognition item keyword table 295 does not include the item keyword (character string) indicating a specific date, the separate heading flag is not turned on.

Next, the image recognition device 2 selects the character string 1335 "gender" and compares it with each of the item keywords in the recognition item keyword table 295. The recognition item keyword table 295 of FIG. 6 includes an item keyword of gender. Therefore, when the third loop processing (steps S88 to S91) is completed, the separate heading flag is turned ON (step S92; YES). Therefore, the image recognition device 2 ends the second loop process, returns the data character string 1339 currently held, and ends the right-hand search process. As described above, when the image recognition device 2 according to the present embodiment includes a character string matching another item keyword on the right side of the heading character string matching the item keyword in the recognition item keyword table 295, the other Only the data candidate character string located between the character string matching the item keyword and the heading character string is used as the data character string. This makes it possible to prevent the recognition of a character string such as "gender" that cannot be used for identity verification and the identity verification information that does not correspond to the heading character string as a character string that can be used for identity verification.

In some of the identity verification documents, the identity verification information corresponding to the heading character string is printed below the heading character string. If the above-mentioned right-hand search process is performed after binarizing the image of such an identity verification document and extracting the character string, the search for the character string that can be used for identity verification corresponding to the heading character string fails. When the image recognition device 2 of the present embodiment fails to search for the character string by the right-side search process, the image recognition device 2 performs the downward search process as described above. In the downward search process, the image recognition device 2 extracts the data character string corresponding to the heading character string by the same method as the right side search process. In the downward search process, as shown in the flowchart of FIG. 14, a character string that is below the heading character string and whose width coordinate range is included in the width coordinate range of the heading character string is set as a data candidate character string.

FIG. 20 is a diagram illustrating an example of a method of determining whether or not to use as a data candidate character string in the downward search process.

In FIGS. 20 (a) to 20 (c), the heading character string 1340 "expiration date" and the specific date "March 31, 1989" located below the heading character string 1340 are shown, respectively. The relationship between the position and the dimension of the character string 1341 indicating the above is shown.

The coordinate range W1 of the width of the heading character string 1340 is calculated by the abscissa (x coordinate) of the upper left corner portion and the abscissa (coordinates) of the lower right corner portion in the rectangular area surrounding the heading character string 1340. Therefore, in the image recognition device 2, the coordinate range of the width of each of the character strings located below the heading character string 1340 including the character string 1341 of "March 31, 1989" is the width of the heading character string 1340. It is determined whether or not it is included in the coordinate range W1 (step S103).

For example, in FIG. 20A, the left end of the character string 1341 "March 31, 1989" is to the left of the right end of the heading character string 1340, and the right end of the character string 1341 is the heading character string 1340. It is to the right of the right edge of. In such a case, the coordinate range W1 having the width of the heading character string 1340 includes the coordinate range W2 having the width of the character string 1341 (step S103; YES). Therefore, the image recognition device 2 holds the character string 1341 "March 31, 1989" as a data candidate character string (step S104).

Further, in FIG. 20B, the character string 1341 "March 31, 1989" is to the left of the left end of the heading character string 1340, and the right end of the character string 1341 is the right end of the heading character string 1340. To the right of. Even in such a case, the coordinate range W1 having the width of the heading character string 1340 includes the coordinate range W2 having the width of the character string 1341 (step S103; YES). Therefore, the image recognition device 2 holds the character string 1341 "March 31, 1989" as a data candidate character string (step S104).

On the other hand, in FIG. 20 (c), the left end of the character string 1341 "March 31, 1989" is to the right of the right end of the heading character string 1340. In such a case, the coordinate range W1 having the width of the heading character string 1340 does not include the coordinate range W2 having the width of the character string 1341 (step S103; NO). Therefore, the image recognition device 2 does not use the character string 1341 "March 31, 1989" as a data candidate character string.

As shown in FIG. 20, when the character string 1341 corresponding to the heading character string 1340 is printed below the heading character string 1340 and each character string is written horizontally, it corresponds to the heading character string 1340. It is very rare for a string to be on multiple lines. Further, when the character string 1341 corresponding to the heading character string 1340 is printed below the heading character string 1340 and each character string is written horizontally, the heading character string 1340 is below the heading character string 1340. It may contain different item keywords and parts of the corresponding string. Therefore, when searching for the character string 1341 located below the heading character string 1340, the search range of the character string may be limited, for example, based on the coordinate range H11 of the height of the heading character string 1340. FIG. 20A shows an example in which the search range below the heading character string 1340 is set to 5 times (5 × H11) the coordinate range H11 of the height of the heading character string 1340.

By limiting the search range below the heading character string 1340 in this way, it is possible to prevent the character string unrelated to the heading character string 1340 from being recognized as a part of the character string corresponding to the heading character string 1340. Further, by limiting the search range below the heading character string 1340, the downward search process can be efficiently performed in a short time.

In the character recognition process by the non-ruled line keyword recognition method according to the present embodiment, after searching for a data character string for character recognition by the above-mentioned right side search process or downward search process, the data character string correction process (step S67) is performed. Performs character recognition and correction of data character strings. The image recognition device 2 performs, for example, a process according to the flowcharts of FIGS. 15A and 15B as the data character string correction process. In the data character string correction process, the image recognition device 2 performs a process of combining a plurality of data character strings that are adjacent to each other in the horizontal or vertical direction and whose distance is equal to or less than a threshold value to form one data character string. After that, the image recognition device 2 performs character recognition of each data character string according to a known character recognition method. Further, the image recognition device 2 has a relationship between the character recognition result of the data character string and the heading character string in the character recognition result based on the format in the identity verification document of the recognition item corresponding to the data character string. Make a correction to delete the part without.

FIG. 21 is a diagram illustrating a method for correcting a data character string.
FIG. 21 (a) shows a partial area 11C including a line of names in the image of the health insurance card 10. Of the character strings extracted from the subregion 11C, the character string 1320 "name" matches the item keyword in the recognition item keyword table 295, and thus becomes a heading character string. Therefore, the image recognition device 2 performs a right-side search process using the character string 1320 "name" as a heading character string. In the right-hand search process using the character string 1320 of "name" as the heading character string, the image recognition device 2 uses, for example, the character string 1329 of "Fujitsu Hanako" and the character string 1350 of "generic hope" as data candidate character strings. And. The two data candidate character strings do not match the item keywords in the recognition item keyword table 295. Therefore, the image recognition device 2 sets the character string 1329 "Fujitsu Hanako" and the character string 1350 "generic hope" as data character strings for the character string 1320 "name", and ends the right search process. After completing the right-side search process, the image recognition device 2 omits the downward search process, and then performs the data character string correction process.

In the data character string correction process, the image recognition device 2 first determines, when there are a plurality of data character strings corresponding to the heading character string, a data character string for character recognition from the plurality of data character strings. The first loop processing (steps S124 to S129) is performed. In the first loop processing, first, the character string closest to the heading character string among the plurality of data character strings is the first character string (correction data character string) and the next adjacent to the right of the first character string. It is determined whether or not the distance from the character string of is within the distance threshold between the heading character string (recognition item) and the corresponding character string in the recognition item table. In the example of (a) of FIG. 21, the image recognition device 2 has two character strings, with the character string 1329 "Fujitsu Hanako" as the first character string and the character string 1350 "generic hope" as the next character string. The distance D2 of is calculated. Here, since the heading character string 1320 is a character string "name", the image recognition device 2 acquires, for example, the distance threshold value (10 mm) between the character strings from the recognition item table 296 of FIG. 7. That is, when the calculated distance D2 is within 10 mm (step S127; YES), the image recognition device 2 holds the character string 1350 "generic hope" as a correction data character string (step S128), and the first loop. End the process. In this case, the image recognition device 2 combines the character string 1329 "Fujitsu Hanako" and the character string 1350 "generic hope" into one character string (step S131), and the combined character string "Fujitsu". Character recognition of "Hanako Generic Hope" is performed (step S132). On the other hand, when the calculated distance D2 is more than 10 mm (step S127; NO), the image recognition device 2 ends the first loop process and recognizes the character string 1329 "Fujitsu Hanako".

Further, after performing character recognition, the image recognition device 2 determines whether or not the recognition result is within the number of digits set in the recognition item table 296.

For example, FIG. 21B shows a partial region 11D containing a gender row in the image of health insurance card 10. Of the character strings extracted from the subregion 11D, the character string 1335 "gender" matches the item keyword in the recognition item keyword table 295, and thus becomes a heading character string. Therefore, the image recognition device 2 performs a right-hand search process using the character string 1335 "gender" as a heading character string. In the right-hand search process using the character string 1335 of "gender" as the heading character string, the image recognition device 2 uses, for example, the character string 1336 of "woman" and the character string 1351 of "** company" as data candidate character strings. And. The two data candidate character strings do not match the item keywords in the recognition item keyword table 295. Therefore, the image recognition device 2 sets the character string 1336 of "woman" and the character string 1351 of "** company" as the data character string for the character string 1335 of "gender", and ends the right-hand search process. After completing the right-side search process, the image recognition device 2 omits the downward search process, and then performs the data character string correction process.

In the data character string correction process, the image recognition device 2 uses the character string 1336 as the first character string as the first character string and the character string 1351 as the next character string as the next character string, and the distance D3 between the two character strings. Is calculated. Here, since the heading character string 1335 is a character string of "gender", the image recognition device 2 acquires, for example, the distance threshold value (10 mm) between the character strings from the recognition item table 296 of FIG. 7. That is, when the calculated distance D3 is within 10 mm (step S127; YES), the image recognition device 2 holds the character string 1351 "** company" as a correction data character string (step S128), and the first step is made. End the loop processing. In this case, the image recognition device 2 combines the character string 1336 "woman" and the character string 1351 "** company" into one character string (step S131), and the combined character string "woman". The character recognition of "** company" is performed (step S132).

After recognizing the characters of "female ** company", the image recognition device 2 determines whether or not the recognition result is within the number of digits set in the recognition item table 296. In the recognition item table 296 of FIG. 7, the number of digits for the gender is 1. Therefore, the image recognition device 2 determines that the result of character recognition is larger than the number of digits set in the recognition item table 296 (step S133; NO), and the set number of digits in the recognition result + 1 character string or later. (That is, the character string after the second digit) is deleted. As a result, the result of character recognition of the data character string for the heading character string 1335 "gender" is the single digit "woman" character (character string 1336). Therefore, the image recognition device 2 of the present embodiment has a heading character from an image of an identity verification document in which other information not corresponding to the heading character string is printed near the information available for identity verification corresponding to the heading character string. Only the information available for identity verification corresponding to the column can be properly recognized.

Further, the image recognition device 2 determines whether or not the final character set in the recognition item table 296 is included in the character recognition result.

For example, FIG. 21 (c) shows the partial region 11E including the line of date of birth in the image of the health insurance card 10. Of the character strings extracted from the subregion 11E, the character string 1330 "date of birth" matches the item keyword in the recognition item keyword table 295, and thus becomes a heading character string. Therefore, the image recognition device 2 performs a right-hand search process using the character string 1330 "date of birth" as a heading character string. In the right-hand search process using the character string 1330 "date of birth" as the heading character string, the image recognition device 2 uses, for example, the character string 1339 "November 11, 1980" and "XX Co., Ltd." The character string 1352 is used as a data candidate character string. The two data candidate character strings do not match the item keywords in the recognition item keyword table 295. Therefore, the image recognition device 2 uses the character string 1339 "November 11, 1980" and the character string 1352 "XX Co., Ltd." as the data character string for the character string 1330 "date of birth". , Ends the right search process. After completing the right-side search process, the image recognition device 2 omits the downward search process, and then performs the data character string correction process.

In the data character string correction process, the image recognition device 2 uses the character string 1339 "November 11, 1980" as the first character string and the character string 1352 "XX Co., Ltd." as the next character string. The distance D4 between the two character strings is calculated. Here, since the heading character string 1330 is a character string "date of birth", the image recognition device 2 acquires, for example, the distance threshold value (10 mm) between the character strings from the recognition item table 296 of FIG. That is, when the calculated distance D4 is within 10 mm (step S127; YES), the image recognition device 2 holds the character string 1352 "XX Co., Ltd." as a correction data character string (step S128), and the first Ends the loop processing of. In this case, the image recognition device 2 combines the character string 1339 "November 11, 1980" and the character string 1352 "XX Co., Ltd." into one character string (step S131), and the combination is performed. The character recognition of one character string "November 11, 1980 XX Co., Ltd." is performed (step S132).

After recognizing the characters of "November 11, 1980, XX Co., Ltd.", the image recognition device 2 determines whether or not the recognition result is within the number of digits set in the recognition item table 296. In the recognition item table 296 of FIG. 7, the number of digits of the year is 2 and the number of digits of the date is 8 among the dates of birth. Therefore, the image recognition device 2 determines that the character recognition result is larger than the number of digits (10 digits) set in the recognition item table 296 (step S133; NO), and the set number of digits (10) in the recognition result. ) +1 Delete the character string after the digit (that is, the character string after the 11th digit). In this case, for example, if each of the numbers indicating the year, month, and day is one digit, such as January 1, 1990, a part of "XX Co., Ltd." will remain without being deleted. There is.

Therefore, in the image recognition device 2 of the present embodiment, it is next determined whether or not the final character set in the recognition item table 296 is included in the character recognition result. In the recognition item table 296 of FIG. 7, the last character of the date of birth is "day". Therefore, the image recognition device 2 searches for the character "day" in a character string such as "November 11, 1980 stock", and the character string 1339 "November 11, 1980". The character 14 of "day" contained in is detected. In this case, the image recognition device 2 deletes the character string after the character 14 of "day" in the character string of "November 11, 1980 stock". As a result, the information (XX Co., Ltd.) that is not related to the date included in the result of character recognition of the date of birth is deleted, and only the information that can be used for identity verification corresponding to the heading character string is appropriately recognized. be able to.

As described above, the image recognition device 2 of the present embodiment is based on the positional relationship between the item name described in the identity verification document and the character string indicating the information that can be used for identity verification corresponding to the item name. The position of the character string corresponding to the item name can be specified. Therefore, according to the image recognition device 2 of the present embodiment, even if the position of the character string for character recognition among the character strings included in the image of the identity verification document is not predetermined, it can be used for identity verification. Can properly recognize various character strings. Therefore, in the image recognition device 2 of the present embodiment, for example, a character string that can be used for identity verification can be appropriately recognized from an image of an identity verification document such as a health insurance card whose format is not unified.

In the document identification table 291 of FIG. 3 and the document type table 292 of FIG. 4, the type of identity verification document referred to by the document type identification unit 202, the identification keyword (character string), the position of the identification keyword in the image, and It is just an example of information that shows the correspondence between character recognition methods. Further, the recognition item keyword table 295 of FIG. 6 is only an example of information indicating an item name (item keyword) included in the identity verification document in the image referred to by the keyword recognition unit 212 and the non-ruled line keyword recognition unit 213. Further, the recognition item table 296 of FIG. 7 and the recognition category table 297 of FIG. 8 are merely examples of information that is referred to by the recognition result correction unit 205 and indicates a portion that can be deleted from the recognition result without any problem.

Further, a part or all of various information such as the document identification table 291 stored in the storage unit 290 in the image recognition device 2 of FIG. 2 is stored in a device different from the image recognition device 2 (for example, a server device 6 or the like). You may memorize it.

Further, in the present embodiment, a right-hand search process for searching the right side of the character string (heading character string) matching the item keyword in the recognition item keyword table 295 and a downward search process for searching the lower part of the heading character string are performed. , The character string that performs character recognition corresponding to the heading character string was specified. However, the search direction is not limited to the right side and the lower side, and is appropriate depending on the positional relationship between the item name (heading character string) in the identity verification document and the information available for identity verification corresponding to the item name. It can be changed.

Further, the image recognition device 2 according to the present embodiment can be realized by a computer and a program executed by the computer. Hereinafter, the image recognition device 2 realized by the computer and the program will be described with reference to FIG. 22.

FIG. 22 is a diagram showing a hardware configuration of a computer.
As shown in FIG. 22, the computer 20 includes a CPU (Central Processing Unit) 2001, a memory 2002, an auxiliary storage device 2003, an input / output interface 2004, a display control device 2005, a communication control device 2006, and a medium drive. The device 2007 is provided. These elements 2001 to 2007 in the computer 20 are connected to each other by a bus 2010, and data can be exchanged between the elements.

The CPU 2001 is a processor that controls the overall operation of the computer 20 by executing various programs including an operating system. For example, the CPU 2001 executes an image recognition program including the processes of steps S1 to S6 of FIG. The application process of step S2 in the image recognition program is made into a subroutine as a process including the processes of steps S11 to S22 of FIG. 10, for example. The identity verification document identification process of step S15 in the application process of FIG. 10 is made into a subroutine as a process including, for example, the processes of steps S31 to S52 of FIGS. 11A and 11B. Further, the character recognition process by the ruled line keyword recognition method in step S21 in the application process of FIG. 10 is made into a subroutine as a process including the processes of steps S61 to S70 of FIG. 12, for example. Further, the right side search process of step S64 in the character recognition process of FIG. 12 is made into a subroutine as a process including, for example, the processes of steps S81 to S95 of FIG. Similarly, the downward search process of step S66 in the character recognition process of FIG. 12 is made into a subroutine as a process including, for example, the processes of steps S101 to S115 of FIG. Further, the data character string correction process of step S67 in the character recognition process of FIG. 12 is made into a subroutine as a process including, for example, the processes of steps S121 to S138 of FIGS. 15A and 15B.

The memory 2002 includes a ROM (Read Only Memory) and a RAM (Random Access Memory). In the ROM of the memory 2002, for example, a predetermined basic control program or the like read by the CPU 2001 when the computer 20 is started is recorded in advance. Further, the RAM of the memory 2002 is used by the CPU 2001 as a work storage area as needed when executing various programs. The RAM of the memory 2002 can be used, for example, to store an image of an identity verification document, a character string extracted from a binarized image of an identity verification document, a record read from a document identification table 291 or the like, and the like.

The auxiliary storage device 2003 is, for example, a magnetic disk such as an HDD (Hard Disk Drive) and a non-volatile memory disk such as a flash memory disk. The auxiliary storage device 2003 can store various programs, various data, and the like executed by the CPU 2001. The auxiliary storage device 2003 can be used, for example, for storage of the above-mentioned image recognition program and the like. Further, the auxiliary storage device 2003 includes, for example, a document identification table 291, a document type table 292, a recognition method table 293, a fixed position recognition table 294, a recognition item keyword table 295, a recognition item table 296, a recognition category table 297, and a print character. It can be used to store the recognition dictionary 298. Further, the auxiliary storage device 2003 can also be used to store an image of the identity verification document, a character string extracted from the binarized image of the identity verification document, a character recognition result, and the like.

The input / output force interface 2004 is a hardware interface that receives information input from the scanner device 3 and other input devices 4 and outputs information to an output device (not shown). The input device 4 is, for example, a keyboard device, a mouse device, a touch panel device, and the like. The output device (not shown) is, for example, a printer device.

The display control device 2005 controls the display screen of the display device 5 such as a liquid crystal display device. The display control device 2005 generates, for example, a screen including the operating state of the computer 20, the image of the identity verification document stored in the auxiliary storage device 2003, the result of character recognition, and the like, and displays the screen on the display device 5.

The communication control device 2006 is a device that connects a computer 20 to a communication network 7 and controls various communications between the computer 20 and another electronic device (for example, a server device 6) via the communication network.

The medium drive device 2007 reads out the programs and data recorded in the portable recording medium 21, and writes the data stored in the auxiliary storage device 2003 to the portable recording medium 21. As the portable recording medium 21, for example, there is a memory card (flash memory) of the Secure Digital (SD) standard. The portable recording medium 21 can be used for recording each of the above programs. Further, the portable recording medium 21 can be used for storing, for example, the above-mentioned image recognition program. Further, the portable recording medium 21 is, for example, a document identification table 291, a document type table 292, a recognition method table 293, a fixed position recognition table 294, a recognition item keyword table 295, a recognition item table 296, a recognition category table 297, and a print. It can be used to store the character recognition dictionary 298. Further, the portable recording medium 21 can also be used for storing an image of an identity verification document, a character string extracted from a binarized image of an identity verification document, a result of character recognition, and the like.

Further, when the computer 20 is equipped with an optical disc drive that can be used as the medium drive device 2007, various optical discs that can be recognized by the optical disc drive can be used as the portable recording medium 21. Optical discs that can be used as the portable recording medium 21 include, for example, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs (Blu-ray is a registered trademark), and the like.

When the computer 20 is started, the CPU 2001 reads out an image recognition program including each of the above processes from the auxiliary storage device 2003 or the like and executes the computer 20. While executing the image recognition program, the CPU 2001 uses the document type identification unit 202, the recognition method selection unit 203, the item position identification unit 204, the recognition result correction unit 205, the character recognition unit 206, and the recognition result of the image recognition device 2. It functions (operates) as the notification unit 207 and the main control unit 209. Further, while the above image recognition program is being executed, the non-temporary recording medium such as the RAM in the memory 2002 of the computer 20, the auxiliary storage device 2003, and the portable recording medium 21 is stored in the storage unit of the image recognition device 2. Functions as 290.

The hardware configuration of the computer 20 operated as the image recognition device 2 is not limited to the configuration shown in FIG. 22, and some elements may be omitted or added depending on the application and conditions. Is. For example, the computer 20 may be the one in which the medium driving device 2007 is omitted.

1 Image recognition system 2 Image recognition device 201 Scanner control unit 202 Document type identification unit 203 Recognition method selection unit 204 Item position identification unit 205 Recognition result correction unit 206 Character recognition unit 207 Recognition result notification unit 208 Communication unit 209 Main control unit 211 Position Fixed recognition unit 212 Keyword recognition unit 213 Non-ruled line Keyword recognition unit 290 Storage unit 291 Document identification table 292 Document type table 293 Recognition method table 294 Position fixed recognition table 295 Recognition item keyword table 296 Recognition item table 297 Recognition category table 298 Print character recognition Dictionary 3 Scanner device 4 Input device 5 Display device 6 Server device 7 Communication network 10 Insurance certificate 1320, 1330, 1335, 1340 Character string (heading character string)
1329, 1336, 1339, 1341, 1350, 1351, 1352 string 20 computer 2001 CPU
2002 Memory 2003 Auxiliary storage device 2004 Input interface 2005 Display control device 2006 Communication control device 2007 Media drive device 2010 Bus 21 Portable recording medium

Claims (8)

A character string is extracted from the image of the identity verification document, and the position and characters in the image of the extracted character string and an item keyword indicating an item included in the identity verification document predetermined for each type of the identity verification document. A document type identification unit that identifies the type of the identity verification document in the image based on the degree of matching of
Based on the identified type of the identity verification document and the information of the name of the item for character recognition predetermined for each type of the identity verification document, the character recognition is performed from the character string extracted from the image. The heading character string that matches the name of the item is specified, and the character in the image is based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string. Item that specifies the position to be recognized The position identification part and
A character recognition unit that recognizes characters at the specified position in the image, and
A recognition result correction unit that corrects the result of the character recognition based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document .
Based on the type of the identified identity verification document, the first recognition method for performing the character recognition based on the information for designating the position for performing the character recognition in the predetermined image, and the predetermined heading character. The character recognition is performed based on the second recognition method of performing the character recognition based on the columns and the layout of the ruled lines in the image, and the positional relationship between the heading character string and the information available for identity verification. It is provided with a recognition method selection unit for selecting which of the third recognition methods is used for character recognition .
The item position specifying unit has a positional relationship between the heading character string in the identity verification document and information that can be used for identity verification corresponding to the heading character string when the character recognition is performed by the third recognition method. An image recognition device characterized by specifying a position in the image for performing the character recognition based on the above . A character string is extracted from the image of the identity verification document, and the position and characters in the image of the extracted character string and an item keyword indicating an item included in the identity verification document predetermined for each type of the identity verification document. A document type identification unit that identifies the type of the identity verification document in the image based on the degree of matching of
Based on the identified type of the identity verification document and the information of the name of the item for character recognition predetermined for each type of the identity verification document, the character recognition is performed from the character string extracted from the image. The heading character string that matches the name of the item is specified, and the character in the image is based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string. Item that specifies the position to be recognized The position identification part and
A character recognition unit that recognizes characters at the specified position in the image, and
A recognition result correction unit that corrects the result of the character recognition based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document.
Equipped with
The item position specifying unit confirms the identity of the character string whose position in the image is below the heading character string and whose width coordinate range is included in the width coordinate range of the heading character string. An image recognition device characterized in that a position for performing the character recognition is specified as a character string containing information that can be used in the above. A character string is extracted from the image of the identity verification document, and the position and characters in the image of the extracted character string and an item keyword indicating an item included in the identity verification document predetermined for each type of the identity verification document. A document type identification unit that identifies the type of the identity verification document in the image based on the degree of matching of
Based on the identified type of the identity verification document and the information of the name of the item for character recognition predetermined for each type of the identity verification document, the character recognition is performed from the character string extracted from the image. The heading character string that matches the name of the item is specified, and the character in the image is based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string. Item that specifies the position to be recognized The position identification part and
A character recognition unit that recognizes characters at the specified position in the image, and
A recognition result correction unit that corrects the result of the character recognition based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document.
Equipped with
The item position specifying unit is the image when there is no character string whose position in the image is to the right of the heading character string and whose height coordinate range is included in the height coordinate range of the heading character string. An image recognition device characterized by searching for a character string whose width coordinate range is included in the width coordinate range of the heading character string among the character strings whose position in is below the heading character string. A character string is extracted from the image of the identity verification document, and the position and characters in the image of the extracted character string and an item keyword indicating an item included in the identity verification document predetermined for each type of the identity verification document. A document type identification unit that identifies the type of the identity verification document in the image based on the degree of matching of
Based on the identified type of the identity verification document and the information of the name of the item for character recognition predetermined for each type of the identity verification document, the character recognition is performed from the character string extracted from the image. The heading character string that matches the name of the item is specified, and the character in the image is based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string. Item that specifies the position to be recognized The position identification part and
A character recognition unit that recognizes characters at the specified position in the image, and
A recognition result correction unit that corrects the result of the character recognition based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document.
Equipped with
The recognition result correction unit is characterized in that when the distance between two character strings existing in a predetermined direction with respect to the heading character string is within a predetermined distance threshold, the two character strings are combined . Image recognition device. In the recognition result correction unit, the number of characters included in the character recognition result of the character string including the information available for identity verification corresponding to the heading character string is larger than the number of characters associated with the heading character string. In this case, among claims 1 to 3, the part of the characters included in the character recognition result that exceeds the number of characters associated with the heading character string, counting from the left end, is deleted. The image recognition device according to any one . The recognition result correction unit is in the information that can be used for the identity verification corresponding to the heading character string in the character recognition result of the character string including the information that can be used for the identity verification corresponding to the heading character string. The present invention according to any one of claims 1 to 3 , wherein when the final character is included, the character to the right of the final character among the characters included in the character recognition result is deleted. Image recognition device. The computer
Extract the character string from the image of the identity verification document and
The said in the image based on the position in the image and the degree of matching of the characters with the extracted character string and the item keyword indicating the item included in the identity verification document predetermined for each type of the identity verification document. Identify the type of identity verification document and
Based on the identified type of the identity verification document and the information of the name of the item for character recognition predetermined for each type of the identity verification document, the character recognition is performed from the character string extracted from the image. Identify the heading string that matches the item name and
A position for performing the character recognition in the image is specified based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string.
Character recognition of the specified position in the image is performed, and
The result of the character recognition is corrected based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document .
Based on the type of the identified identity verification document, the first recognition method for performing the character recognition based on the information for designating the position for performing the character recognition in the predetermined image, and the predetermined heading character. The character recognition is performed based on the second recognition method of performing the character recognition based on the columns and the layout of the ruled lines in the image, and the positional relationship between the heading character string and the information available for identity verification. Select which of the third recognition methods to perform the character recognition, and select
When the character recognition is performed by the third recognition method, the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string are in the image based on the positional relationship. An image recognition method characterized by executing a process of specifying a position where character recognition is performed . Extract the character string from the image of the identity verification document and
The said in the image based on the position in the image and the degree of matching of the characters with the extracted character string and the item keyword indicating the item included in the identity verification document predetermined for each type of the identity verification document. Identify the type of identity verification document and
Based on the identified type of the identity verification document and the information of the name of the item for character recognition predetermined for each type of the identity verification document, the character recognition is performed from the character string extracted from the image. Identify the heading string that matches the item name and
A position for performing the character recognition in the image is specified based on the positional relationship between the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string.
Character recognition of the specified position in the image is performed, and
The result of the character recognition is corrected based on the characteristics of the character arrangement of the character string including the information that can be used for the identity verification in the identity verification document .
Based on the type of the identified identity verification document, the first recognition method for performing the character recognition based on the information for designating the position for performing the character recognition in the predetermined image, and the predetermined heading character. The character recognition is performed based on the second recognition method of performing the character recognition based on the columns and the layout of the ruled lines in the image, and the positional relationship between the heading character string and the information available for identity verification. Select which of the third recognition methods to perform the character recognition, and select
When the character recognition is performed by the third recognition method, the heading character string in the identity verification document and the information available for identity verification corresponding to the heading character string are in the image based on the positional relationship. An image recognition program characterized by causing a computer to execute a process of specifying a position where character recognition is performed .

Images