JP5669041B2 - Document processing apparatus and document processing method - Google Patents

Description

  The present invention relates to a character recognition technique, and more particularly to a technique for recognizing a handwritten character using a result of recognizing a printed character.

  There has been disclosed a character recognition technique for optically reading handwritten characters and printed characters and converting them into character data for use.

  For example, Patent Document 1 discloses a technique for accurately and efficiently reading a character string including difficult characters or unreadable characters.

  Patent Document 2 discloses a technique for extracting a character string registered in a dictionary from a text including a notation fluctuation caused by a typo or transposition of a word.

JP 2009-265758 A JP 2007-25834 A

  The character recognition technology has various uses, and an example thereof is a window service in a government office or a company. For example, by automatically converting various application documents handwritten by the applicant into character data, the window service can be made more efficient. However, it has been a problem of the conventional character recognition technology that the recognition accuracy of handwritten characters is significantly lower than the recognition accuracy of type characters. In general, various kinds of documents are processed at the counter, but when the format is different for each type of document, the position where the handwritten character to be read is also different for each type. Sometimes it fails.

  Furthermore, when accepting such application documents, the identity of the applicant may be required. For example, the application documents handwritten by the applicant and the identity verification document (for example, a car driver's license) brought by the applicant are collated, and the application documents are accepted when it is determined that the contents of both match. . If such collation work can be automated using character recognition technology, the window service can be greatly improved, and unauthorized processing by the window operator can be prevented.

  However, when automating such identity verification, in addition to the above-mentioned problem of recognition accuracy of handwritten characters, there is a problem that identity verification fails due to the difference in character type. The printed characters described in the identity verification document are limited to those that can be used in the system that created the document. For this reason, the typeface of the printed character described in the identification document may be different from the type of the character handwritten by the applicant. Such a difference typically occurs when Kanji variants are used, but may also occur in other cases. For example, when the identity verification document is a passport and the application document requires entry of kanji, the kanji handwritten character and the roman character are collated. In these cases, even if the character recognition itself is successful, the identity verification will fail.

  The present invention has been made in view of the above-described problems, and it is an object of the present invention to improve the accuracy of character recognition and prevent failure of identity verification due to a difference in character type.

A typical example of the present invention is as follows. That is, a document processing apparatus comprising an input device, a processor connected to the input device, a storage device connected to the processor, and an output device connected to the processor, wherein the storage device is print character recognition dictionary including information indicating the characteristic quantity of printed characters, handwritten character recognition dictionary including information indicating a characteristic of the handwritten characters, as well as a plurality of characters, the connection between the plurality of characters, and of the connection A character string pattern dictionary including a score indicating the likelihood of occurrence is held, and the processor acquires image data including a handwritten character string and a character string input via the input device, and the character character recognition dictionary the print string recognizes, connection already said string pattern prefix between characters included in the obtained character string and the character string as a recognition result of the printed character string using The character string obtained as a recognition result of the printed character string and the connection between the characters included in the character string are increased by increasing the value of the score relating to the connection between the registered characters. If it is not registered in the character string pattern dictionary, the character string newly included in the character string acquired as a recognition result of the printed character string, a connection between those characters and a score indicating the likelihood of the connection are newly added to the character string Registering in a pattern dictionary, recognizing the handwritten character string using the handwritten character recognition dictionary and the character string pattern dictionary, and outputting the recognition result of the handwritten character string via the output device .

  According to one embodiment of the present invention, the recognition accuracy of handwritten characters can be improved.

It is a block diagram which shows the structure of the window reception system of embodiment of this invention. It is explanatory drawing of the image read by the window reception system of embodiment of this invention. It is a flowchart of the character recognition process for the window work efficiency improvement by the conventional OCR apparatus. It is a flowchart of the character recognition process which the window reception system of embodiment of this invention performs. It is explanatory drawing of the item name dictionary of embodiment of this invention. It is explanatory drawing of the data character string pattern dictionary of embodiment of this invention. It is a flowchart of the dynamic addition process of the data character string pattern dictionary which the window reception system of embodiment of this invention performs. It is a flowchart of the handwritten character recognition process in the case of not holding the data character string pattern dictionary previously performed by the window reception system of the embodiment of the present invention. It is a flowchart which shows another example of the handwritten character recognition process in the case of not hold | maintaining the data character string pattern dictionary previously performed by the window reception system of embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of a window reception system according to an embodiment of this invention.

  The window reception system 10 of the present embodiment is a computer that performs document processing using character recognition in order to support a window service, and includes, for example, an input device 11, a display device 12, a CPU (Central Processing Unit) 13, A printing apparatus 14, a work area 15, and information holding means 16 are provided.

  The input device 11 includes an input device 110 and an image input device 111. The input device 110 is a device that accepts an input such as an instruction from a user, and may be a keyboard, a mouse, a touch panel, or the like, for example. The image input device 111 is a so-called image scanner that optically reads a document on which handwritten characters and printed characters are written and converts them into image data.

  The display device 12 is a device that outputs information such as various characters and images to the user, and may be an image display device such as a liquid crystal display.

  The CPU 13 is a processing device that implements various functions by executing a program stored in the work area 15. The processing of each program described below is actually executed by the CPU 13.

  The printing device 14 prints various information provided to the user as necessary.

  The work area 15 is a storage area for storing programs executed by the CPU 13. The work area 15 of this embodiment stores an OS (Operating System) 151, a communication program 152, a document processing program 153, and a character recognition program 154. Still other programs may be stored in the work area 15, data referred to when the CPU 13 executes these programs may be stored, and results of processing executed by the CPU 13 may be stored. Also good. Processing based on these programs will be described later.

  The information holding means 16 stores information that the CPU 13 refers to in order to execute various processes based on each program. The information holding means 16 of the present embodiment stores a printed character recognition dictionary 161, a handwritten character recognition dictionary 162, an external character / Roman character dictionary 163, an arrangement knowledge dictionary 164, an item name dictionary 165, and a data character string pattern dictionary 166. The

  The printed character recognition dictionary 161 includes information indicating the characteristic amount of printed characters, and is referred to when the window reception system 10 performs printed character recognition. The handwritten character recognition dictionary 162 includes information indicating the characteristic amount of the handwritten character, and is referred to when the counter reception system handwritten character recognition is executed. These dictionaries may be similar to those used in conventional character recognition.

  The external character / Romaji dictionary 163 stores information indicating a range of characters determined to be the same by the window reception system 10. For example, when a variant of a specific kanji is used in place of that kanji and the window reception system 10 is allowed to determine that they are the same character, the information for associating them is an external character / roman character It is stored in the dictionary 163. Furthermore, information that associates a Chinese character with a Roman character notation corresponding to the Chinese character may be stored in the external character / Roman character dictionary 163. An example of the external character / Romaji dictionary 163 and processing using the dictionary will be described later (see FIG. 9).

  The arrangement knowledge dictionary 164, the item name dictionary 165, and the data character string pattern dictionary 166 will be described later (see FIG. 4). As will be described later, the window reception system 10 may not hold the data character string pattern dictionary 166 in advance.

  Other information may be stored in the information holding unit 16, but the description is omitted in this embodiment.

  The work area 15 and the information holding unit 16 correspond to a storage device installed in the window reception system 10. Typically, the work area 15 is a high-speed and volatile storage device such as a DRAM (Dynamic Random Access Memory), and the information holding means 16 is a large-capacity and nonvolatile memory such as an HDD (Hard Disk Drive) or a flash memory. However, other types of storage devices may be used. Each program may be stored in the information holding unit 16 and copied to the work area 15 when executed by the CPU 13. Further, at least a part of the dictionary stored in the information holding unit 16 may be temporarily copied to the work area 15 as necessary.

  The window reception system 10 may be further connected to one or more file servers 21 via the communication network 19. Each file server 21 is a computer connected to the communication network 19, and its hardware configuration may be the same as that of the window reception system 10.

  Although FIG. 1 shows an example in which the window reception system 10 is realized by one computer, the same function as the window reception system 10 may be realized by a plurality of computers. For example, various dictionaries such as the printed character recognition dictionary 161 may be stored in any one of the file servers 21 instead of being stored in the information holding unit 16. Alternatively, for example, the image input device 111 may be provided in one or a plurality of file servers 21, and the image input device 111 may read an application form and an identification document described later. In that case, the image data read by the file server 21 is transmitted to the window reception system 10 via the communication network 19. In the above case, the input device 11 may include a communication device connected to the communication network 19. Communication via the communication network 19 is controlled by the communication program 152.

  FIG. 2 is an explanatory diagram of an image read by the window reception system 10 according to the embodiment of this invention.

  The window reception system 10 optically reads an application form and an identification document necessary for receiving the application form using the image input device 111, and uses the image data obtained thereby as a work area 15 or information holding means 16. To store. Such reading may be performed by any procedure, but typically, the documents are read so as to be included in one image. FIG. 2 shows, as an example, an image 200 when an application form 211 for obtaining a resident card and a driver's license 212 (that is, an identity confirmation document) are read.

  The application form 211 is handwritten with the address, name, date of birth, etc. of the applicant. Hereinafter, although the recognition of the character of the name handwritten in the name field 201 (“Name field 1” in FIG. 2) of the application 211 will be described as an example, the present embodiment also applies to the handwritten characters described in other fields. The present invention can be applied to any document other than the application as long as it includes a handwritten character. In the example of FIG. 2, the handwritten character string “Taro Hizaki” is described as the name.

  In the driver's license 212, the address, name, date of birth, etc. of the person who has received the license are written in print. In the following description, the type characters of the name described in the name field 202 (“name field 2” in FIG. 2) are used. The driver's license is an example of an identity verification document, and other documents (for example, a passport or a health insurance card) may be used as the identity verification document. In the example of FIG. 2, the type character string “Taro Hisaki” is described as the name.

  FIG. 3 is a flowchart of a character recognition process for improving the efficiency of window work by a conventional OCR (optical character reading) apparatus.

  For example, when the image of the application form 211 of FIG. 2 is input (step 301), the conventional OCR device (not shown) determines the position of the name field 201 in the input image (step 302), and the name field 201 Are recognized (step 303).

  On the other hand, for example, when the image of the driver's license 212 of FIG. 2 is input (step 311), the conventional OCR device determines the position of the name field 202 in the input image (step 312), and the name field 202 is displayed. The printed character is recognized (step 313). The type character recognition processing (steps 301 to 303) and the handwritten character recognition processing (steps 311 to 312) may be executed by one OCR device, or may be executed by different OCR devices.

  Thereafter, the conventional OCR device collates the recognition result in step 303 with the recognition result in step 313 (step 304), and determines whether or not they match (step 305). If the two match, the identity verification has succeeded and the application is accepted and application processing based on the application is executed. Since this application process is the same as the conventional process, a description thereof will be omitted. On the other hand, if the two do not match, identity verification by character recognition has failed, and identity verification by another method (for example, identity verification by the hand of the window operator, etc.) is executed.

  The determination of the name column in step 312 is performed as follows, for example. The OCR device holds an arrangement knowledge dictionary, an item name dictionary, and a data character string pattern dictionary in advance. These may be the same as the arrangement knowledge dictionary 164, the item name dictionary 165, and the data character string pattern dictionary 166 of the embodiment of the present invention. The OCR device determines the likelihood of the item name of the character string on the identity verification document based on the item name dictionary (for example, a character string indicating an item name such as “name”), and based on the data character string pattern dictionary 166 Determine the likelihood of the item value of the character string on the confirmation document (for example, a character string indicating a name such as “Tanaka”), and the character string determined to be the item name and the character string determined to be the item value When the positional relationship matches the positional relationship registered in the arrangement knowledge dictionary, the area is extracted as the name field 202.

  However, according to this method, extraction of the name field 202 may fail due to the input of a name that is not registered in advance in the data character string pattern dictionary. Even if both of the name columns 202 and 201 are successfully extracted, if the written name is not registered in the data character string pattern dictionary, there is a high possibility that subsequent character recognition will fail.

  In addition, although the accuracy of recognition of type characters by the current character recognition technology is high enough for practical use, the accuracy of recognition of handwritten characters is significantly lower than that. For this reason, the application and identity verification document that should have been correct are input (that is, the same name is written in the name field 201 and the name field 202), and the written name is registered in the dictionary. However, the frequency at which it is determined in step 305 that the two do not match increases.

  As described above, if the OCR device frequently fails to verify the identity, the operator's workload is not sufficiently reduced, and therefore the window service cannot be made sufficiently efficient.

  FIG. 4 is a flowchart of character recognition processing executed by the window reception system 10 according to the embodiment of this invention.

  First, an application form and an identification document are input using the image input device 111 of the window reception system 10 (step 401). For example, the application form 211 and the driver's license 212 (that is, the identity confirmation document) are read by the image input device 111, and the image 200 shown in FIG.

  Next, the window reception system 10 extracts the item area of the personal identification document (step 402). The item area is an area in which the value of an item such as an address, name, or date of birth is described. Here, in order to explain the recognition of the name as an example, at least the name column 202 is extracted as an item area. This extraction is performed based on the arrangement knowledge dictionary 164 and the item name dictionary 165 held in the information holding unit 16.

  The item name dictionary 165 stores item names displayed corresponding to the respective item areas. For example, in the vicinity of the name field 202 of the driver's license 212, the characters “name” are displayed as the item name. This display is used as a sign (label) for specifying the position of the name field 202. In this case, the character string “name” is stored in the item name dictionary 165.

  Documents other than the driver's license may be used as the identity verification document, and in those documents, a character string other than “name”, for example, “Name” may be displayed as the item name in the name column. As will be described later, the item name dictionary 165 is also used to specify the position of an item area (for example, the name field 201 of the application 211) in the application. The character string displayed as the item name on the application form may differ from that of the identity verification document. For example, a character string of “name” may be printed on the application form, and an area for handwriting the name of the applicant may be secured in the vicinity thereof. Further, the window reception system 10 may process a plurality of types of application forms, and the character string displayed as the item name may differ depending on the type of application form. For this reason, the item name dictionary 165 may further store character strings other than “name” such as “name” and “Name” as item names in the name field.

  For item areas other than the name field 202, one or more character strings of the item names are stored for each item area. An example of the item name dictionary 165 will be described later with reference to FIG.

  The arrangement knowledge dictionary 164 stores information indicating the arrangement of each item area, more specifically, information for weighting the positional relationship between the display of each item name and the item area corresponding to each item name. For example, a predetermined range on the right side of the item name “name” displayed on the driver's license 212 is the name field 202. In this case, the arrangement knowledge dictionary 164 stores information indicating the positional relationship between the display of the character string “name” and the name column 202. As in the case of the item name dictionary 165 described above, the arrangement knowledge dictionary 164 stores information indicating the positional relationship between the item name display and the item area in various identity verification documents and various application forms.

  The window reception system 10 extracts item areas such as the name field 202 based on these dictionaries. For example, the window reception system 10 determines the likelihood of the item name of the character string on the identification document based on the item name dictionary 165 and registers the positional relationship with the character string determined to be the item name in the arrangement knowledge dictionary 164. An area matching the determined positional relationship is extracted as an item area.

  Next, the window reception system 10 recognizes the printed characters such as the address, name and date of birth described in the item area extracted in Step 402 (Step 403). Since this procedure is the same as the conventional character recognition, a detailed description is omitted.

  Next, the window reception system 10 creates a candidate character string pattern dictionary for handwritten character recognition using the recognition result of the printed character in step 403 (step 404). When the window reception system 10 does not hold the data character string pattern dictionary 166 in advance, it uses the created candidate character string pattern dictionary as the data character string pattern dictionary 166 and holds the data character string pattern dictionary 166 in advance. If so, the data character string pattern dictionary 166 is updated using the created candidate character string pattern dictionary. These processes will be described later (see FIGS. 7 and 8).

  Next, the window reception system 10 extracts item areas in which handwritten characters are described in the application form (step 405). The item area extracted here is an area in which an address, name, date of birth, or the like in the application is described, and handwritten characters by the applicant are described. This extraction is performed with reference to the arrangement knowledge dictionary 164 and the item name dictionary 165, similarly to the extraction in step 402.

  Next, the window reception system 10 recognizes handwritten characters such as an address, name and date of birth described in the item area extracted in step 405 (step 406). Since this procedure is the same as the conventional character recognition except that the data character string pattern dictionary 166 updated or created in step 404 is referred to, detailed description thereof will be omitted. That is, the dictionary reference method and the character recognition method based thereon are the same as in the prior art.

  Next, the window reception system 10 outputs the recognized result (step 407) and ends the process. For example, the window reception system 10 may display the character string itself obtained by the character recognition (step 403) in the name field 201 on the display device 12, or the character recognition (step 403) in the name field 202. It may be determined whether or not the character string obtained in step 406) matches, and the result may be displayed on the display device 12. Specifically, for example, when it is determined that the two do not match, an alert indicating that the identity verification has failed may be displayed on the display device 12.

  More specifically, the above processing is executed by the CPU 13 of the window reception system 10 according to a program in the work area 15. For example, steps 402, 404 and 405 are executed according to the document processing program 153, and steps 403 and 406 are executed according to the character recognition program.

  FIG. 5 is an explanatory diagram of the item name dictionary 165 according to the embodiment of this invention.

  The item name dictionary 165 includes information that defines character strings of item names of the personal identification document and the application form. For example, the item name dictionary 165 illustrated in FIG. 5 includes “Surname” 501, “Name” 502, “Address” 503, “Name” 504, and “Address” 505. “Surname” 501 is an example of a label of an item area in which a name (last name) is described when the identification document is a passport. “Name” 502 is an example of an item area label in which the name is written when the identification document is a driver's license. “Address” 503 is an example of a label of an item area in which an address is described when the identification document is a driver's license. “Name” 504 is an example of the label of the item area in which the name is handwritten in the application form. “Address” 505 is an example of an item area label in which an address is handwritten in the application form.

  FIG. 6 is an explanatory diagram of the data character string pattern dictionary 166 according to the embodiment of this invention.

  The data character string pattern dictionary 166 may be stored in the information holding unit 16 and part or all of the data character string pattern dictionary 166 may be loaded into the work area 15 as necessary.

  In the data character string pattern dictionary 166, character strings that may be included in the handwritten character string are registered. For example, in the data character string pattern dictionary 166 used for recognition of the character string of the full name, a character string that may be used as the full name, for example, “Tanaka” as a character string <NAME1> used as the surname, “Ichiro”, “Taro”, “Hanako”, “Yoshiko” and the like are registered as character strings <NAME2> used as names such as “Suzuki”, “Sato”, and “Yamada”. Furthermore, a character string <N400> consisting of the above-mentioned surname character string <NAME1> and the subsequent name character string <NAME2> is registered.

  Similarly, for example, character strings of place names nationwide are registered in the data character string pattern dictionary 166 used for recognition of address character strings.

  As will be described later, the data character string pattern dictionary 166 is loaded into the work area 15 as necessary. In the present embodiment, the loaded data character string pattern dictionary 166 is expressed in a graph format in order to avoid holding the same character repeatedly. That is, each character registered in the data character string pattern dictionary 166 is represented as a node, and the connection between the characters is represented as an edge. For example, when the character string “Tanaka” of the surname is registered in the data character string pattern dictionary 166, the character string includes a node corresponding to the character “Ta”, a node corresponding to the character “Middle”, and connecting them. Expressed by the edge to be.

  Although omitted in FIG. 6, the data character string pattern dictionary 166 includes information indicating the possibility of connection between characters. This information is, for example, a score given to the edge, and is extracted in advance from text that is actually distributed, such as a newspaper document.

  The score is a weight indicating the likelihood of connection of characters corresponding to each edge (in other words, likelihood). In the present embodiment, the larger the score value, the easier connection of the character corresponding to the edge to which the score is given, and the score value “0” means that the character corresponding to the edge Indicates that no connection can occur.

  For example, as registered in the data character string pattern dictionary 166, character strings such as the character strings “Tanaka” and “Suzuki” can appear, but the character string “Tauzu” cannot appear. A value larger than “0” is given as a score to the edge connecting “field” and “middle” and the edge connecting “bell” and “tree”, and the characters “field” and “bell” are assigned. The edge to be connected is given “0” as a score. As will be described later, based on these scores, the most likely character string is acquired as a handwritten character recognition result.

  In the present embodiment, the fact that an edge connecting two characters is not registered in the dictionary means that a character string connected via the edge is not registered. The fact that the edge is not registered is equivalent to the fact that the edge is registered and its score is zero.

  The structure of the dictionary as described above and the character recognition method using the same may be the same as the conventional one. The score determination method as described above is an example, and the score may be determined according to a rule different from the above as long as the score is determined so that the connection is likely to occur.

  However, for example, not all character strings (namely, connections between characters) that may be inputted are registered in the data character string pattern dictionary 166 in which character strings of names are registered. . When performing character recognition using only the data character string pattern dictionary 166 prepared in advance as in the prior art, the recognition accuracy when a name that is not registered in the dictionary is input is significantly reduced. In order to prevent such a situation, it may be possible to perform additional registration of a new character string, but in this case, the cost of maintenance increases.

  In this embodiment, in order to improve the recognition accuracy of handwritten characters without increasing the maintenance cost, the recognition result of the printed characters described in the personal identification document is added to the data character string pattern dictionary 166. The specific procedure will be described below.

  FIG. 7 is a flowchart of the dynamic addition process of the data character string pattern dictionary executed by the window reception system 10 according to the embodiment of this invention.

  The process shown in FIG. 7 is executed in step 404 of FIG. Here, as an example, the handwritten character string “Taro Watanabe” is written in the name field 201 of the application form 211, and the type character string “Taro Watanabe” is written in the name field 202 of the driver's license 212. The case will be described.

  First, the window reception system 10 loads a predefined data character string pattern dictionary 701 into the work area 15. This corresponds to the data character string pattern dictionary 166 before step 403 of FIG. 4 is executed. As described with reference to FIG. 6, the loaded data character string pattern dictionary 701 is expressed in a graph format. Specific examples thereof will be described later.

  Next, the window reception system 10 acquires the recognition result of the printed character in step 403 in FIG. 4 and additionally registers it in the data character string pattern dictionary 701.

  Here, an example of the data character string pattern dictionary 701 shown in FIG. 7 will be described. The data character string pattern dictionary 701 at the time of loading includes “pass” (character 721A), “side” (character 721B), “thick” (character 721C), “ro” (character 721D), “well” (character 721E) and “next” (character 721F) are registered as nodes, and “Wat” and “Side”, “Side” and “Thick”, “Tat” and “Akira”, “Wat” and “I”, Edges with a score greater than 0 are included that connect “well” and “thick”, “side” and “next”, and “next” and “ro”. This means that the character strings “Taro Watanabe”, “Jiro Watanabe”, and “Taro Watai” are registered.

  On the other hand, in the example of FIG. 7, the character string “Taro Watanabe” is acquired as the recognition result in step 403. Since such a character string is not registered in the data character string pattern dictionary 701, when “Taro Watanabe” is input as a handwritten character, the recognition may fail (even if a correct handwritten character is input). Is expensive. Therefore, the window reception system 10 adds the character string “Taro Watanabe” acquired in step 403 to the data character string pattern dictionary 701. Specifically, the window reception system 10 adds “邊” (character 712) as a node, and further, an edge 713A connecting “pass” and “邊”, and “邊” and “thick”. Are added, and a value greater than 0 is registered as the score of those edges.

  On the other hand, the character string acquired in step 403 may already be registered in the data character string pattern dictionary 166. For example, when “Taro Watanabe” is acquired, the edge corresponding to the character string “Taro” included therein, that is, the edge 713C connecting “Tai” and “Roh” has already been registered. In this case, the window reception system 10 increases the score of the edge 713C.

  In addition, the score value of the edge newly added in this way and the increment of the existing score can be arbitrarily determined as long as it is larger than 0. However, the larger the score value, the more the edge corresponds. A character string is easily acquired as a recognition result. For example, the greater the score of the edge corresponding to “Taro Watanabe”, the lower the possibility that an error that the handwritten character string “Taro Watanabe” is not recognized as “Taro Watanabe” will occur. Therefore, in order to reduce such an error, the score value may be determined so that the score of the edge corresponding to the character string acquired in step 403 is larger than the score of the other edges. However, the greater the score of the edge corresponding to “Taro Watanabe”, the higher the possibility of an error that a handwritten character string other than “Taro Watanabe” is recognized as “Taro Watanabe”.

  The window reception system 10 recognizes handwritten characters using the data character string pattern dictionary 702 to which information has been added as described above (step 732). The data character string pattern dictionary 702 corresponds to the data character string pattern dictionary 166 after step 403 in FIG. 4 is executed, and step 732 corresponds to step 406 in FIG. Since this character recognition can be executed by the same method as in the prior art, a detailed description thereof will be omitted, but the outline will be described as follows.

  First, the window reception system 10 cuts out each character constituting the image from the input handwritten character string based on the handwritten character recognition dictionary 162. Usually, a plurality of candidates are listed as the extracted characters. For example, when an image including the handwritten character string “Watana” is input, “Wa” and “H” can be cut out therefrom, but “three”, “degree”, and “H” can also be cut out. A plurality of candidates such as “Watanabe” and “Third degree” are created as a character segmentation / identification hypothesis net 741. Edges of the character segmentation / identification hypothesis net 741 (for example, an edge connecting “pass” and “邊”, an edge connecting “three” and “degree”, and an edge connecting “degree” and “邊” ), A score corresponding to the likelihood of the position to be cut out and the character likeness of the cut out image pattern is calculated.

  Then, the window reception system 10 searches the character segmentation / identification hypothesis net 741 of the handwritten character by using the graph of the data character string pattern dictionary 702 to which the information has been added (step 732). This search can be performed by a known method such as DP (Dynamic Programming) matching.

  For example, the character segmentation / identification hypothesis net 741 includes the character string “Third-throw”, but since this character string is not registered in the data character string pattern dictionary 702, it is not acquired as a search result. On the other hand, since the character string “Watanabe” is registered in the data character string pattern dictionary 702, it may be acquired as a search result. In the window reception system 10, the character string “Taro Watanabe” is most likely based on the score given to the edge of the character segmentation / identification hypothesis net 741 and the score given to the edge of the data character string pattern dictionary 702. If it is determined, the character string is output as a handwritten character recognition result (step 733). At this time, information indicating whether or not the recognition result of the handwritten character string matches the recognition result of the printed character string may be output.

  As described above, according to the method shown in FIG. 7, dictionary information created from the recognition result of type characters is added to a data character string pattern dictionary prepared in advance, and the dictionary is used for character recognition. If the character string obtained as a result of type character recognition is not registered in the data character string pattern dictionary, the input handwritten character string is the same as the type character string by registering the character string anew. It becomes easy to be recognized as a character string. On the other hand, if the character string obtained as a result of recognition of type characters has already been registered in the data character string pattern dictionary, the input handwritten character string is typed by increasing the score corresponding to the character string. It becomes easy to be recognized as the same character string as the character string. Thus, since the recognition accuracy of handwritten characters is improved, the frequency of failure of identity verification is also reduced, thereby improving the efficiency of the counter work and preventing unauthorized processing by the counter operator.

  In the above example, information based on the recognition result of type characters is added to the data character string pattern dictionary 166 prepared in advance. However, even if the data character string pattern dictionary 166 is not prepared in advance, The recognition accuracy of handwritten characters can be improved by performing recognition in conjunction with recognition of type characters. This will be described below.

  FIG. 8 is a flowchart of handwritten character recognition processing executed when the window reception system 10 according to the embodiment of the present invention does not hold a data character string pattern dictionary in advance.

  Here, as an example, a case where “Taro Sakurai” is input as a handwritten character string and a printed character string will be described.

  An input image 801 input to the window reception system 10 includes image data of an application form 802 and a personal identification document 803. The application form 802 includes a handwritten character string “Taro Sakurai”, and the identification document 803 includes a printed character string “Taro Sakurai”. These correspond to the application form 211 and the driver's license 212 in FIG.

  Although omitted in FIG. 8, the window reception system 10 extracts an item area in which the handwritten character string of the name is described by the same method as in step 405 in FIG. 4. Then, the window reception system 10 creates a character segmentation / identification hypothesis net 811 for the handwritten character string “Taro Sakurai” described in the extracted item area by the same method as in FIG. In FIG. 8, for example, in addition to the character string “Sakurai”, a character string “Kimai” is extracted as a candidate for a recognition result of handwritten characters.

  On the other hand, the window reception system 10 extracts an item area in which a printed character string of a name is described by the same method as in step 402 in FIG. Then, the window reception system 10 executes recognition of the type character string described in the extracted item area by the same method as in Step 403 in FIG. 4, and as a result, the recognition candidate obtained from the type character string The character string “Taro Sakurai” 812 is acquired. In the example of FIG. 8, since the window reception system 10 does not hold the data character string pattern dictionary 166 in advance, the character string “Taro Sakurai” 812 acquired here is referred to as the data character string pattern dictionary 166 in the next step. Is done.

  The order in which the above two steps are executed is arbitrary, and both may be executed in parallel.

  Next, the window reception system 10 searches the character segmentation / identification hypothesis net using the recognition result of the printed character string (step 813). In the example of FIG. 8, a path corresponding to the recognition result “Taro Sakurai” of the printed character string exists in the character cutout / identification hypothesis net 811, and the total score of the character strings extracted from the character cutout / identification hypothesis net is When it is larger than the predetermined value, the window reception system 10 outputs “Taro Sakurai” as a recognition result of handwritten characters (step 814). At this time, information indicating whether or not the recognition result of the handwritten character string matches the recognition result of the printed character string may be output. On the other hand, when the path corresponding to the recognition result of the printed character string does not exist in the character cutout / identification hypothesis net, the window reception system 10 may output information indicating that the recognition of the handwritten character has failed.

  The above search is executed in the same manner as Step 732 in FIG. For example, each edge included in the path corresponding to “Taro Sakurai” extracted from the printed character string is given a score greater than 0, and the total score of the character strings extracted from the character segmentation / identification hypothesis net is predetermined. The character string (for example, “Taro Sakurai”) may be output as the recognition result.

  The creation of the character segmentation / identification hypothesis net 811 and the step 813 correspond to the step 406 in FIG. 4. The acquisition of the recognition candidate character string 812 and the determination of the cost of the edge corresponding thereto are performed in the steps of This corresponds to 404.

  Thus, the recognition accuracy of handwritten characters can be improved without using a dictionary prepared in advance by using the recognition result of printed characters.

  Note that the character recognition described with reference to FIG. 8 is the edge included in the path corresponding to the character string (“Taro Watanabe” in the example of FIG. 7) obtained as a result of the type character recognition in the processing of FIG. This is equivalent to setting the scores of all edges other than 0 to 0. In other words, in the example of FIG. 8, handwritten character recognition is executed using only the recognition result of the printed character as the data character string pattern dictionary, whereas in the example of FIG. Handwritten character recognition is executed using information combined with the prepared dictionary as a data character string pattern dictionary.

  In the examples shown so far, the input handwritten character string and the typed character string are required to be the same at the time of identity verification. In practice, however, perfect identity may not be required. A typical example is a case where a variation in the notation of kanji variants is allowed.

  Kanji written in the identification document is limited to those that can be used in the system that created the document. For this reason, if the kanji of the name is something that cannot be used in the system (so-called external characters), the kanji of the name will be replaced with the corresponding variant and displayed in the identity verification document. As a result, the actual name The kanji and the kanji described in the identification document will be different.

  Such notation fluctuation also occurs when a variant character is commonly used as an abbreviation. For example, this may be the case when the original notation is “Watanabe” but “Watanabe” is used as an abbreviation when writing it. Depending on the policy of the system, there may be a case where such a variation in the notation is allowed and the character string is not required to be completely identical at the time of identity verification.

  Another example of the case where perfect identity is not required is when the character types of the handwritten character string and the printed character string are different. For example, when a passport is used as the identity verification document and handwritten kanji is used in the application form, the handwritten kanji character string is compared with the roman character string printed on the passport. In this case, it is not required that the written characters match, but it is required that the reading of kanji and the reading of romaji match.

  In such a case, the external character / Romaji dictionary 163 is used for character recognition. Hereinafter, character recognition in such a case will be described.

  FIG. 9 is a flowchart showing another example of handwritten character recognition processing executed by the window reception system 10 according to the embodiment of the present invention when the data character string pattern dictionary is not held in advance.

  Here, in the case where the data character string pattern dictionary 166 is not prepared in advance as in FIG. 8, the character string “Taro Sakurai” is described in the identity verification document as in FIG. Indicates an example in which the handwritten character string “Taro Sakurai” is described. "Cherry blossom" and "Samurai" are variant characters that are commonly used interchangeably. Hereinafter, in the processing shown in FIG. 9, detailed description of the same parts as those in FIG. 8 is omitted.

  An input image 901 illustrated in FIG. 9 includes image data of the application form 902 and the identity verification document 903. The application form 902 includes a handwritten character string “Taro Sakurai”, and the identity verification document 903 includes a printed character string “Taro Sakurai” as in the example of FIG. These correspond to the application form 211 and the driver's license 212 in FIG.

  The window reception system 10 creates a character segmentation / identification hypothesis net 911 for the handwritten character string “Taro Sakurai” by the same method as in FIG. In FIG. 9, for example, in addition to the character string “Sakurai”, a character string “Kimoi” and the like are extracted.

  Further, the window reception system 10 acquires the recognition candidate character string “Taro Sakurai” 912. This is the same as the acquisition of the recognition candidate character string 812 in the example of FIG.

  Next, the window reception system 10 executes expansion using external characters based on the external character / Romaji dictionary 163 for the acquired recognition candidate character string 912 (step 921). The external character / Romaji dictionary 163 includes information that associates a plurality of characters or a plurality of character strings that are allowed to be determined to be the same (in other words, can be replaced). For example, when it is allowed to determine that “sakura” and “櫻” are the same, the external character / Roman dictionary 163 includes information that associates “sakura” with “櫻”. As a result, the character string “Taro Sakurai” in which “Sakura” of “Taro Sakurai” is replaced by the character “櫻” associated therewith is added as the character string of the recognition candidate.

  Then, the window reception system 10 searches the character segmentation / identification hypothesis net 911 using the character string 922 of the recognition result of the developed type character string (step 913). In the example of FIG. 9, the character string 922 of the recognition result includes the character strings “Taro Sakurai” and “Taro Sakurai”. Among these, a path corresponding to “Taro Sakurai” exists in the character segmentation / identification hypothesis net 911. Therefore, the window reception system 10 outputs “Taro Sakurai” as a handwritten character recognition result (step 914). At this time, information indicating whether or not the recognition result of the handwritten character string matches the recognition result of the printed character string may be output.

  On the other hand, if the path corresponding to any character string included in the character string 922 of the recognition result of the printed character string does not exist in the character cutout / identification hypothesis net, the window reception system 10 indicates that the handwritten character recognition has failed. The information shown may be output.

  When handwritten characters are written on the application form and Roman characters are written on the identification document, the same processing as above is performed. This will be described as an example in which the personal identification document 903 is a passport.

  In this example, the personal identification document 903 includes a printed character string “Sakurai Taro” (not shown) as a name, and information that associates the character string “Sakurai Taro” with “Taro Sakurai” in the external character / Roman character dictionary 163. included. In this case, as a result of step 921, a character string 922 of a recognition result of a type character string including the character string “Sakurai Taro” (not shown) and “Taro Sakurai” is acquired. Similarly to the above, the window reception system 10 searches the character segmentation / identification hypothesis net 911 using “Taro Sakurai” included in the character string 922, and outputs “Taro Sakurai” as a handwritten character recognition result (step) 914).

  Such development using the external character / Romaji dictionary 163 can also be applied to the processing shown in FIG.

  For example, in the example of FIG. 7, when “Taro Watanabe” is written as the type name character string in the identity verification document, and the external character / Romaji dictionary 163 includes information associating the characters “side” and “邊”, The reception system 10 adds the character string “Taro Watanabe” and the character string “Taro Watanabe” in which the “side” is replaced with “邊” to the data character string pattern dictionary 701. The addition of “Taro Watanabe” has already been described with reference to FIG. On the other hand, since “Taro Watanabe” has already been registered in the data character string pattern dictionary 701, the window reception system 10 increases the score of the edge included in the path corresponding to “Taro Watanabe”. If “Taro Watanabe” is not registered in the data character string pattern dictionary 701, it is added to the data character string pattern dictionary 701 in the same manner as “Taro Watanabe”.

  By using the external character / Romaji dictionary 163 as described above, it is possible to prevent a failure in identity verification due to a variation in character notation or a difference in character type. It should be noted that the above-described replacement of Kanji characters with variants and Roman characters is an example, and even when replacement with other characters is allowed, the dictionary similar to the external character / Roman character dictionary 163 (that is, mutually replaceable characters) Alternatively, it is possible to execute the same processing as described above using information that associates character strings.

  In addition, although this embodiment demonstrated the window service in the government office or a company as an example, if this invention is a service in which a handwritten character and a type letter corresponding to it are input, it is applied to work other than a window service. You can also.

10 Window reception system 11, 110 Input device 111 Image input device 12 Display device 13 CPU
14 Printing device 15 Work area 151 OS
152 Communication program 153 Document processing program 154 Character recognition program 16 Information holding means 161 Typed character recognition dictionary 162 Handwritten character recognition dictionary 163 External character / Romaji dictionary 164 Location knowledge dictionary 165 Item name dictionary 166 Data character string pattern dictionary 19 Communication network 21 file server

Claims (6)

A document processing apparatus comprising: an input device; a processor connected to the input device; a storage device connected to the processor; and an output device connected to the processor;
Said storage device, printed characters recognition dictionary including information indicating the characteristic quantity of printed characters, handwritten character recognition dictionary including information indicating a characteristic of the handwritten characters, as well as a plurality of characters, the connection between the plurality of character And a string pattern dictionary including a score indicating the likelihood of the connection ,
The processor is
Obtaining image data including a handwritten character string and a print character string input via the input device;
Recognizing the printed character string using the printed character recognition dictionary;
When the character string acquired as a recognition result of the type character string and the connection between the characters included in the character string are already registered in the character string pattern dictionary, the score of the score related to the connection between the registered characters When the value is incremented and the character string acquired as a recognition result of the printed character string and the connection between the characters included in the character string are not registered in the character string pattern dictionary, acquired as the recognition result of the printed character string Newly registered in the character string pattern dictionary, a character indicating the character included in the character string, the connection between those characters and the likelihood of the connection ,
Recognizing the handwritten character string using the handwritten character recognition dictionary and the character string pattern dictionary,
A document processing apparatus that outputs a recognition result of the handwritten character string via the output apparatus. The processor is
Based on the handwritten character recognition dictionary, generate a plurality of character strings as candidates for the recognition result of the handwritten character string,
Search the character string included in the character string pattern dictionary from the plurality of generated character strings,
The document processing apparatus according to claim 1, wherein the character string acquired by the search is acquired as a recognition result of the handwritten character string. The storage device further holds a replacement dictionary that associates a plurality of characters or character strings determined to be the same,
  The processor generates a character string in which a character or character string included in a character string acquired as a recognition result of the type character string is replaced with a character or character string determined to be the same based on the replacement dictionary. The document processing apparatus according to claim 1, wherein the generated character string is stored in the storage device as the character string pattern dictionary. A document processing method executed by a device including an input device, a processor connected to the input device, a storage device connected to the processor, and an output device connected to the processor,
  The storage device includes a printed character recognition dictionary including information indicating a characteristic amount of a printed character, a handwritten character recognition dictionary including information indicating a characteristic amount of a handwritten character, a plurality of characters, and a connection between the plurality of characters. And a string pattern dictionary including a score indicating the likelihood of the connection,
  The document processing method includes:
  A first procedure in which the processor acquires image data including a handwritten character string and a printed character string input via the input device;
  A second procedure in which the processor recognizes the type character string using the type character recognition dictionary;
  When the processor acquires a character string acquired as a recognition result of the printed character string and a connection between characters included in the character string is already registered in the character string pattern dictionary, the connection between the registered characters When the connection between the character string acquired as a recognition result of the type character string and the characters included in the character string is not registered in the character string pattern dictionary, the score value of the type character string is increased. A third procedure for newly registering a character included in a character string acquired as a recognition result, a connection between those characters, and a score indicating the likelihood of the connection in the character string pattern dictionary;
  A fourth procedure for the processor to recognize the handwritten character string using the handwritten character recognition dictionary and the character string pattern dictionary;
  A document processing method comprising: a fifth procedure in which the processor outputs a recognition result of the handwritten character string via the output device. The fourth procedure includes
  The processor generates a plurality of character strings as candidates for the recognition result of the handwritten character string based on the dictionary for handwritten character recognition;
  A step of searching for a character string included in the character string pattern dictionary from the plurality of generated character strings;
  The document processing method according to claim 4, further comprising: a procedure in which the processor acquires a character string acquired by the search as a recognition result of the handwritten character string. The storage device further holds a replacement dictionary that associates a plurality of characters or character strings determined to be the same,
  In the document processing method, a character or character string that is determined to be the same as a character or character string included in a character string acquired as a recognition result of the type character string by the processor based on the replacement dictionary. Including a procedure to generate a string replaced with
  The document processing method according to claim 4, wherein the third procedure includes a procedure of storing the generated character string in the storage device as the character string pattern dictionary.

Images