JP2015170130A - recognition device, recognition method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To synchronize processing different from collation processing with the collation processing and execute the processing different therefrom.SOLUTION: A recognition device comprises: a candidate detection section detecting character candidates being a group of pixels estimated to include characters from an input image; a recognition section recognizing individual ones of the character candidates and generating at least one recognition candidate being the characters of the candidate of recognition results; a collation section collating each of at least one recognition candidate with a knowledge dictionary obtained by modeling a character string of a recognition object and generating collation results obtained by collating a character string estimated to be included in the input image with the knowledge dictionary; and a command processing section executing an issued command. The knowledge dictionary is a finite automaton and is allocated with any one of an actual character code representing the character and a virtual character code specifying a command to an edge. The collation section issues the command specified by the virtual character code allocated to the edge to the command processing section when the virtual character code causes a state of the knowledge dictionary to be subjected to transition in accordance the edge allocated with the virtual character code.

Description

  Embodiments described herein relate generally to a recognition apparatus, a recognition method, and a program.

  As a method for performing knowledge processing in character recognition, a method is known in which a character string to be recognized is modeled and stored in a knowledge dictionary, and a character string matching the model is used as a result of the knowledge dictionary. For example, a system is known that includes a word dictionary unit that registers words to be collated and a word collation unit that has a finite automaton that accepts words registered in the word dictionary unit, and outputs the words received by the finite automaton as a result of knowledge processing. (For example, refer to Patent Document 1). In addition, a technique for performing knowledge processing by collating a place name notation described in a context-free grammar with a character recognition candidate is known (see, for example, Patent Document 2).

JP-A-11-143893 Japanese Patent No. 4006176

  However, in the conventional technology, a process different from the knowledge process such as detecting a space between character strings and reflecting the space detection result on the matching score is executed in synchronization with the knowledge process. It was difficult.

  The problem to be solved by the present invention is to perform character recognition using a knowledge dictionary and execute a process different from the collation process in synchronization with the collation process.

  The recognition apparatus according to the embodiment recognizes each of the character candidates from a candidate detection unit that detects a character candidate that is a set of pixels estimated to include a character from the input image, and recognizes each of the character candidates. A recognition unit that generates at least one recognition candidate, and each of the at least one recognition candidate is compared with a knowledge dictionary that models a character string to be recognized, and is estimated to be included in the input image. A collation unit that generates a collation result obtained by collating the character string with the knowledge dictionary, and an instruction processing unit that executes a given command. The knowledge dictionary is a finite automaton, and represents an actual character. Either one of the character code and the virtual character code designating the instruction is assigned to the edge, and the collation unit determines the state of the knowledge dictionary according to the edge to which the virtual character code is assigned. When causing the transfer, giving the specified instruction by the virtual character code allocated to the edge to the instruction processing unit.

FIG. 1 is a diagram illustrating a configuration of a recognition device 10 according to the embodiment. FIG. 2 is a flowchart showing processing of the recognition apparatus 10 according to the embodiment. FIG. 3 is a diagram illustrating an example of an input image. FIG. 4 is a diagram showing the configuration of style data. FIG. 5 is a diagram illustrating a process for generating a series of character candidates from an input image. FIG. 6 is a diagram showing the structure of fragment data. FIG. 7 is a diagram illustrating an example of a fragment number. FIG. 8 is a diagram illustrating a configuration of character candidate data. FIG. 9 is a diagram illustrating an example of character candidate numbers. FIG. 10 is a diagram illustrating an example of the start point number and end point number of a character candidate. FIG. 11 is a diagram illustrating an example of a character candidate matrix. FIG. 12 is a diagram showing the configuration of the character recognition dictionary. FIG. 13 is a diagram showing the configuration of recognition candidate sequences. FIG. 14 is a diagram illustrating a configuration of the knowledge dictionary. FIG. 15 is a diagram illustrating an example of a code encoding method. FIG. 16 is a diagram illustrating an example of instruction data. FIG. 17 is a diagram illustrating a configuration of the collation result data. FIG. 18 is a flowchart showing the matching process. FIG. 19 is a flowchart showing the knowledge dictionary search process. FIG. 20 is a diagram illustrating an example of the flow of data access in the knowledge dictionary search process. FIG. 21 is a flowchart showing instruction execution processing. FIG. 22 is a diagram illustrating an example of the flow of data access in the instruction execution process. FIG. 23 is a diagram showing how to check the space between characters. FIG. 24 is a flowchart showing the result extraction process. FIG. 25 is a diagram illustrating an example of data referred to in the result extraction and a character code stacked on the stack. FIG. 26 is a diagram illustrating a hardware configuration of the recognition apparatus 10 according to the embodiment.

  FIG. 1 is a diagram illustrating a configuration of a recognition device 10 according to the embodiment. For example, the recognition device 10 recognizes a character string included in an input image read by a scanner or the like, and outputs the recognized character string.

  The recognition device 10 includes an input unit 30, an input image storage unit 32, a style data storage unit 34, a candidate detection unit 36, a candidate storage unit 38, a character recognition dictionary storage unit 40, a recognition unit 42, knowledge A dictionary storage unit 44, a verification unit 46, a verification result storage unit 48, an instruction processing unit 51, a result extraction unit 54, and an output unit 56 are provided.

  The input unit 30 inputs an input image captured by a scanner or the like. The input unit 30 may input an input image from another computer via a network or the like. The input image storage unit 32 stores the input image input by the input unit 30.

  The form data storage unit 34 stores form data for specifying an area in which a character string in the input image is described.

  The candidate detection unit 36 detects a character candidate from the input image based on the format data stored in the format data storage unit 34. Each character candidate is a set of pixels estimated to include one character. The candidate detection unit 36 writes the detected character candidate in the candidate storage unit 38.

  The candidate storage unit 38 stores character candidates. Furthermore, the candidate memory | storage part 38 memorize | stores the recognition candidate which is a candidate character of the recognition result of the character candidate corresponding to each of a character candidate.

  The character recognition dictionary storage unit 40 stores a character recognition dictionary. The character recognition dictionary stores information for calculating the degree of similarity between the image to be recognized and each character registered in advance.

  The recognition unit 42 recognizes each of the character candidates stored in the candidate storage unit 38 based on the character recognition dictionary stored in the character recognition dictionary storage unit 40. Then, the recognition unit 42 generates at least one recognition candidate representing a recognition result candidate character for one character candidate. The recognition unit 42 writes the generated at least one recognition candidate in the candidate storage unit 38 in association with the character candidate.

  The knowledge dictionary storage unit 44 stores a knowledge dictionary obtained by modeling a character string to be recognized. In the present embodiment, the knowledge dictionary is a deterministic finite automaton that models a character string to be recognized. In the knowledge dictionary, one of a real character code representing a character and a virtual character code designating a command is assigned to the edge.

  The collation unit 46 collates each of at least one recognition candidate with the knowledge dictionary, and generates a collation result obtained by collating the character string estimated to be included in the input image with the knowledge dictionary. In this process, the collation unit 46 performs collation by sequentially changing the state of the knowledge dictionary according to the edge to which the recognition candidate is assigned. Moreover, the collation part 46 produces | generates the collation result containing the score showing the likelihood of a corresponding character string. Then, the matching unit 46 writes the generated matching result in the matching result storage unit 48.

  Furthermore, when the state of the knowledge dictionary is changed according to the edge to which the virtual character code is assigned, the matching unit 46 gives the instruction specified by the virtual character code assigned to the edge to the instruction processing unit 51.

  The verification result storage unit 48 stores the verification result generated by the verification unit 46. The collation result storage unit 48 stores collation results at the start point, the middle stage, and the completion point in the process of collating character strings obtained by arranging recognition candidates of character candidates with the knowledge dictionary from the top. Note that the collation unit 46 may delete a collation result having a low score for the purpose of saving the storage area in the middle of the collation.

  The instruction processing unit 51 executes the instruction given from the collation unit 46. For example, the command processing unit 51 executes a process of adding a value designated to the score or a process of checking a space between characters.

  The result extraction unit 54 selects one or more collation results based on the scores from the collation results stored in the collation result storage unit 48 after the collation by the collation unit 46 and the processing by the instruction processing unit 51 are all completed. A character string specified by one or more selected matching results is extracted. As an example, the result extraction unit 54 extracts a character string specified by a matching result having the best score.

  The output unit 56 outputs the character string extracted by the result extraction unit 54 to the outside.

  FIG. 2 is a flowchart showing processing of the recognition apparatus 10 according to the embodiment. First, in step S1, the recognition apparatus 10 inputs an input image.

  Subsequently, in step S <b> 2, the recognition apparatus 10 detects a character candidate that is a set of pixels presumed to include one character from the input image. Subsequently, in step S3, the recognition apparatus 10 recognizes each of the character candidates based on the character recognition dictionary, and generates at least one recognition candidate that is a candidate character of the recognition result.

  Subsequently, in step S4, the recognition apparatus 10 collates each knowledge candidate with at least one recognition candidate, and obtains a collation result obtained by collating the character string estimated to be included in the input image with the knowledge dictionary. Generate. At the same time, in step S4, when the recognition device 10 changes the state of the knowledge dictionary according to the edge to which the virtual character code is assigned, the recognition device 10 executes the instruction specified by the virtual character code assigned to the edge.

  Subsequently, in step S5, the recognition device 10 selects one collation result based on the score from the collation result after extracting all the collation processes, extracts a character string specified by the selected collation result, and recognizes it. The result string. When the number of character candidates is 0, that is, when no character is included in the input image, a collation result to be selected is not generated in step S5. In this case, the character string of the recognition result is an empty character string. Finally, in step S6, the recognition device 10 outputs a character string as a recognition result.

  FIG. 3 is a diagram illustrating an example of an input image. In the present embodiment, as shown in FIG. 3, the input image is image data obtained by taking an order form for ordering a product with a scanner or the like. The name of the orderer is entered inside a predetermined entry frame of the input image. In the present embodiment, the recognition apparatus 10 recognizes a character string with a Japanese name entered inside a predetermined entry frame, and outputs text data representing the recognized character string.

  FIG. 4 is a diagram showing the configuration of style data. The form data storage unit 34 stores form data created in advance.

  As shown in FIG. 4, the format data includes a value indicating the number of entry frames included in the input image and an array for storing entry frame records for the number of entry frames. In this example, the index of the first entry in the array is zero. That is, the sequence is 0 origin. Other sequences used in this embodiment are also 0 origin unless otherwise specified. Each entry frame record corresponds to each entry frame included in the input image on a one-to-one basis.

  Each entry frame record includes information indicating the position of the corresponding entry frame in the input image. In this example, the information indicating the position of the entry frame is the X coordinate (horizontal coordinate) of the left and right ends and the Y coordinate (vertical coordinate) of the upper and lower ends of the corresponding entry frame.

  FIG. 5 is a diagram illustrating a process for generating a series of character candidates from an input image. The candidate detection unit 36 specifies an area of the entry frame based on the information indicated in the entry frame record (for example, an area surrounded by a dotted line in FIG. 5), and extracts a partial area image from the specified area. Subsequently, the candidate detection unit 36 binarizes the extracted partial region image to generate a binary image. Subsequently, the candidate detection unit 36 extracts black pixel connected components on the binary image and performs labeling on each connected component. Each labeled connected component is an element constituting a character and is called a fragment. Subsequently, the candidate detection unit 36 generates a character candidate by combining one or more pieces arranged in a row. A character candidate is a set of pixels presumed to represent one character.

  FIG. 6 is a diagram showing the structure of fragment data. The candidate storage unit 38 stores fragment data representing a fragment. As shown in FIG. 6, the fragment data includes a value indicating the number of fragments and an array for storing fragment records for the number of fragments. Each fragment record has a one-to-one correspondence with each fragment.

  Each fragment record includes information indicating the position of the corresponding fragment and a binary image indicating the shape of the fragment. In this example, the information indicating the position of the fragment is the X coordinate of the left and right ends of the corresponding fragment and the Y coordinate of the upper and lower ends, and indicates the circumscribed rectangle of the fragment. A binary image indicating the shape of a fragment is an image in which pixels on the connected component are black pixels and the rest are white pixels in a circumscribed rectangle of the fragment.

  The candidate detection unit 36 calculates the center X coordinate and the center Y coordinate for each fragment. The center X coordinate is an average value of the X coordinates of the left and right ends. The center Y coordinate is an average value of the Y coordinates of the upper and lower ends. Then, the candidate detection unit 36 sorts the plurality of fragment records in the array in ascending order of the center X coordinate. Thereby, the candidate detection unit 36 can align the plurality of fragment records in the array in the character entry direction in the entry frame (in this example, the direction from left to right).

  FIG. 7 is a diagram illustrating an example of a fragment number. Each fragment record is identified by an array index. The index after the fragment records are arranged in the character entry direction is called a fragment number. Therefore, each fragment is associated with a fragment number as shown in FIG.

  FIG. 8 is a diagram illustrating a configuration of character candidate data. The candidate detection unit 36 generates a character candidate by combining one or more pieces arranged in a row. In this process, the candidate detection unit 36 generates a character candidate by combining one or more fragments in all patterns in which the lateral width L of the circumscribed rectangle is equal to or less than a predetermined length (Lmax).

  The candidate storage unit 38 stores character candidate data representing character candidates. As shown in FIG. 8, the character candidate data includes a value indicating the number of generated character candidates, a character candidate matrix (described later in detail), and an array for storing character candidate records for the number of character candidates. Each character candidate record has a one-to-one correspondence with each character candidate.

  Each character candidate record includes information indicating the position of the corresponding character candidate, a start point number and an end point number (described later in detail) of the corresponding character candidate, a binary image indicating the shape of the character candidate, and a recognition candidate entry. Includes recognition candidate sequences (described in detail below). In this example, the information indicating the position of the character candidate is the X coordinate of the left and right ends and the Y coordinate of the upper and lower ends of the corresponding character candidate, and indicates the circumscribed rectangle of the character candidate on the binary image. A binary image showing the shape of a character candidate is an image in which the pixels on the character candidate are black pixels and the rest are white pixels in the circumscribed rectangle of the character candidate. A value of the recognition candidate entry is set by the recognition unit 42, and no value is set by the candidate detection unit 36.

  FIG. 9 is a diagram illustrating an example of character candidate numbers. Each character candidate record is identified by an array index. The index of the character candidate record is called a character candidate number. Therefore, a character candidate number is associated with each character candidate as shown in FIG.

  FIG. 10 is a diagram illustrating an example of the start point number and end point number of a character candidate. Character candidates are generated by combining one or more pieces arranged in series. For this reason, character candidates are uniquely identified by a set of a fragment number for the first fragment in the sequence of one or more fragments as a source and a value obtained by adding 1 to the fragment number for the last fragment. Can do.

  In this embodiment, the fragment number for the first fragment is called the start point number of the character candidate, and the value obtained by adding 1 to the fragment number for the last fragment is called the end point number of the character candidate. Accordingly, each character candidate is associated with a start point number and an end point number as shown in FIG. Note that the start point number and the end point number represent character character delimiter positions, and thus both the start point number and the end point number are collectively referred to as position numbers.

  FIG. 11 is a diagram illustrating an example of a character candidate matrix. As shown in FIG. 11, the character candidate matrix is a two-dimensional array of character candidate numbers having a start point number as a first index and an end point number as a second index. The character candidate matrix is initialized by setting all entries to -1 before starting the generation of character candidate records. Each time the candidate detection unit 36 creates a character candidate, it writes the character candidate number in the corresponding entry of the character candidate matrix.

  FIG. 12 is a diagram showing the configuration of the character recognition dictionary. The character recognition dictionary storage unit 40 stores a character recognition dictionary created in advance. As shown in FIG. 12, the character recognition dictionary includes a value indicating the number of dictionary entries and an array for storing dictionary entries.

Each dictionary entry includes a character code and predetermined D _sub basis vectors. A base vector is a feature vector of a subspace representing a character corresponding to a character code. As an example, the feature vector is obtained by dividing a binary image of a corresponding character by a predetermined number in the vertical direction and the horizontal direction, obtaining a ratio of the number of black pixels in each divided area, It is calculated by using the ratio as an element of the feature vector.

FIG. 13 is a diagram showing the configuration of recognition candidate sequences. The recognition candidate array stored in the character candidate record includes predetermined N _cand recognition candidate entries as shown in FIG. Each recognition candidate entry includes a character code and a similarity.

The recognition unit 42 performs character recognition for each of the character candidates, and generates at least one recognition candidate that is a candidate character of the recognition result. In the present embodiment, the recognition unit 42 generates N _cand recognition candidate entries determined in advance for each character candidate record, and writes them in the recognition candidate array.

More specifically, the recognition unit 42 extracts a feature vector from a binary image included in the corresponding character candidate record, and collates it with a base vector stored in each dictionary entry of the character recognition dictionary by a subspace method. To calculate the similarity. The recognizing unit 42 extracts the character code stored in the dictionary entry for each of the top N _cand dictionary entries having the similarity, and generates a recognition candidate entry including the extracted character code and the calculated similarity. Then, the recognition unit 42 _writes the generated N _cand recognition candidate entries in the recognition candidate array of the corresponding character candidate record. Furthermore, the recognition unit 42 arranges the recognition candidate entries included in the recognition candidate arrays of the respective character candidate records in descending order of similarity.

  FIG. 14 is a diagram illustrating a configuration of the knowledge dictionary. The knowledge dictionary storage unit 44 stores a knowledge dictionary created in advance by a designer or the like.

  In the present embodiment, the knowledge dictionary is a deterministic finite automaton that models a character string to be recognized. In the present embodiment, a knowledge dictionary that is a deterministic finite automaton is also referred to as DFAα. For example, DFAα is generated by a designer describing a character string to be recognized in a regular expression and converting the regular expression into a deterministic finite automaton. A method for generating a nondeterministic finite automaton or a deterministic finite automaton from a regular expression, and a method for generating a deterministic finite automaton from a nondeterministic finite automaton are disclosed in, for example, A. V. Aiho, R.A. Cessie, J.H. D. Written by Ullman, translated by Kenichi Harada, Compiler I, first edition 1986, pp. 134-172 and the like.

  As shown in FIG. 14, DFAα includes a value indicating the number of states, a state array for storing state records for the number of states, and an edge array for storing edge records for the number of edges.

  Each state record has a one-to-one correspondence with each state included in DFAα, and the state is uniquely identified by the number of the state record, that is, the state number. It should be noted that the state array is 0 origin, and therefore the state with the state number 0 is the start state. Each status record includes an acceptance status flag, a pointer to the edge record in the edge array, and the number of elements in the edge record.

  The acceptance state flag indicates whether or not the state is an acceptance state. As an example, the acceptance status flag indicates that it is in an acceptance state when it is 1, and indicates that it is not in an acceptance state when it is 0.

  The pointer to the edge record indicates the storage position of the set of edges that exit from the state in the edge array. The number of elements in the edge record represents the number of edges that exit from the state. The edge record corresponding to all the edges going out from the state can be specified by the pointer to the edge record and the number of elements.

  Each edge record has a one-to-one correspondence with each edge included in DFAα. Each edge record includes a transition destination state number and a code.

  The state number of the transition destination represents a state number that specifies the state of the transition destination by the edge. Note that the state having a state number of 0 is a start state.

  The code represents the input symbol that causes the transition represented by the edge. In the present embodiment, the code stores a real character code representing a character or a virtual character code designating an instruction. In DFAα, when an actual character code is assigned to an edge, a transition from one state to another state occurs according to the edge by a character code representing a recognition candidate character. In addition, in DFAα, when a virtual character code is assigned to an edge, a transition occurs from one state to another state according to the edge regardless of the recognition candidate character, and the virtual character code is designated. The instruction code is output.

  FIG. 15 is a diagram illustrating an example of a code encoding method. As an example, the code included in the edge record of the knowledge dictionary represents the distinction between the first bit representing a sign bit and the second bit representing a real character code or a virtual character code. For example, when the second bit is 0, it represents an actual character code, and when the second bit is 1, it represents a virtual character code, and an instruction code described later is designated by the code.

  FIG. 16 is a diagram illustrating an example of instruction data. The instruction processing unit 51 executes a command interpreter, interprets an instruction code specified by a virtual character code, and executes a process defined by the instruction code. The instruction data is defined by, for example, an array of character strings, and is specified by using the number obtained by replacing the upper 2 bits of the virtual character code with 0 as the element number of the array. For example, the instruction code # 0 in FIG. 16 is described as a character string “ADD_SCORE 300”, and is an addition instruction for adding a value 300 designated to a score to be described later. Further, for example, the instruction code # 1 in FIG. 16 is described as a character string “CHEK_SPACE 5 mm”, and is an interval confirmation instruction for confirming whether the space between characters is 5 mm or more.

  FIG. 17 is a diagram illustrating a configuration of the collation result data. The collation unit 46 collates each of the recognition candidates included in the character candidates with the knowledge dictionary sequentially from the top, and collates the character string estimated to be described in the entry frame of the input image with the knowledge dictionary. The verification result obtained is obtained. Then, the matching unit 46 writes the generated matching result in the matching result storage unit 48.

  The verification result storage unit 48 stores verification result data. The matching result data includes the number of matching results and an array of matching results for each position number.

  The number of matching results represents the number of matching results associated with the position number. The collation result array stores the collation result associated with the position number. Each collation result includes a pair of a state number α, a score, a code, a position number, and a collation result number. Each collation result is uniquely identified by a pair of a position number associated with the storage destination array and a number as an array element in the storage destination array. From now on, the position number associated with the collation result storage array is referred to as "position number associated with the collation result", and the number as the array element in the collation result storage array is referred to as "collation result number". It is called.

  The state number α represents the state of the knowledge dictionary (DFAα). That is, the state number α indicates a state reached when DFAα is sequentially shifted from the start state in accordance with each character from the first recognition candidate to the recognition candidate at the position.

  The score represents a value obtained by accumulating the degrees of similarity associated with the recognition candidates at the head to the recognition candidates at the position. That is, the score represents the likelihood of the character string from the first recognition candidate to the recognition candidate at the position. The code is a character code representing a recognition candidate character at the position.

  The pair of the position number and the number of the matching result is obtained immediately after the character candidate is traced one by one from the head to the corresponding position while the matching result is generated while the DFAα is transitioned using the character candidate recognition candidate as the input symbol. It represents the position number associated with the collation result and the number of the previous collation result. The pair of the position number and the collation result number is referred to when the result extraction unit 54 extracts the character string of the recognition result.

  FIG. 18 is a flowchart showing the matching process. Details of the collation process shown in step S4 of FIG. 2 will be described with reference to FIG.

  First, in step S11, the collation unit 46 initializes collation result data. Specifically, the collation unit 46 sets the number of collation results to 0 for all the position numbers of the collation result data and empties the collation result array.

  Subsequently, in step S12, the collation unit 46 generates a new collation result in association with the position number 0. For one new collation result, the state number α is set to 0, the score is set to 0, the position number and the collation result number are set to -1, and the code is set to -1. Subsequently, in step S13, the matching unit 46 sets the number of matching results associated with the position number 0 to 1. Subsequently, in step S14, the collation unit 46 substitutes 0 for a variable Pst representing the position number.

  Subsequently, in step S15, the collation unit 46 determines whether or not Pst is equal to or less than Pstmax. Pstmax is a value obtained by subtracting 1 from the last position number Ped. When Pst is equal to or less than Pstmax (true in step S15), the collation unit 46 advances the process to step S16.

  In step S <b> 16, the collation unit 46 determines whether an edge to which a virtual character code is assigned exists at an edge that exits from the corresponding state. Then, when there is an edge to which the virtual character code is assigned, the collating unit 46 changes the state and gives the instruction processing unit 51 the instruction code specified by the virtual character code to execute the instruction. . The details of the instruction execution process will be described later with reference to FIGS.

  Subsequently, in step S <b> 17, the matching unit 46 narrows down the matching results associated with Pst from the highest score to the Npr-th score. That is, the matching unit 46 deletes the matching result whose score is lower than the Nprth.

  Subsequently, in step S18, the collation unit 46 executes a knowledge dictionary search process using the knowledge dictionary for each collation result associated with Pst. Thereby, the collation part 46 can produce | generate the new collation result linked | related with the position number after Pst. The knowledge dictionary search process will be described later with reference to FIGS. 19 and 20.

  Subsequently, in step S19, the collation unit 46 adds 1 to Pst. The collation part 46 will return a process to step S15, after complete | finishing step S19. And the collation part 46 repeats the process of step S16 to step S19 until Pst exceeds Pstmax.

  When Pst is no longer equal to or less than Pstmax (No in step S15), the collation unit 46 advances the process to step S20. In step S20, the matching unit 46 performs the same process as in step S16 on each matching result associated with the last position number Ped. And the collation part 46 complete | finishes this flow, after finishing the process of step S20.

  FIG. 19 is a flowchart showing the knowledge dictionary search process. FIG. 20 shows an example of the flow of data access in the knowledge dictionary search process.

  The knowledge dictionary search process in step S18 of FIG. 18 will be described with reference to FIGS. First, in step S31, the collation unit 46 refers to the collation result data and lists all the collation results associated with Pst.

  Subsequently, in step S32, the collation unit 46 refers to the character candidate records in the character candidate data array, and lists all the character candidates starting from Pst in the character candidate data array. The collating unit 46 scans all entries whose starting point numbers match Pst in the character candidate matrix and collects all character candidates having Pst as the starting point position by collecting the numbers of character candidates other than -1. be able to.

  Subsequently, the collation unit 46 performs the processing of step S34 to step S48 for each of all the character candidate records listed in step S32 (loop processing between step S33 and step S49). Hereinafter, the character candidate corresponding to the character candidate record to be processed in this loop processing is referred to as “character candidate Cc”.

  In step S34, the collation unit 46 refers to the recognition candidate array of the character candidate record corresponding to the character candidate Cc, and lists all recognition candidate entries of the character candidate.

  Subsequently, the collation unit 46 performs the processing of Step S36 to Step S47 for each of all the recognition candidate entries listed in Step S34 (loop processing between Step S35 and Step S48). Hereinafter, the recognition candidate corresponding to the recognition candidate entry to be processed in this loop processing is referred to as “recognition candidate Cr”.

  Subsequently, the collation unit 46 performs the processing of step S37 to step S46 on each of all the collation results associated with Pst listed in step S31 (loop between step S36 and step S47). processing). Hereinafter, the verification result to be processed in this loop process is referred to as “matching result Mp”.

  In step S37, the collation unit 46 refers to the knowledge dictionary (DFAα) and enumerates state records corresponding to the state number α included in the collation result Mp to be processed.

  Subsequently, in step S38, the collation unit 46 creates an edge record representing an edge that exits from the state of the state number α based on the pointer to the edge record included in the state record enumerated in step S37 and the number of elements of the edge record. By specifying the stored range, all edge records representing the edges going out from the state of the state number α are listed.

  Subsequently, the collation unit 46 performs the processing of Step S40 to Step S45 for each of all the edge records listed in Step S38 (loop processing between Step S39 and Step S46). Hereinafter, the edge record to be processed in this loop processing is referred to as “edge record Er”.

  In step S40, the collation unit 46 determines whether or not the character code set in the recognition candidate entry of the recognition candidate Cr matches the character code set in the edge record Er. If they do not match (No in step S40), the collation unit 46 moves the process to the next edge record and repeats the process from step S40. If they match (Yes in step S40), the collation unit 46 advances the process to step S41.

  In step S41, the collation unit 46 generates a new collation result Mn in association with the end point position of the character candidate record of the character candidate Cc, and writes it in the collation result data.

  Subsequently, in step S42, the collation unit 46 sets the state number (transition destination state number) set in the edge record Er as the state number α in the new collation result Mn.

  Subsequently, in step S43, the collation unit 46 sets the character code set in the recognition candidate entry of the recognition candidate Cr as a code for the new collation result Mn.

  Subsequently, in step S44, the collation unit 46 sets the position number Pst associated with the collation result Mp as the position number for the new collation result Mn. The collation unit 46 stores the number of the collation result Mp to be processed as the collation result number in the new collation result Mn.

  Subsequently, in step S45, the collation unit 46 uses the score stored in the collation result Mp to be processed as a score for the new collation result Mn and the value obtained by adding the similarity stored in the recognition candidate entry of the recognition candidate Cr. Set.

  In step S46, the collation unit 46 exits from the loop and proceeds to step S47 after completing the process of steps S40 to S45 for all edge records.

  In step S47, the collation unit 46 exits from the loop and proceeds to step S48 after completing the processes of step S37 to step S46 for all the collation results associated with Pst.

  In step S48, the collation unit 46 exits from the loop and proceeds to step S49 after completing the process of steps S36 to S47 for all the recognition candidate entries corresponding to the character candidate Cc.

  In step S49, the collation unit 46 exits from the loop and ends the present flow after completing the processes in steps S34 to S48 for all the character candidate records.

  As described above, the collation unit 46 writes the number (state number α) indicating the state of the knowledge dictionary (DFAα) reached by collation in the collation result of the first character candidate. Then, when the collation unit 46 collates the second character candidate following the first character candidate with the knowledge dictionary (DFAα), the collation unit 46 indicates the number (state number α) written in the collation result of the first character candidate. The second character candidate is collated by following the edge corresponding to the state transition by the recognition candidate of the second character candidate.

  FIG. 21 is a flowchart showing instruction execution processing. FIG. 22 shows a data flow in the instruction execution process. FIG. 23 is a diagram showing how to check the space between characters.

  The instruction execution process in step S16 in FIG. 18 will be described with reference to FIGS. First, in step S51, the collation unit 46 refers to the collation result data and lists all the collation results associated with Pst.

  Subsequently, the collation unit 46 executes the processing of step S53 to step S61 for each of all the collation results associated with Pst listed in step S51 (loop between step S52 and step S62). processing). Hereinafter, the verification result to be processed in this loop process is referred to as “matching result Mp”.

  In step S53, the collation unit 46 refers to the knowledge dictionary (DFAα) and enumerates state records corresponding to the state number α set in the collation result Mp to be processed.

  Subsequently, in step S54, the collation unit 46 refers to the edge record set in the state record enumerated in step S53 and the number of elements of the edge record, and refers to the edge exiting from the state of the state number α. List all edge records that represent it.

  Subsequently, the collation unit 46 executes the processing of Step S56 to Step S60 for each of all the edge records listed in Step S54 (loop processing between Step S55 and Step S61). Hereinafter, the edge record to be processed in this loop processing is referred to as “edge record Er”.

  In step S56, the collation unit 46 determines whether or not the code of the edge record Er is a virtual character code. As an example, the collating unit 46 detects the second bit from the beginning of the code and determines whether or not the character code is a virtual character code. If it is not a virtual character code (No in step S56), the collation unit 46 moves the process to the next edge record and repeats the process from step S56. When it is a virtual character code (Yes of step S56), the collation part 46 advances a process to step S57.

  In step S57, the collation unit 46 generates a new collation result Mn in association with Pst and writes it in the collation result data.

  Subsequently, in step S58, the collation unit 46 sets the state number (transition destination state number) set in the edge record Er as the state number α in the new collation result Mn.

  Subsequently, in step S59, the collation unit 46 copies the corresponding value of the collation result Mp as an element (score, code, position number, and collation result number) other than the state number α to the new collation result Mn. Thereby, the collation part 46 can change a state according to the edge to which the virtual character code was allocated.

  Subsequently, in step S60, the collation unit 46 calls the instruction processing unit 51, gives the instruction code specified by the virtual character code, and executes the instruction. The instruction processing unit 51 is mounted with a command interpreter, interprets a given instruction code, and executes a process defined by the instruction code.

  For example, when the addition instruction “ADD_SCORE” is given, the instruction processing unit 51 adds the value specified by the argument to the corresponding matching result score. Thus, the instruction processing unit 51 can operate the evaluation of the likelihood of the character string by adding a weight to the score in the case of a predetermined condition.

  Further, for example, when an instruction code including an interval check instruction “CHECK_SPACE” is executed by a virtual character code during the processing of the matching result Mp associated with the position number Pst in step S60 of FIG. 21, as shown in FIG. In addition, the instruction processing unit 51 confirms the space between the characters before and after the position number Mp by confirming the interval between the fragment immediately before and after the position number Pst. Then, the instruction processing unit 51 deletes the matching result Mp when the interval is equal to or less than the specified distance. Thereby, for example, the instruction processing unit 51 can confirm the separation space between the sex and the name, and can delete the collation result when a space longer than a predetermined distance is not detected.

  Subsequently, in step S61, the collation unit 46 exits from the loop and proceeds to step S62 after completing the processes of steps S56 to S60 for all the edge records.

  In step S62, the collation unit 46 exits the loop and terminates the present flow after completing the processes of step S53 to step S61 for all the collation results associated with Pst.

  As described above, the collation unit 46 writes the number (state number α) indicating the state of the knowledge dictionary (DFAα) reached by collation in the collation result of the first character candidate. Then, when the collation unit 46 collates the second character candidate following the first character candidate with the knowledge dictionary (DFAα), the collation unit 46 indicates the number (state number α) written in the collation result of the first character candidate. The second character candidate is collated by following the edge corresponding to the state transition by the recognition candidate of the second character candidate from the state to be generated, and the collation result of the second character candidate is generated.

  Furthermore, when an edge to which a virtual character code is assigned is detected, the matching unit 46 changes the state according to the edge, and gives an instruction code specified by the virtual character code to the instruction processing unit 51. The instruction can be executed.

  FIG. 24 is a flowchart showing the flow of character string extraction processing performed by the result extraction unit. FIG. 25 is a diagram illustrating data referred to in the result extraction and a state of character codes stacked on the stack.

  The result extraction shown in step S5 of FIG. Hereinafter, details of the result extraction will be described with reference to FIGS. 24 and 25. First, in step S70, the result extraction unit 54 checks whether or not the number of character candidates is 0. If the number of character candidates is 0, the flow ends with the recognition result character string as an empty character string in step S84. To do. If the number of character candidates is not 0, in step S71, the result extraction unit 54 enumerates all the collation results associated with the last position number Ped, and then executes the processing after step S72.

  Subsequently, in step S72, the result extraction unit 54 acquires a state record corresponding to the state number α from the knowledge dictionary (DFAα) for each of the matching results listed in step S71, and confirms the acceptance state flag.

  Subsequently, in step S73, the result extraction unit 54 determines whether there is a collation result in which the state corresponding to the state number α is an accepted state. Hereinafter, the collation result in which the state corresponding to the state number α is the accepting state is referred to as “the collation result of the accepting state with DFAα”. If there is a matching result in the accepted state in DFAα (Yes in step S73), in step S74, the result extracting unit 54 sets the matching result having the maximum score among the matching results in the accepted state in DFAα as the matching result Mx. select. When there is no collation result in the accepted state in DFAα (No in step S73), in step S75, the result extraction unit 54 selects the collation result having the maximum score as the collation result Mx among all the collation results listed. To do.

  Subsequent to step S74 or step S75, in step S76, the result extraction unit 54 substitutes the position number px associated with the selected matching result Mx for the variable p representing the position number. Further, the result extraction unit 54 substitutes the number mx of the selected matching result Mx into the variable m representing the number of the matching result.

  Subsequently, in step S77, the result extraction unit 54 empties the stack which is a FILO (First In Last Out) memory.

  Subsequently, in step S78, it is determined whether the collation result code indicated by p and m is -1. When the code of the collation result indicated by p and m is not −1 (false in step S78), the result extraction unit 54 advances the process to step S79.

  In step S79, the result extraction unit 54 loads the code stored in the matching result indicated by p and m on the stack. Subsequently, in step S80, the result extraction unit 54 sets the position number stored in the matching result indicated by p and m to p, and the number of the matching result stored in the matching result indicated by p and m in p. substitute.

  And the result extraction part 54 complete | finishes the process of step S80, returns a process to step S78, and until the code stored in the collation result which p and m point to becomes -1, the process of step S79 and step S80 is carried out. Repeat the process. Thereby, as shown in FIG. 25, the result extraction unit 54 can select character codes in order from the end of the character string and accumulate them in the stack.

  When the code of the matching result pointed to by p and m is −1 (true in step S78), that is, when the position number indicates the matching result associated with 0, the result extraction unit 54 performs the process. Proceed to S81. In step S81, the result extraction unit 54 initializes the character string of the recognition result stored in the memory to an empty character string.

  Subsequently, in step S82, it is determined whether the stack is empty. If the stack is not empty (false in step S82), the result extraction unit 54 extracts one code from the top of the stack and adds it to the end of the character string of the recognition result stored in the memory in step S83.

  When the process of step S83 is completed, the result extraction unit 54 returns the process to step S82 and repeats the process of step S83 until the stack becomes empty. As a result, the result extraction unit 54 can generate the character string from the beginning to the end.

  Then, when the stack becomes empty (true in step S82), the result extraction unit 54 ends the process of this flow.

  As described above, the recognition apparatus 10 according to the present embodiment can perform character recognition using the knowledge dictionary and execute processing different from the collation processing in synchronization with the collation processing.

  FIG. 26 is a diagram illustrating a hardware configuration of the recognition apparatus 10 according to the embodiment.

  The recognition apparatus 10 can be realized by a general computer system that can execute a program. As an example, the recognition device 10 includes a display 110, a keyboard 112, a scanner 114, an external storage device 116, a communication device 118, and a computer 120.

  The display 110 is a display device and displays a recognized character string or the like. The keyboard 112 is an input device, and receives information from a user and inputs information. The scanner 114 obtains an input image or the like by reading information written on a sheet or the like. The external storage device 116 is a hard disk drive or an optical disk drive, and stores various types of information. The communication device 118 inputs and outputs information with an external computer or the like via the Internet or the like, and acquires, for example, an input image from the outside or outputs a character string to the outside.

  As an example, the computer 120 includes a CPU 122, an input / output control unit 124, and a storage device 126. The CPU 122, the input / output control unit 124, and the storage device 126 are connected by a bus 128.

  The CPU 122 executes a program to control the entire recognition apparatus 10. The input / output control unit 124 is an interface with the display 110, the keyboard 112, the scanner 114, the external storage device 116, the communication device 118, and the like. The input / output control unit 124 also controls data transfer via the bus 128.

  The storage device 126 includes a ROM, a RAM, a hard disk drive, or the like. In the storage device 126, any device such as a ROM, a RAM, or a hard disk drive can be accessed by the same address space. The storage device 126 stores programs, input images, style data, dictionary data (character recognition dictionaries and knowledge dictionaries), work data (character candidates and collation results), command data, and the like. These data may be stored in any device (ROM, RAM, and hard disk drive) constituting the storage device. Further, part or all of these data may be stored in a server or the like accessed via the external storage device 116 or the communication device 118.

  The program executed by the recognition apparatus 10 of the present embodiment is an installable or executable file, and is a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD. Recorded and provided. The program executed by the recognition apparatus 10 of the present embodiment may be configured to be stored by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed by the recognition apparatus 10 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

  The program executed by the recognition apparatus 10 according to the present embodiment includes the above-described units (the input unit 30, the candidate detection unit 36, the recognition unit 42, the collation unit 46, the instruction processing unit 51, the result extraction unit 54, and the output unit 56). As the actual hardware, the CPU (processor) reads out the program from the storage medium and executes it to load the respective units onto the main storage device. The input unit 30 and the candidate detection unit 36 The recognition unit 42, the collation unit 46, the instruction processing unit 51, the result extraction unit 54, and the output unit 56 are generated on the storage device 126. Note that the input unit 30, the candidate detection unit 36, the recognition unit 42, the collation unit 46, the instruction processing unit 51, the result extraction unit 54, and the output unit 56 may be partially or entirely configured by hardware.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 10 Recognition apparatus 30 Input part 32 Input image memory | storage part 34 Style data memory | storage part 36 Candidate detection part 38 Candidate memory | storage part 40 Character recognition dictionary memory | storage part 42 Recognition part 44 Knowledge dictionary memory | storage part 46 Collation part 48 Collation result memory | storage part 51 Command processing part 54 Result Extractor 56 Output Unit 110 Display 112 Keyboard 114 Scanner 116 External Storage Device 118 Communication Device 120 Computer 122 CPU
124 I / O controller 126 Storage device 128 Bus

Claims (8)

A candidate detection unit that detects a character candidate that is a set of pixels estimated to include a character from the input image;
A recognition unit that recognizes each of the character candidates and generates at least one recognition candidate that is a candidate character of a recognition result;
Each of the at least one recognition candidate is collated with a knowledge dictionary obtained by modeling a character string to be recognized, and a collation result obtained by collating a character string estimated to be included in the input image with a knowledge dictionary is obtained. A verification unit to be generated;
An instruction processing unit for executing a given instruction;
With
The knowledge dictionary is a finite automaton, and either one of a real character code representing a character and a virtual character code designating an instruction is assigned to an edge,
The verification unit, when transitioning the state of the knowledge dictionary according to an edge to which the virtual character code is assigned, gives an instruction specified by the virtual character code assigned to the edge to the instruction processing unit. The matching unit generates a matching result including a score representing the likelihood of the corresponding character string,
The recognition apparatus according to claim 1, further comprising: a result extraction unit that selects one matching result based on the score from the matching result and extracts a character string specified by the selected matching result. The recognition apparatus according to claim 2, wherein the instruction processing unit processes an addition instruction for adding a designated value to a score of a corresponding matching result. The instruction processing unit checks a space between characters before and after a position associated with a collation result, and processes an interval confirmation command for deleting a corresponding collation result when the distance is equal to or less than a specified distance. The recognition device according to claim 2. The knowledge dictionary is a deterministic finite automaton;
The collation unit
Write a number indicating the state of the deterministic finite automaton reached by collation in the collation result of the first character candidate,
When the second character candidate following the first character candidate is collated with the deterministic finite automaton, the second character candidate recognition candidate from the state indicated by the number written in the collation result of the first character candidate The recognition apparatus according to claim 2, wherein the second character candidate is collated by following an edge corresponding to the state transition by. A candidate detection unit that detects a character candidate that is a set of pixels estimated to include a character from the input image;
A recognition unit that recognizes each of the character candidates and generates at least one recognition candidate that is a candidate character of a recognition result;
Each of the at least one recognition candidate is collated with a knowledge dictionary obtained by modeling a character string to be recognized, and a collation result obtained by collating a character string estimated to be included in the input image with a knowledge dictionary is obtained. A verification unit to be generated;
An instruction processing unit for executing a given instruction;
With
The knowledge dictionary is a deterministic finite automaton, character codes are assigned to edges,
The collation unit writes a number indicating the state of the deterministic finite automaton reached by collation to the collation result of the first character candidate, and sets the second character candidate following the first character candidate as the deterministic finite automaton. When collating, the second character candidate is collated by following the edge corresponding to the state transition by the recognition candidate of the second character candidate from the state indicated by the number written in the collation result of the first character candidate. A recognition device for generating a collation result of the second character candidate. A candidate detection step of detecting a character candidate that is a set of pixels presumed to include a character from the input image;
A recognition step of recognizing each of the character candidates and generating at least one recognition candidate that is a candidate character of a recognition result;
Each of the at least one recognition candidate is collated with a knowledge dictionary obtained by modeling a character string to be recognized, and a collation result obtained by collating a character string estimated to be included in the input image with a knowledge dictionary is obtained. A matching step to generate,
An instruction processing step for executing a given instruction;
Including
The knowledge dictionary is a finite automaton, and one of a real character code representing a character and a virtual character code including an instruction is assigned to an edge,
In the collating step, when the state of the knowledge dictionary is changed according to an edge to which the virtual character code is assigned, a command included in the virtual character code assigned to the edge is executed. A program for causing a computer to function as a recognition device,
In the computer,
A candidate detection step of detecting a character candidate that is a set of pixels presumed to include a character from the input image;
A recognition step of recognizing each of said character candidates and generating at least one recognition candidate representing a candidate character of a recognition result;
Each of the at least one recognition candidate is collated with a knowledge dictionary obtained by modeling a character string to be recognized, and a collation result obtained by collating a character string estimated to be included in the input image with a knowledge dictionary is obtained. A matching step to generate,
An instruction processing step for executing a given instruction;
And execute
The knowledge dictionary is a finite automaton, and either one of a real character code representing a character and a virtual character code designating an instruction is assigned to an edge,
In the collation step, when the state of the knowledge dictionary is changed according to an edge to which the virtual character code is assigned, a program for executing an instruction specified by the virtual character code assigned to the edge.

Images