JP7279381B2 - Information processing device and information processing program - Google Patents

Description

The present invention relates to an information processing device and an information processing program.

In Patent Document 1, in an information retrieval device for retrieving predetermined information from information holding means holding a plurality of information, a document image input means, a character recognition means, and a normalization level is determined from the certainty of character recognition. normalization means for replacing representative characters based on the normalization level; search means for searching information containing similar text from the information holding means; and search result output means for outputting the search results. An information retrieval device characterized by the following is disclosed.

Patent Document 2 discloses an information processing device for searching a character string for a specific term contained in a given dictionary, wherein partial characters of the character string are searched for the given dictionary using the n-gram method. An extracting means for performing a string search and extracting one or more specific term candidates included in the character string from the given dictionary, and using a DP matching method for each of the one or more specific term candidates. derivation means for deriving an edit distance between a specific term candidate and a substring in the character string corresponding to the specific term candidate, and the edit distance corresponding to the one or more specific term candidates shorter than a predetermined threshold and output means for outputting a partial character string in the character string.

JP 2007-323415 A JP 2018-81451 A

According to the present invention, a character string included in an image can be identified with high accuracy as compared with the case of identifying the character string included in the image from the character string recognized from the image by character recognition processing without using a predetermined reference list. An object of the present invention is to provide an information processing device and an information processing program that can be specified.

An information processing apparatus according to a first aspect includes an acquisition unit that acquires a character string recognized from an image by character recognition processing; An extraction unit for extracting a character string candidate, a correction unit for correcting the character string candidate using an erroneous recognition pattern recognized by the character recognition process, and a predetermined reference list that is corrected by the correction unit. an output unit for outputting a character string candidate specified from among the character string candidates obtained.

An information processing apparatus according to a second aspect is the information processing apparatus according to the first aspect, wherein the extraction unit uses a modified extraction pattern represented by a modified regular expression obtained by modifying the regular expression using the misrecognition pattern. to extract the character string candidate.

An information processing apparatus according to a third aspect is the information processing apparatus according to the first aspect or the second aspect, wherein the acquisition unit acquires the image, and the correction unit performs character recognition of the image on a specific character. a character recognition processing unit that performs processing; and a generation unit that generates misrecognized character string candidates that are character string candidates of the same type as the character string recognized by the character recognition processing unit. Output the character string candidates specified from among the recognized character string candidates.

An information processing apparatus according to a fourth aspect is the information processing apparatus according to the third aspect, wherein the generation unit adds a character string generated using an erroneous recognition pattern specified by a user to the erroneously recognized character string candidates. .

An information processing apparatus according to a fifth aspect is the information processing apparatus according to the third aspect or the fourth aspect, wherein the generating unit generates the misrecognized character string candidates by excluding the character string candidates represented by the extraction patterns. do.

An information processing device according to a sixth aspect is the information processing device according to any one of the third to fifth aspects, wherein the character recognition processing unit calculates a score representing accuracy of character recognition, and the output unit and outputting the identified character string candidate when the score is equal to or greater than a predetermined threshold.

An information processing device according to a seventh aspect is the information processing device according to any one of the third to fifth aspects, wherein the character recognition processing unit calculates a score representing accuracy of character recognition, and the output unit and outputting the specified character string candidate as an estimated character string candidate when the score is less than a predetermined threshold.

An information processing device according to an eighth aspect is the information processing device according to any one of the third to fifth aspects, wherein the character recognition processing unit calculates a score representing accuracy of character recognition, and the output unit and outputting the plurality of specified character string candidates as estimated character string candidates when the score is equal to or greater than a predetermined threshold value and the number of the specified character string candidates is plural.

An information processing device according to a ninth aspect is the information processing device according to any one of the third to fifth aspects, wherein a required character string is a necessary character string from character strings recognized by the character recognition processing unit. and the generation unit generates a misrecognized character string candidate, which is a character string candidate of the same type as the necessary character string extracted by the necessary character string extraction unit.

The information processing apparatus according to a tenth aspect is the information processing apparatus according to any one of the first to fifth aspects, wherein the output unit includes a character string candidate corrected by the correction unit that does not exist in the reference list. In this case, out of the character strings included in the reference list, character strings close to the character string candidates corrected by the correction unit are output as estimated character string candidates.

An information processing apparatus according to an eleventh aspect is the information processing apparatus according to the tenth aspect, wherein the output unit generates a character string that is different from the character string candidate corrected by the correction unit, out of the character strings included in the reference list. Output the character string with the smallest number as the estimated character string candidate.

An information processing program according to a twelfth aspect is an information processing program for causing a computer to function as each part of the information processing apparatus according to any one of the first to eleventh aspects.

According to the first and twelfth aspects, compared to the case of specifying a character string included in an image from a character string recognized from the image by character recognition processing without using a predetermined reference list, the image can be obtained with high accuracy. has the effect of being able to identify the character string contained in .

According to the second aspect, compared to the case of extracting a character string candidate without using a modified extraction pattern represented by a modified regular expression obtained by modifying a regular expression using an erroneously recognized pattern, the character string that should be extracted It is possible to extract a character string in which some of the characters are erroneously recognized.

According to the third aspect, there is an effect that a character string included in an image can be specified with high accuracy as compared with the case where no erroneously recognized character string candidate is generated.

According to the fourth aspect, it is possible to specify a character string included in an image with high accuracy as compared with the case of generating incorrectly recognized character string candidates without using an incorrectly recognized pattern specified by the user. have

According to the fifth aspect, it is possible to shorten the time required to specify a character string included in an image, compared to the case where misrecognized character string candidates are generated without excluding character string candidates represented by extraction patterns. has the effect of being able to

According to the sixth aspect, there is an effect that the reliability of output of the specified character string candidate is increased compared to the case where the specified character string candidate is output regardless of the score representing the accuracy of character recognition. have

According to the seventh aspect, the reliability of being output as an estimated character string candidate is higher than when the specified character string candidate is output as an estimated character string candidate regardless of the score representing the character recognition accuracy. It has the effect of increasing

According to the eighth aspect, output as a plurality of estimated character string candidates compared to outputting a plurality of specified character string candidates as estimated character string candidates irrespective of the score representing character recognition accuracy has the effect of increasing the reliability of

According to the ninth aspect, it is possible to appropriately generate misrecognized character string candidates compared to generating misrecognized character string candidates without extracting necessary character strings from character-recognized character strings. has the effect of

According to the tenth aspect, there is an effect that it is possible to grasp the character strings that may be included in the image, compared to the case where nothing is output when no character string candidate exists in the reference list.

According to the eleventh aspect, the reliability of the estimated character string candidate is increased compared to the case of specifying the estimated character string candidate regardless of the number of different characters between the character string and the character string candidate included in the reference list. has the effect of

1 is a configuration diagram of an image forming apparatus; FIG. 3 is a functional block diagram of a CPU according to the first embodiment; FIG. 4 is a flowchart of information processing according to the first embodiment; It is a figure which shows an example of the image which read drawing. FIG. 10 is a diagram for explaining a region to be re-recognized; It is a figure which shows an example of misrecognition character information. It is a figure which shows an example of misrecognition pattern information. FIG. 10 is a diagram showing an example of a reference list; FIG. FIG. 10 is a diagram showing an example of a reference list; FIG. FIG. 10 is a diagram showing an example of a reference list; FIG. FIG. 10 is a diagram showing an example of a reference list; FIG. FIG. 7 is a functional block diagram of a CPU according to the second embodiment; FIG. 9 is a flowchart of information processing according to the second embodiment; FIG. 10 is a diagram showing an example of a reference list; FIG. FIG. 10 is a diagram showing an example of a reference list; FIG.

Embodiments for carrying out the present invention will be described in detail below with reference to the drawings.

(First embodiment)

FIG. 1 is a configuration diagram of an image forming apparatus 1 according to this embodiment. As shown in FIG. 1, the image forming apparatus 1 includes a scanner section 2, an image forming section 3, an operation section 4, a display section 5, a communication section 6, a storage section 7, and an information processing device . Note that FIG. 1 omits illustration of a supply mechanism for supplying a recording medium such as paper, a conveying mechanism for conveying the recording medium, an ejection mechanism for ejecting the recording medium, and the like.

The image forming apparatus 1 functions as a multifunction device having multiple functions such as a scanning function for reading an original, a copying function for copying an original, a printing function for printing print data, and a facsimile transmission function for facsimile transmission of an original.

The scanner unit 2 optically reads a recording medium such as paper on which an image is formed, generates a read image, and outputs the read image to the information processing apparatus 10 .

The image forming unit 3 has a function of forming an image on a recording medium such as paper by, for example, an electrophotographic method or an inkjet recording method, and forms an image on the recording medium according to instructions from the information processing device 10 .

The operation unit 4 includes operation keys for receiving various operations.

The display unit 5 is composed of a liquid crystal display or the like, and is composed of, for example, a touch panel that accepts various operations by touching the screen with a finger or the like.

The communication unit 6 is an interface for performing data communication with an external device or the like.

The storage unit 7 is configured by a non-volatile storage device such as a hard disk, and stores information processing programs and the like, which will be described later.

The information processing apparatus 10 includes a CPU (Central Processing Unit) 10A, a ROM (Read Only Memory) 10B, a RAM (Random Access Memory) 10C, a nonvolatile memory 10D, and an input/output interface (I/O) 10E. A CPU 10A, a ROM 10B, a RAM 10C, a nonvolatile memory 10D, and an I/O 10E are connected via a bus 10F. The scanner section 2, the image forming section 3, the operation section 4, the display section 5, the communication section 6, and the storage section 7 are connected to the I/O 10E. CPU 10A reads and executes an information processing program stored in storage unit 7 .

The image forming apparatus 1 has a drawing information extracting function in addition to the basic functions of a multifunction machine such as the above-described copying function. The drawing information extraction function is a function that reads drawings used in the manufacturing industry, equipment/construction industry, etc., and extracts specified character strings described in the drawings from the read drawing images as drawing information. Designated character strings described in drawings include basic attributes including product number, product name, plate, etc., various information such as specification information representing specifications of the product shown in the drawing, material information, design standard number, etc. Contains a string representing The drawing information extracted from the drawing is printed on a recording medium such as paper, stored as a data file in the storage unit 7, transmitted to an external device via the communication unit 6, or displayed on the display unit 5. .

Examples of cases where the drawing information extraction function is used include the following cases. For example, if the character strings described in the drawings are based on a certain technical standard, and the technical standards are updated and it is necessary to correct the character strings described in the drawings, it is described in the old technical standard For example, when searching for a drawing containing a specified character string.

The functional configuration of the CPU 10A when the image forming apparatus 1 executes the drawing information extraction function will be described below.

As shown in FIG. 2, the CPU 10A functionally includes an OCR processing section 20, an acquisition section 22, an extraction section 24, a correction section 26, and an output section .

The OCR processing unit 20 performs character recognition processing, that is, OCR (Optical Character Recognition) processing on the image read from the document by the scanner unit 2 , and outputs character strings included in the read image to the acquisition unit 22 . Note that the OCR processing executed by the OCR processing unit 20 is, for example, rule-based OCR processing. That is, all character strings included in the read image are extracted based on a predetermined rule.

The acquisition unit 22 acquires character strings recognized from the image by the OCR processing of the OCR processing unit 20 . Acquisition unit 22 outputs the acquired character string to correction unit 26 .

The extraction unit 24 extracts character string candidates from the character strings acquired by the acquisition unit 22 using extraction patterns represented by regular expressions. Character strings recognized by OCR processing by the OCR processing unit 20 often include erroneously recognized characters. Therefore, the extraction unit 24 extracts the character string candidate using the extraction pattern represented by the modified regular expression corrected using the erroneous recognition pattern that is erroneously recognized in the OCR processing of the OCR processing unit 20 .

The correcting unit 26 corrects the character string candidate extracted by the extracting unit 24 by using an erroneously recognized pattern that is erroneously recognized by the OCR process.

Specifically, the correction unit 26 includes a re-recognition processing unit 30 and a generation unit 32 .

The re-recognition processing unit 30 re-performs the OCR processing of the image for a specific character. The re-recognition processing unit 30 executes OCR processing of the image using machine learning such as a neural network. Note that the re-recognition processing unit 30 is an example of a character recognition processing unit.

The generation unit 32 generates misrecognized character string candidates that are character string candidates of the same type as the character string candidates recognized by the re-recognition processing unit 30 .

Here, the erroneously recognized character string candidate is a character string candidate recognized by the re-recognition processing unit 30, which has the same type as the recognized character and is difficult to recognize, or a character string candidate that is difficult to recognize by the re-recognition processing unit 30. It is a recognized character string, and it is a character string candidate with a high possibility that the recognized characters are incorrect. In the following, it is assumed that the character string recognized by the re-recognition processing unit 30 is also included in the erroneously recognized character string candidates.

The output unit 28 uses a predetermined reference list to output character string candidates specified from the character string candidates corrected by the correction unit 26, that is, the incorrectly recognized character string candidates generated by the generation unit 32.

Next, operation of the information processing apparatus 10 according to the present embodiment will be described with reference to FIG. As shown in FIG. 2, the information processing program 40 is stored in the storage section 7 . The information processing shown in FIG. 3 is executed by reading and executing the information processing program 40 by the CPU 10A. The information processing shown in FIG. 3 is executed, for example, when the user sets a drawing on the scanner unit 2 and operates the operation unit 4 to instruct execution of drawing information extraction processing.

In step S100, the CPU 10A instructs the scanner unit 2 to read the drawing, and acquires the image of the drawing read by the scanner unit 2. FIG.

In step S102, CPU 10A executes rule-based OCR processing on the image acquired in step S100. As a result, all character strings included in the read image are obtained. Also, the positions of all character strings contained in the read image, that is, the coordinates within the image are acquired. A character string recognized by OCR processing is obtained as a character string in XML format, for example, but the format of the character string is not limited to XML format.

In step S104, CPU 10A extracts character string candidates included in the image acquired in step S100 from the character strings obtained by the OCR processing in step S102. Specifically, character string candidates included in the image are extracted using extraction patterns represented by regular expressions. In this embodiment, for example, as shown in FIG. 2, regular expression information 42 in which regular expressions are defined is stored in the storage unit 7 in advance. Use patterns to extract string candidates. It should be noted that the regular expression information 42 may be obtained from an external device via the communication section 6 without being limited to storing the regular expression information 42 in advance in the storage section 7 .

Here, a regular expression is a method of expressing a set of character strings with one character string. A regular expression corresponds to a character string that can be included in a drawing from which drawing information is to be extracted. That is, the regular expression information 42 pre-stored in the storage unit 7 defines regular expressions corresponding to character strings that can be included in the drawing from which the drawing information is to be extracted.

For example, a character string that can be included in a drawing from which drawing information is to be extracted consists of four characters: the first character is "A", the second character is "B", and the third character is "C". , the fourth character is a character string of any one of 10 types of numerals "0" to "9". In this case, a regular expression of character strings that can be included in a drawing from which drawing information is to be extracted is represented by a single character string “[A][B][C][0-9]”. In this case, the regular expression information 42 defines "[A][B][C][0-9]" as a regular expression. Note that the character string of the extraction pattern represented by the regular expression “[A][B][C][0-9]” is 10 patterns from “ABC0” to “ABC9”.

By the way, the character string recognized by the OCR processing in step S102 often includes erroneously recognized characters. Therefore, in the above example, if the character string candidate is extracted using the extraction pattern represented by the regular expression "[A][B][C][0-9]", the character string that should be extracted A character string in which some characters are incorrectly recognized will not be extracted. Therefore, in step S104, a character string candidate is extracted using an extraction pattern represented by a modified regular expression corrected using an erroneous recognition pattern that is erroneously recognized in OCR processing. That is, in the regular expression information 42, a corrected regular expression corrected using an erroneous recognition pattern that is erroneously recognized in OCR processing is defined.

For example, in the case of the above example, "A" is likely to be erroneously recognized as "H", for example. Also, "B" is likely to be erroneously recognized as, for example, "8" or "day". "C" is likely to be erroneously recognized as "[" or "(". Also, "0" is likely to be erroneously recognized as "o" or "O." A modified regular expression modified using the recognition pattern is "[AH] [B8 days] [C [(] [0oO-9]". By extracting the character string candidates, a character string in which some of the characters of the character string that should be extracted is erroneously recognized is also extracted.

For example, assume that the image obtained in step S100 is the image 50 shown in FIG. The image 50 includes an image representing the character string "ABC0", an image representing the character string "ABC8", and an image representing the character string "XYZ1".

In this case, in the OCR processing in step S102, all character strings included in the image 50 are recognized, so all of the character strings "ABC0", "ABC8", and "XYZ1" are all recognized as some character strings. Recognized. Since at least some characters may be erroneously recognized in the OCR processing in step S102, not all character strings are necessarily recognized correctly. For example, the character string "ABC0" may be erroneously recognized as "ABCo" or "ABCO". Also, the character string "ABC8" may be erroneously recognized as "ABCB". On the other hand, in step S104, character string candidates are extracted using the extraction pattern represented by the modified regular expression. .

In step S106, CPU 10A executes OCR processing (hereinafter referred to as re-recognition processing) using a learning model for the character string candidates extracted in step S104. That is, the re-recognition process is performed only for the character string candidates extracted in step S104, instead of performing the re-recognition process on the entire image acquired in step S100. Note that the target area for re-recognition processing may be set based on the positions of the character string candidates obtained by the rule-based OCR processing in step S102.

For example, as shown in FIG. 5, the re-recognition process is not performed on the entire image 50, but the re-recognition process is performed on a region R1 containing the character string candidate "ABC0" and a region R2 containing the character string candidate "ABC8". to run.

In this embodiment, an example of OCR processing using a learning model is OCR processing using machine learning such as a convolutional neural network (CNN), but is not limited to this.

The learning model is trained on specific characters. Here, the specific characters are characters that can be included in the target drawing for which the drawing information extraction function is to be executed, such as alphabets, numbers, and symbols.

Thus, OCR processing using a learning model trained on specific characters has a higher character string recognition rate than rule-based OCR processing executed in step S102.

For example, assume that the character string "ABC0" included in the image 50 is erroneously recognized as "ABCo" in the OCR processing in step S102, and the character string "ABC8" is erroneously recognized as "ABCB". Even in such a case, as a result of performing the re-recognition processing on the regions R1 and R2 shown in FIG. 5 in the re-recognition processing in step S106, "ABC0" and "ABC8" may be correctly recognized, respectively.

In the re-recognition process executed in step S106, a score representing the accuracy of character recognition is calculated. The higher the score, the higher the accuracy of character recognition, and the lower the score, the lower the accuracy of character recognition. For example, the score is a value in the range of 0 to 1, and the higher the value, the higher the accuracy of character recognition.

The processing after step S108 below is executed for each character string recognized in the re-recognition processing.

In step S108, CPU 10A generates a misrecognized character string candidate, which is a character string candidate of the same type as the character string recognized by the re-recognition process in step S106.

Specifically, as shown in FIG. 6, misrecognized character information 44 in which misrecognized characters are defined is stored in advance in the storage unit 7, and misrecognized character string candidates are identified using this misrecognized character information 44. to generate The erroneously recognized character information 44 defines erroneously recognized patterns to be erroneously recognized in the OCR processing of step S102. In the example of FIG. 6, "H" is defined which is likely to be erroneously recognized as opposed to "A". Also, "8", which is likely to be erroneously recognized with respect to "B", is defined. Also, "o" and "O" which are likely to be erroneously recognized with respect to "0" are defined.

As a result, for example, when the character string recognized by the re-recognition process in step S106 is "ABC0", "HBC0", "A8C0", "ABCo", "ABCO", "H8C0", "H8Co", " H8CO” and the like are generated as erroneously recognized character string candidates.

In step S110, CPU 10A adds the character string generated using the misrecognition pattern specified by the user to the misrecognition character string candidates. Here, the erroneous recognition pattern specified by the user is, for example, an erroneous recognition pattern set according to the font of characters included in the drawing. For example, typical examples of characters that are likely to be misrecognized as "0" are "o" and "O", but depending on the font used in the drawing, characters other than "o" and "O" may be used. In some cases, it is easy to be misrecognized as other characters. In this case, for example, as shown in FIG. 7, misrecognition pattern information 46 defining a user-specified misrecognition pattern is stored in advance in the storage unit 7, and a character string generated using this misrecognition pattern information 46 is generated. Add to misrecognized string candidates.

In the example of the erroneous recognition pattern information 46 in FIG. 7, "day" is set as a character likely to be erroneously recognized for "B", and "D" is set as a character likely to be erroneously recognized for "0". For example, if the character string recognized by the re-recognition process in step S106 is "ABC0", "A day C0" and "ABCD" are added as erroneously recognized character string candidates.

Of the erroneously recognized character string candidates generated in steps S108 and S110, the erroneously recognized character string candidates other than character strings represented by regular expressions are unnecessary for specifying character string candidates included in the image. be.

Therefore, in step S112, the CPU 10A excludes, from among the erroneously recognized character string candidates generated in steps S108 and S110, erroneously recognized character string candidates other than the character string of the extraction pattern represented by the regular expression.

For example, when the regular expression is "[A][B][C][0-9]" in the above example, the misrecognized character string candidates generated in steps S108 and S110 are "HBC0" and "A8C0". , "ABCo", "ABCO", "H8C0", "H8Co", "H8CO", "A day C0", and "ABCD". Since the first three characters of a character string represented by a regular expression are alphabetic characters, and the fourth character is a number, among the above misrecognized character string candidates, "ABCo" whose fourth character is an alphabetic character, "ABCO", "H8Co", "H8CO", "ABCD", and "A day C0" whose second character is a Chinese character are excluded from the erroneously recognized character string candidates.

In step S114, CPU 10A identifies a character string candidate from the erroneously recognized character string candidates. For example, as shown in FIG. 8, character string candidates are identified using a reference list 48 pre-stored in the storage unit 7 . The reference list 48 is a list of character strings that can be included in the drawing from which drawing information is to be extracted. As shown in FIG. 8, the reference list 48 contains, by way of example, five character strings "ABC0", "ABC1", "ABC2", "ABC5" and "ABC8".

Here, among the erroneously recognized character string candidates, character strings included in the reference list are specified as character string candidates. On the other hand, if the reference list does not contain any misrecognized character string candidates, no character string candidates are identified.

In step S116, CPU 10A determines whether or not a character string candidate has been identified in step S114. Then, if a character string candidate is specified, that is, if a misrecognized character string candidate exists in the reference list, the process proceeds to step S118. On the other hand, if no character string candidate is specified, that is, if there is no erroneously recognized character string candidate in the reference list, the process proceeds to step S124.

In step S118, CPU 10A determines whether or not the score calculated in the re-recognition process in step S106 is equal to or greater than a predetermined threshold. Note that the threshold is set to a value at which, if the score is equal to or higher than the threshold, the reliability of the recognition result of OCR processing is equal to or higher than the allowable level. For example, it is set to a value that is 90% or more of the maximum value that the score can take. Specifically, when the range of values that the score can take is 0 to 1, the threshold is set to 0.95 as an example.

Then, when the score calculated in the re-recognition process of step S106 is equal to or higher than a predetermined threshold value, that is, when the reliability of the recognition result of the re-recognition process of step S106 is equal to or higher than the allowable level, the process proceeds to step S119. On the other hand, if the score calculated in the re-recognition process of step S106 is less than the predetermined threshold, that is, if the reliability of the recognition result of the re-recognition process of step S106 is less than the allowable level, the process proceeds to step S122.

In step S119, it is determined whether or not there is one character string candidate identified in step S114. If there is one specified character string candidate, the process proceeds to step S120, and if there are a plurality of specified character string candidates, the process proceeds to step S122.

In step S120, CPU 10A outputs one character string candidate specified in step S114. That is, the specified character string candidate is printed on a recording medium such as paper, stored as a data file in the storage unit 7, transmitted to an external device via the communication unit 6, or displayed on the display unit 5. .

On the other hand, step S122 is executed when it is determined in step S118 that the reliability of the recognition result of the re-recognition process is below the allowable level or when a plurality of character string candidates are identified in step S119. Therefore, the character string candidate specified in step S114 is output as an estimated character string candidate, which is a character string candidate that is estimated to be included in the drawing.

Further, step S124 is executed when there is no erroneously recognized character string candidate in the reference list. Therefore, the reference list is searched for a character string close to the erroneously recognized character string candidate. Specifically, the reference list is searched for a character string having the smallest number of characters different from the misrecognized character string candidate as a character string close to the misrecognized character string candidate. Then, the retrieved character string is output as an estimated character string candidate.

In step S126, it is determined whether or not the processing of steps S108 to S124 has been executed for all the character strings recognized in step S106. Then, when the processing of steps S108 to S124 has been executed for all the character strings recognized in step S106, this routine ends. On the other hand, if the processing of steps S108 to S124 has not been executed for all the character strings recognized in step S106, the process proceeds to step S108, and the processing of steps S108 to S124 is continued until the processing of all character strings is completed. repeat.

As described above, in the present embodiment, character string candidates are extracted from character strings recognized from an image by OCR processing using extraction patterns represented by regular expressions. Then, the character string candidate is corrected using the erroneously recognized pattern that is erroneously recognized by the OCR processing, and the character string candidate specified from the corrected character string candidates is output using a predetermined reference list.

In the first embodiment, in step S110 of FIG. 3, the character string generated using the misrecognition pattern information 46 defining the misrecognition pattern specified by the user is added to the misrecognition character string candidates. The processing of S110 may be omitted.

Next, a specific example of identifying character string candidates will be described.

・Specific example 1

In example 1, the regular expression is [F][X][1][0][A-Z][A-Z][0-9][0-9][0-9], and the actual character string contained in the drawing is "FX10OG106" and the reference list contains 9 strings as shown in FIG.

When OCR processing was executed under the above conditions, the recognized character string candidate was "FXIOO6IO6". That is, the third character "1" is erroneously recognized as "I", the fourth character "0" as "O", and the sixth to eighth characters "G10" as "6IO".

Next, the result of the re-recognition processing is "FX100G106", and the fifth character "O" is erroneously recognized as "0". Note that the score of the re-recognition process was equal to or higher than the threshold. In addition, "FX1o0G106", "FX10OG106", "FX10oG106", etc. were generated as misrecognized character string candidates. Note that in Specific Example 1, the process of further generating misrecognition character string candidates using user-specified misrecognition pattern information was not executed.

If unnecessary character string candidates are excluded from the incorrectly recognized character string candidates using regular expressions, only "FX10OG106" remains. This misrecognized character string candidate is included in the reference list in FIG. Therefore, "FX10OG106" is identified and output as a character string candidate described in the drawing. This string exactly matches the actual string contained in the drawing.

・Specific example 2

In specific example 2, the regular expression, the character strings actually included in the drawing, the reference list, and the result of OCR processing are the same as in specific example 1. FIG.

The result of the re-recognition processing is "FX1006106", in which the fifth character "O" is erroneously recognized as "0" and the sixth character "G" is erroneously recognized as "6". Note that the score of the re-recognition process was equal to or higher than the threshold.

In addition, "FX1O06106", "FX1oO6106", "FX10O606", "FX10o6106", etc. were generated as erroneously recognized character string candidates.

Further, in Specific Example 2, a character string generated using misrecognition pattern information specified by the user is added to the misrecognition character string candidates. In the user-specified recognition error pattern information, "6:G" is defined as the recognition error pattern. Therefore, "FX10OG106" etc. was generated and added as an erroneously recognized character string candidate.

Of the character strings recognized in the re-recognition process and the erroneously recognized character string candidates, if unnecessary erroneously recognized character string candidates are excluded using regular expressions, only "FX10OG106" remains. This misrecognized character string candidate is included in the reference list in FIG. Therefore, "FX10OG106" is identified and output as a character string candidate described in the drawing. This string exactly matches the actual string contained in the drawing.

・Specific example 3

Specific example 3 is the same as specific example 2 except that the score of the re-recognition process is less than the threshold.

In Concrete Example 3, as in Concrete Example 2, "FX10OG106", which completely matches the actual character string included in the drawing, is specified as a character string candidate described in the drawing, but the re-recognition processing score is Reliability is low because it is below the threshold. Therefore, the character string candidate "FX10OG106" is output as an estimated character string candidate, which is a character string candidate presumed to be included in the drawing.

・Specific example 4

Specific example 4 is the same as specific example 1 except that the reference list includes 10 character strings, as shown in FIG.

In Concrete Example 4, "FX10OG106" is specified as a character string candidate as in Concrete Example 1, but this character string is not included in the reference list of FIG. For this reason, among the character strings included in the reference list shown in FIG. , “FX10OG102”, “FX10OG103”, “FX10OG104”, and “FX10OG105” are output as estimated character string candidates.

・Specific example 5

In example 5, the regular expression is [F][X][1][0][A-Z][0-9,A-Z][0-9][0-9][0-9], which is included in the drawing The actual string is "FX10OG104" and the reference list contains 8 strings as shown in FIG.

When OCR processing was performed under the above conditions, the recognized character string was "FXIOO6IO4". That is, the 3rd and 4th characters "10" are erroneously recognized as "IO", and the 6th to 9th characters "G104" are erroneously recognized as "6IO4".

Next, the result of the re-recognition processing is "FX1006104", and the 5th and 6th characters "OG" are erroneously recognized as "06". Note that the score of the re-recognition process was equal to or higher than the threshold. In addition, "FX1O06104", "FX1o06104", "FX10O6104", "FX10o6104", etc. were generated as misrecognized character string candidates.

In addition, character strings generated using misrecognition pattern information specified by the user are added to misrecognition character string candidates. In the user-specified recognition error pattern information, "6:G" is defined as the recognition error pattern. For this reason, "FX10OG104" etc. was generated and added as an erroneously recognized character string candidate.

When unnecessary misrecognized character string candidates are excluded from the misrecognized character string candidates using regular expressions, "FX10OG104" and "FX10O6104" remain. Since these character string candidates are all included in the reference list shown in FIG. 11, they cannot be narrowed down to one character string candidate. Therefore, these two character string candidates are output as estimated character string candidates.

(Second embodiment)

Next, a second embodiment will be described. In addition, the same code|symbol is attached|subjected to the same part as 1st Embodiment, and detailed description is abbreviate|omitted. In the second embodiment, the case where the information to be extracted from the drawing is a character string representing the material of the plastic part will be described.

JIS (Japanese Industrial Standards) stipulates the method of indicating the material of plastic parts. For example, the designation of a part such as a single polymer is to enclose the abbreviation of the material specified in JIS K 6899-1 between the boundary characters ">" and "<". For example, if the material is acrylonitrile-butadiene-styrene and the abbreviation is ABS, it is written as ">ABS<".

In addition to the material abbreviations, "-", "+", "(", ")", etc. are used as delimiters, and mass fractions or material content rates are used to indicate the materials of plastic parts. A number representing is used. For example, the notation ">PA66-(GF25+MD15)<" denotes polyamide 66 containing a mixture of 25% by weight glass fiber (GF) and 15% by weight mineral powder (MD).

Thus, a string between boundary characters may contain multiple material abbreviations, but only the material abbreviations are required and no delimiters are required. For this reason, in the second embodiment, when a character string sandwiched between boundary characters includes a delimiter, the character string is divided by the delimiter, and a character string composed of material abbreviations and numbers is converted into a required character string. Extract as

FIG. 12 shows a functional block diagram of the CPU 10A according to the second embodiment. The functional block diagram shown in FIG. 12 differs from the functional block diagram shown in FIG. 2 in that a required character string extraction unit 31 is provided.

The required character string extraction unit 31 extracts a required character string from the character strings recognized by the re-recognition processing by the re-recognition processing unit 30 . Specifically, it is determined whether or not the character string recognized by the re-recognition process includes a boundary character. Then, if the character string recognized by the re-recognition process contains boundary characters, the character string sandwiched between the boundary characters is divided by the delimiter, and the character string composed of the material abbreviations and numbers is Extract as a required string.

Next, operation of the information processing apparatus 10 according to the present embodiment will be described with reference to FIG. The information processing shown in FIG. 10 differs from the information processing shown in FIG. 3 in that the processing of step S107 is added and the processing of step S114 is slightly different.

In step S107, processing for extracting necessary character strings is performed for each of the character strings recognized by the re-recognition processing in step S106. That is, the character strings sandwiched between the boundary characters “>” and “<” are extracted from the character strings recognized by the re-recognition process. Among the character strings sandwiched between boundary characters ">" and "<", character strings that do not contain delimiters such as "-", "+", "(", and ")" Extract the string excluding characters as the required string. For example, if the character string recognized in the re-recognition process is ">ABS<", then "ABS" excluding the boundary characters is extracted as the required character string.

On the other hand, if a character string sandwiched between boundary characters “>” and “<” contains delimiters such as “-”, “+”, “(”, “)”, The character string is divided and character strings composed of material abbreviations and numbers are extracted as required character strings. For example, if the character string recognized in the re-recognition process is ">PA66-(GF25+MD15)<", extract "PA66", "GF25", and "MD15" excluding boundary characters and delimiters as required character strings. do. Therefore, the processes of steps S108 to S126 are executed for each required character string. That is, generation of erroneously recognized character string candidates in step S108 is also generated for each required character string.

Further, in step S114, for example, using a reference list 48A as shown in FIG. 14, character string candidates are identified from the misrecognized character string candidates including the necessary character string. As shown in FIG. 14, the reference list 48A is a list in which only material abbreviations are defined, but misrecognized character string candidates including necessary character strings are character strings including material abbreviations and numbers. Therefore, if there is a character string partially matching the character string defined in the reference list 48A and the abbreviation of the material among the erroneously recognized character string candidates including the necessary character string, the character string is included in the image. Identifies string candidates that can be

For example, if the required character string is "PA66", it matches "PA" defined in the reference list 48A, so "PA" is identified as a character string candidate included in the image.

Next, a specific example of identifying character string candidates will be described.

・Specific example 1

In example 1, the regular expression is [>][][][][][][][][][][][][][][][][][][][][ ][][][][<] , the actual string contained in the drawing is ">PA66-(GF25+MD15)<" and the reference list contains 11 strings as shown in Figure 15 is

When OCR processing was performed under the above conditions, the recognized character string candidate was ">PAGG-(GF2S+MDI5)<". That is, the 4th and 5th characters "66" are erroneously recognized as "GG", the 10th character "5" as "S", and the 14th character "1" as "I".

Next, the result of the re-recognition process was ">PA66-(GF25+MD15)<", and the score of the re-recognition process was above the threshold. Also, "PA66", "GF25", and "MD15" were extracted by the necessary character extraction processing. These strings are included in the reference list shown in FIG. 15, which strings represent material abbreviations. Therefore, "PA", "GF", and "MD" are identified and output as character string candidates described in the drawing.

・Specific example 2

Specific example 2 is the same as specific example 1 except that the score of the re-recognition process is less than the threshold. In this case, since the reliability of the recognition result of the re-recognition process is low, "PA", "GF", and "MD" are output as estimated character string candidates.

Although the present invention has been described above using each embodiment, the present invention is not limited to the scope described in each embodiment. Various changes or improvements can be made to each embodiment without departing from the gist of the present invention, and forms with such changes or improvements are also included in the technical scope of the present invention.

For example, in each of the above-described embodiments, the case where the information processing apparatus is applied to a multi-function peripheral has been described as an example, but it may be applied to a smart phone, a tablet terminal, or the like. In this case, the drawing may be photographed with a camera mounted on a smartphone or tablet terminal, and the processing of FIG. 3 or FIG. 10 may be performed on the photographed image.

Further, for example, the information processing shown in FIGS. 3 and 13 may be realized by hardware such as ASIC (Application Specific Integrated Circuit). In this case, the speed of processing can be increased as compared with the case of realizing by software.

Further, in each embodiment, a form in which the information processing program is installed in the storage unit 7 has been described, but the present invention is not limited to this. The information processing program according to this embodiment may be provided in a form recorded on a computer-readable storage medium. For example, the information processing program according to the present embodiment is recorded in an optical disk such as CD (Compact Disc)-ROM and DVD (Digital Versatile Disc)-ROM, or a semiconductor such as USB (Universal Serial Bus) memory and memory card It may be provided in a form recorded in memory. Also, the information processing program according to the present embodiment may be acquired from an external device via a communication line connected to the communication section 6 .

1 Image forming device 2 Scanner unit 3 Image forming unit 4 Operation unit 5 Display unit 6 Communication unit 7 Storage unit 10 Information processing device 20 OCR processing unit 22 Acquisition unit 24 Extraction unit 26 Correction unit 28 Output unit 30 Re-recognition processing unit 31 Necessary Character string extraction unit 32 Generation unit 40 Information processing program 42 Regular expression information 44 Misrecognition character information 46 Misrecognition pattern information 48, 48A Reference list 50 Image

Claims (11)

an acquisition unit that acquires a character string recognized from an image by character recognition processing;
an extraction unit that extracts a character string candidate from the character string acquired by the acquisition unit using an extraction pattern represented by a regular expression;
a correction unit that corrects the character string candidate using an erroneous recognition pattern that is erroneously recognized by the character recognition process;
an output unit configured to output character string candidates identified from the character string candidates corrected by the correction unit using a predetermined reference list;
with
The acquisition unit acquires the image,
a character recognition processing unit in which the correction unit performs character recognition processing again on only the image of the character string candidate extracted by the extraction unit using a learning model learned about a specific character;
a generating unit that generates misrecognized character string candidates that are character string candidates of the same type as the character string candidates recognized by the character recognition processing unit;
including
The output unit outputs a character string candidate identified from the misrecognized character string candidates.
Information processing equipment. The information processing apparatus according to claim 1, wherein the extraction unit extracts the character string candidate using a modified extraction pattern represented by a modified regular expression obtained by modifying the regular expression using the misrecognition pattern. 3. The information processing apparatus according to claim 1 , wherein the generation unit adds a character string generated using an erroneous recognition pattern specified by a user to the erroneously recognized character string candidates. The information processing apparatus according to any one of claims 1 to 3, wherein the generation unit generates the misrecognized character string candidates by excluding character string candidates represented by the extraction patterns. The character recognition processing unit calculates a score representing accuracy of character recognition,
The information processing apparatus according to any one of claims 1 to 4 , wherein the output unit outputs the identified character string candidate when the score is equal to or greater than a predetermined threshold. The character recognition processing unit calculates a score representing accuracy of character recognition,
The information processing apparatus according to any one of claims 1 to 4 , wherein the output unit outputs the specified character string candidate as an estimated character string candidate when the score is less than a predetermined threshold. The character recognition processing unit calculates a score representing accuracy of character recognition,
5. The output unit outputs a plurality of the specified character string candidates as estimated character string candidates when the score is equal to or greater than a predetermined threshold and the number of the specified character string candidates is plural . 1. The information processing apparatus according to 1. an acquisition unit that acquires a character string recognized from an image by character recognition processing;
an extraction unit that extracts a character string candidate from the character string acquired by the acquisition unit using an extraction pattern represented by a regular expression;
a correction unit that corrects the character string candidate using an erroneous recognition pattern that is erroneously recognized by the character recognition process;
an output unit configured to output character string candidates identified from the character string candidates corrected by the correction unit using a predetermined reference list;
with
The acquisition unit acquires the image,
The correction unit includes a character recognition processing unit that performs character recognition processing on the image with a specific character as a target;
a required character string extraction unit for extracting a required character string, which is a required character string, from the character string recognized by the character recognition processing unit;
a generating unit that generates misrecognized character string candidates that are character string candidates of the same type as the necessary character string extracted by the necessary character string extracting unit;
including
The output unit outputs a character string candidate identified from the misrecognized character string candidates.
Information processing equipment. If the reference list does not contain the character string candidate corrected by the correction unit, the output unit selects a character string similar to the character string candidate corrected by the correction unit from among the character strings included in the reference list. The information processing apparatus according to any one of claims 1 to 4 , wherein the estimated character string candidate is output. 10. The information according to claim 9 , wherein the output unit outputs, among the character strings included in the reference list, a character string having the smallest number of characters different from the character string candidates corrected by the correction unit as an estimated character string candidate. processing equipment. An information processing program for causing a computer to function as each part of the information processing apparatus according to any one of claims 1 to 10 .

Images