JP7322468B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

There is a technique of recognizing characters from image data of a document or the like using means such as OCR (Optical Character Recognition) and detecting errors in the document or the like from the recognized characters. For example, in Patent Document 1, an image of a document to be reviewed is converted into text data by OCR processing, and from the converted text data, erroneous notation in the manuscript and regular notation that corrects the erroneous notation are detected, and erroneous notation and An image processing apparatus and the like that generate a proofreading result image by synthesizing a regular notation with a document image have been disclosed.

JP 2018-67159 A

However, the invention according to Patent Document 1 only detects errors in the original document, and does not detect errors including erroneous recognition when recognizing characters from image data.

An object of one aspect of the present invention is to provide an information processing apparatus or the like capable of detecting or correcting an error in characters recognized from image data.

An information processing apparatus according to one aspect includes an acquisition unit that acquires image data including a character image, a recognition unit that recognizes characters and coordinates corresponding to the character image, and identifies each element that constitutes the image data. an identifying unit that identifies the structure of each element of the image data including the character by referring to a structured table that defines rules for processing or using a structure identifier that identifies each element that constitutes the image data; a character processing unit that detects an error in the character or corrects the erroneous character based on the identified structure , wherein the character processing unit detects a correct sentence among the sentences included in the sentence group and the correct sentence; The method is characterized by detecting or correcting an error in a character by using a classifier that has already learned a sentence corresponding to a correct sentence and including an erroneous character.

In one aspect, character errors recognized from image data can be detected or corrected.

1 is a schematic diagram showing a configuration example of a document processing system; FIG. It is a block diagram which shows the structural example of a server. FIG. 10 is an explanatory diagram relating to character recognition processing; FIG. 4 is an explanatory diagram showing a comparative example of structured data and unstructured data; FIG. 10 is an explanatory diagram relating to structuring processing; FIG. 4 is an explanatory diagram of a forward order model and a reverse order model; FIG. 4 is an explanatory diagram relating to an error correction model; FIG. 10 is a flow chart showing a procedure of generation processing of a forward-order model and a reverse-order model; FIG. 4 is a flow chart showing the procedure of generation processing of an error correction model; 4 is a flowchart showing the procedure of error detection processing; FIG. 8 is a block diagram showing a configuration example of a server according to Embodiment 2; FIG. 4 is an explanatory diagram of a structured model; 4 is a flow chart showing the procedure of a structured model generation process; It is a functional block diagram which shows operation|movement of the server of the form mentioned above.

Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments.
(Embodiment 1)
FIG. 1 is a schematic diagram showing a configuration example of a document processing system. In this embodiment, a document processing system that recognizes characters in a document from a document image (image data), detects errors in the recognized characters, and corrects the characters to correct characters will be described. The document processing system includes an information processing device 1 and a terminal 2 . Each device is interconnected for communication via a network N such as the Internet.

In this embodiment, the image data to be processed is an image related to a "document". In this system, characters are recognized from image data including character images, and errors in the recognized characters are detected or detected. The image data to be processed is not limited to document images as long as it can be corrected. For example, an illustration, a photograph, or the like may be processed. Further, the character image included in the image data is not limited to printed characters, and may be, for example, handwritten characters.

The information processing device 1 is a device that performs various types of information processing and transmission/reception of information, and is, for example, a server device, a personal computer, or the like. In the present embodiment, the information processing device 1 is assumed to be a server device, and is replaced with the server 1 below for the sake of simplicity. The server 1 recognizes characters in the document by means of OCR or the like from a document image (for example, a PDF file obtained by scanning the document with a scanner) to which text data (character code, etc.) is not attached, and corrects errors in the recognized characters. Detect and correct.

The terminal 2 is a client terminal connected to the server 1, such as a personal computer. For example, the server 1 acquires a document image from the terminal 2 via the network N, and performs processing described later on the acquired document image.

In this embodiment, the server 1 on the cloud performs the processing, but a series of processing may be performed by a local device (for example, the terminal 2).

FIG. 2 is a block diagram showing a configuration example of the server 1. As shown in FIG. The server 1 has a control section 11 , a main storage section 12 , a communication section 13 and an auxiliary storage section 14 .
The control unit 11 has an arithmetic processing unit such as one or more CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), etc., and executes the program P stored in the auxiliary storage unit 14. Various information processing, control processing, etc. are performed by reading and executing the data. The main storage unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores data necessary for the control unit 11 to perform arithmetic processing. Remember. The communication unit 13 is a communication module for performing processing related to communication, and transmits and receives information to and from the outside.

The auxiliary storage unit 14 is a nonvolatile storage area such as a hard disk or a large-capacity memory, and stores programs P and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 14 also stores a structured table 141 , a forward order model 142 , a reverse order model 143 , and an error correction model 144 . The structured table 141 is a table that defines rules for converting a document image, which is unstructured data, into structured data. The normal order model 142 is a trained model that has learned the order of character strings appearing in a sentence in the normal order of the sentence. The reverse order model 143 is a trained model that has learned the appearance order of character strings that appear in a sentence in the order opposite to the order in which the sentences are arranged. The error correction model 144 is a trained model that has learned error appearance patterns by learning pairs of correctly written sentences and sentences containing errors.

Incidentally, the auxiliary storage unit 14 may be an external storage device connected to the server 1 . Moreover, the server 1 may be a multicomputer consisting of a plurality of computers, or may be a virtual machine virtually constructed by software.

Moreover, in the present embodiment, the server 1 is not limited to the configuration described above, and may include, for example, an input unit for receiving operation inputs, a display unit for displaying images, and the like. The server 1 also includes a reader for reading a portable storage medium 1a such as a CD (Compact Disk)-ROM, a DVD (Digital Versatile Disc)-ROM, etc., and reads and executes the program P from the portable storage medium 1a. You can do it. Alternatively, the server 1 may read the program P from the semiconductor memory 1b.

FIG. 3 is an explanatory diagram relating to character recognition processing. FIG. 3 conceptually illustrates how characters in a document image and the coordinates of the characters are recognized.
When a document image (image data) is acquired from the terminal 2, the server 1 identifies each character image (image area) in the acquired document image and recognizes characters corresponding to the character image. For example, the server 1 performs OCR processing to convert character images into text data.

In this specification, "text data" means data representing characters corresponding to a character image, and means data necessary for a computer to interpret and display the characters. The text data may be, for example, plain text (character code information), or may be data including information such as font and character size in addition to plain text.

The server 1 converts the character image into text data and specifies the coordinates of the character corresponding to the character image within the document image. The server 1 inserts the text data converted to the specified coordinates, and generates a document file in which the information of each character is added to the document image.

The server 1 performs a process of converting the generated document file into structured data such as an XML file, and identifies the sentence structure. The server 1 extracts text data based on the identified sentence structure and detects errors in the text data.

FIG. 4 is an explanatory diagram showing a comparative example of structured data and unstructured data. FIG. 4 conceptually illustrates structured data and unstructured data for the same document.

If text data is extracted from a document file without conversion to structured data, misrecognition such as typographical errors or omissions may occur due to differences in document layout. For example, as shown in the lower left of Fig. 4, if inappropriate text (in Fig. 4, the document title "Summary of Achievements") is inserted in an inappropriate position, or if line breaks, spaces, etc. are inserted in inappropriate places. may occur.

Therefore, in this embodiment, as shown in the lower right of FIG. 4, the document file, which is unstructured data, is converted into structured data. The server 1 identifies a correct grouping (structure) of text from the structured document file and performs error detection, which will be described later.

FIG. 5 is an explanatory diagram of the structuring process. FIG. 5 conceptually illustrates how a document file is converted into structured data.
For example, the server 1 refers to the structured table 141 that defines rules for specifying sentence structures, and converts the document file into structured data structured for each predetermined basic element that constitutes the document image. The elements of the document are, for example, the title, text, photograph, chart, caption, etc. of the document, but they may be elements obtained by dividing the document image into predetermined units, and the content (division unit) is not particularly limited. The structured table 141 stores a rule that is used as a reference for identifying each element and metadata to be given to each element for each element that constitutes a document image.

For example, the structured table 141 contains information ( rules) are stored. The server 1 refers to the structured table 141 to identify each element from the document file.

The server 1 gives metadata to each identified element. Metadata is tag information that defines each element, such as a tag name tagged to each element or an attribute value stored as information in the tag. For example, as shown in the lower right of FIG. 4, "title" is given as metadata (tag name) to an element corresponding to the title of a document, and "section" is given to an element corresponding to a subtitle.

As described above, the server 1 refers to the structured table 141 that defines rules for identifying each element, identifies each element that constitutes the document file, and assigns metadata to each element. Then, the server 1 generates structured data by structuring the document file, which is unstructured data, as shown in the upper right of FIG. As a result, as shown in the lower right of FIG. 4, the correct character sequence, line feed position, etc. can be recognized for the text data of the document file.

The server 1 extracts text data based on the structured data generated above, and performs error detection to determine whether or not there is an error in the extracted text data. In this embodiment, the server 1 performs error detection using three types of models (classifiers), a forward order model 142, a reverse order model 143, and an error correction model 144, which are generated from a group of sentences (document corpus).

It should be noted that these models may be generated using the entire text data regardless of tags, or a model may be generated for each tag. For example, there is a case of generating a model for the above "title" text data or a model for "section" text data.

FIG. 6 is an explanatory diagram of the forward order model 142 and the reverse order model 143. As shown in FIG. FIG. 6 conceptually illustrates error detection processing using a forward order model 142 (first discriminator) and a reverse order model 143 (second discriminator). The forward order model 142 and the reverse order model 143 will be described with reference to FIG.

The forward-order model 142 and the reverse-order model 143 are respectively trained models (classifiers) that have learned the sequence (appearance order) of character strings that appear in the document corpus, and are, for example, neural networks generated by deep learning. be. Specifically, the forward order model 142 and the reverse order model 143 are created by LSTM (Long Short-Term Memory), which is a type of RNN (Recurrent Neural Network).

Note that the LSTM is an example of the forward order model 142 and the reverse order model 143, and the forward order model 142 and the reverse order model 143 may be trained models related to other algorithms such as deep learning. Also, it may be a learned model related to segmentation of other characters such as N-grams and spaces.

The forward order model 142 and the reverse order model 143 are composed of an input layer for receiving input of data for each character string appearing in a sentence, and an output layer for outputting estimation results of character strings that appear following each character string. , and an intermediate layer (hidden layer) that performs arithmetic processing between the input layer and the output layer. The input layer has a plurality of neurons that receive input of data of each character string appearing in the sentence according to the order of appearance in the sentence. The intermediate layer has a plurality of neurons for performing operations between the input layer and the output layer corresponding to each neuron in the input layer. The neurons in the middle layer are called LSTM blocks, and by performing calculations on the input values at the next time point using the calculation results at the past time point in the middle layer, the time-series data up to the latest time point is converted to the Calculate values. The output layer outputs an output value representing the estimation result of the character string following the character string input to the corresponding neuron of the input layer, based on the operation value output from the intermediate layer.

When generating the forward order model 142, the server 1 divides the sentences of the document corpus into character strings of a predetermined unit, and sequentially inputs each divided character string into the input layer according to the order (normal order) in the sentence. do. A sentence is divided in units of characters, for example, one character at a time, but the present embodiment is not limited to this, and may be divided in units of words, for example. The server 1 sequentially inputs one or more character strings continuing from the beginning of a sentence, and acquires an estimation result of estimating the next character string following the one or more character strings from the output layer. For example, the output layer outputs the estimated next character string and the occurrence probability of the character string as the estimation result, as shown in FIG.

The server 1 compares the estimated character string with the correct character string that actually appears in the sentence, and optimizes various parameters such as the weight between each neuron and the coefficient of the activation function so that both are approximate. I do. The server 1 performs the above processing on each sentence in the document corpus to generate the normal order model 142 .

When generating the reverse order model 143, the server 1 converts the order of each character string obtained by dividing the text of the document corpus into a reverse order by transposing the beginning and end of each character string. The server 1 sequentially inputs each character string into the input layer in the order after conversion, and based on one or more character strings continuing from the end of the original sentence, the character that appears immediately before the one or more character strings Get the result of estimating the columns from the output layer. The server 1 compares the estimated character string with the correct character string, optimizes various parameters so that both are approximate, and generates the reverse order model 143 .

In this way, the server 1 generates a forward order model 142 that learns the sequence of character strings in a sentence in forward order and a reverse order model 143 that learns in reverse order, and performs error detection using both. .

The server 1 extracts text data (sentences) from a structured document file, and divides the text data into predetermined units of character strings (for example, character by character). The server 1 sequentially inputs each divided character string to the normal order model 142 according to the order of arrangement (normal order) in the text data, and acquires the estimation result (occurrence probability) of the character string appearing at each position as an output. The server 1 compares the estimated character string with character strings (characters) recognized from the document image, and detects, for example, a character string whose occurrence probability is equal to or less than a threshold as an error.

The server 1 reverses the order of the character strings obtained by dividing the text data, and sequentially inputs them to the reverse order model 143 according to the order after conversion (reverse order). The server 1 estimates a character string appearing at each position in the text from the reverse order model 143, compares it with the character string recognized from the character image, and determines an error.

FIG. 7 is an explanatory diagram of the error correction model 144. As shown in FIG. FIG. 7 conceptually illustrates the error detection process using the error correction model 144 which has already learned the appearance pattern of errors by learning pairs of correct sentences containing no errors and sentences containing errors. ing.

The error correction model 144 is a trained model generated by deep learning, like the forward order model 142 and the reverse order model 143, and is generated by, for example, sequence-to-sequence. The server 1 performs learning using correct sentences containing no errors and sentences corresponding to the correct sentences that contain errors, and generates an error correction model 144 .

For example, the server 1 performs learning using a document file containing text data as learning data for generating the error correction model 144 .
Text data attached to the document file for learning is used as the correct sentence. For sentences containing errors, text data containing errors, which is different from the original text data obtained as a result of imaging a document file for learning and applying character recognition such as OCR to the image, is used.

As a method of obtaining text data containing errors, first, the server 1 converts a learning document file into image data. The server 1 performs character recognition on the generated image data, converts the image data into text data again, and generates a document file. That is, the server 1 generates a document file from the image data in the same procedure as the processing described with reference to FIGS. 3 and 4. FIG.

A training data set can be obtained by matching the text data (correct text data) in the original document file with the text data (erroneous text data) in the generated document file. Note that the association can be made by referring to the positional relationship of the text data, tags existing in the original data, structuring, or the like.

Next, the server 1 divides the text data in the learning data set into predetermined units of character strings (for example, one character), and sequentially inputs each divided character string to the error correction model 144 according to the arrangement order. The error correction model 144 performs learning by optimizing various parameters so that the output of the erroneous text data of the input learning data set approximates the correct text data.

In this way, the server 1 uses text data obtained by converting character images by OCR or the like and text data attached to a document file for learning to detect and correct errors that occur during conversion to text data. Generate a model that At the time of error detection, the server 1 extracts text data from the document file structured by the processing described in FIG. 5, inputs it to the error correction model 144, and detects and corrects errors in the text data.

As described above, the server 1 uses three types of models, the forward order model 142, the reverse order model 143, and the error correction model 144, to detect text data errors in each model. For example, when the server 1 determines that the same character (character string) is erroneous in the majority (two or more) models among the three types of models, the character is detected as being erroneous. Alternatively, if the server 1 determines an error in any of the models, the determined part may be detected as an error. Alternatively, the server 1 may perform comprehensive error determination based on output values (probability values such as occurrence probability) output from each of the three types of models to detect erroneous characters.

In the present embodiment, three models, the forward order model 142, the reverse order model 143, and the error correction model 144, are used for error detection. error detection may be performed using one or two models. Further, models other than the above three models may be installed, and error detection may be performed with four or more models.

If an error is detected, the server 1 uses the forward order model 142, the reverse order model 143, and/or the error correction model 144 to correct the erroneous characters to correct characters. For example, when the error correction model 144 detects an error, the server 1 converts the erroneous character into a character corrected by the error correction model 144 . Alternatively, if the error correction model 144 is not detected as an error, but is detected as an error in the forward order model 142 and/or the reverse order model 143, the server 1 may Based on the output estimation result, the character detected as an error is converted into the character with the highest occurrence probability. Alternatively, the server 1 may estimate the correct character based on the output values of all three models and correct the text data.

The server 1 converts the text data of the document file into corrected text data and outputs the document file to the terminal 2 . Note that the server 1 may output the document file in the form of structured data (such as an XML file), or may output the unstructured data (image data such as a PDF file). As a result, the server 1 can provide a document file in which errors in the document are corrected, including erroneous recognition during character recognition.

FIG. 8 is a flow chart showing the procedure of generating the forward order model 142 and the reverse order model 143 . Based on FIG. 8, the contents of machine learning processing for generating the forward order model 142 and the reverse order model 143 will be described.
The control unit 11 of the server 1 acquires a group of sentences (document corpus) used to generate the forward order model 142 and the reverse order model 143 (step S11). The control unit 11 divides the acquired text into character strings of predetermined units (step S12).

The control unit 11 performs machine learning to learn the arrangement order of the divided character strings in the sentence order (forward order), and generates the forward order model 142 (step S13). Specifically, the control unit 11 sequentially inputs the divided character strings according to the arrangement order of the original sentence, and based on one or more character strings that appear in order from the beginning, the next character string following the one or more character strings. Get the estimation result (occurrence probability, etc.) of estimating the character string that appears in . The control unit 11 compares the estimated result with the correct character string, optimizes various parameters such as weights used in the calculation of the forward order model 142, and generates the forward order model 142 so that the two approximate each other.

The control unit 11 converts the order of the character strings divided in step S12 into a reverse order in which the beginning and end of the text are interchanged (step S14). The control unit 11 performs machine learning to learn the order of the character strings converted to the reverse order, and generates the reverse order model 143 (step S15). That is, the control unit 11 sequentially inputs each character string in reverse order, and based on one or more character strings that appear in order from the end of the original sentence, the character string that appears immediately before the one or more character strings is obtained as an output. The control unit 11 compares the estimation result with the correct character string, optimizes various parameters, and generates the reverse order model 143 . The control unit 11 ends the series of processes.

FIG. 9 is a flow chart showing the procedure for generating the error correction model 144 . Based on FIG. 9, the contents of the machine learning processing for generating the error correction model 144 will be described.
The control unit 11 of the server 1 acquires a document file group in which text data corresponding to each character image is added to the document image including the character image, which is learning data for generating the error correction model 144. (step S31). The control unit 11 removes the text data from the acquired document file and converts it into a document image (step S32).

The control unit 11 converts each character image in the document image into text data by means such as OCR, and specifies the coordinates of the text data in the document image (step S33). The control unit 11 generates a document file in which the converted text data is inserted (given) at the specified coordinates (step S34).

The control unit 11 refers to the structured table 141 and converts the document file generated in step S33 and the learning document file obtained in step S31 into structured data (step S35). By performing the structuring process, it is possible to obtain the correspondence relationship between each sentence of correct text data and erroneous text data included in each document file. It is also possible to perform learning for each structured tag.

The control unit 11 generates the error correction model 144 using the structured text data of each document file (step S36). Specifically, the control unit 11 inputs the text data (sentence) obtained by structuring the document file generated in step S34 to the error correction model 144, and acquires the text data in which the erroneous characters are corrected as an output. do. The control unit 11 compares the output text data with the text data of the document file for learning acquired in step S31, optimizes various parameters so that the two approximate each other, and generates the error correction model 144. . The control unit 11 ends the series of processes.

FIG. 10 is a flow chart showing the procedure of error detection processing. Based on FIG. 10, the contents of the process of performing character recognition of character images included in a document image and detecting character errors will be described.
The control unit 11 of the server 1 acquires a document image (image data) to be processed (step S51). The control unit 11 converts each character image in the document image into text data by means such as OCR, and specifies the coordinates of the text data in the document image (step S52). The control unit 11 generates a document file in which the converted text data is inserted (given) at the specified coordinates (step S53). The control unit 11 refers to the structured table 141, identifies each element in the generated document file, and converts each element into structured data in which metadata is added (step S54).

The control unit 11 extracts text data from the structured document file, and detects erroneous characters from the extracted text data (step S55). Specifically, the control unit 11 inputs text data (sentences) to each of the forward order model 142, the reverse order model 143, and the error correction model 144, and detects errors using a plurality of models (classifiers). . If an error is detected, the control unit 11 corrects the erroneous character using the forward order model 142, the reverse order model 143, or the error correction model 144 (step S56). The control unit 11 ends the series of processes.

Although OCR is used as the character recognition means in the above description, the server 1 may use means such as ICR (Intelligent Character Recognition) as long as it can recognize characters corresponding to character images. Further, the character recognition means is not limited to optical means, and any means capable of recognizing characters from image data may be used.

In the above description, the document file (image data) is converted into structured data such as an XML file. , conversion to structured data is not required.

As described above, according to the first embodiment, it is possible to detect or correct errors in characters recognized from image data.

Further, according to the first embodiment, by converting a character image into text data by means of OCR or the like, it is possible to process it favorably.

Further, according to Embodiment 1, errors can be detected or corrected with high accuracy by using a trained model (classifier) that has been trained on a group of sentences (document corpus).

Further, according to the first embodiment, by combining a plurality of trained models, errors can be detected or corrected with higher accuracy.

Further, according to the first embodiment, by using a trained model that has learned the order of characters that appear in a sentence, it is possible to suitably detect or correct an error part from the preceding and succeeding text.

Further, according to the first embodiment, by performing error detection using the forward order model 142 and the reverse order model 143, errors can be detected or corrected with higher accuracy.

Further, according to the first embodiment, by using a trained model that has already learned correct sentences and sentences containing errors, it is possible to suitably detect or correct errors in characters recognized from a document image. can.

Further, according to the first embodiment, by using the error correction model 144 generated based on the document image (document file) to which the text data is added and the document image from which the text data is removed, Errors can be detected or corrected better.

Further, according to the first embodiment, by referring to the structured table 141 and converting the image data into structured data, the text structure can be preferably specified.

(Embodiment 2)
In this embodiment, instead of using the structuring table 141, a structuring model 145 obtained by learning a sentence structure through machine learning is used to structure a document image. In addition, the same code|symbol is attached|subjected about the content which overlaps with Embodiment 1, and description is abbreviate|omitted.
FIG. 11 is a block diagram showing a configuration example of the server 1 according to the second embodiment. The auxiliary storage unit 14 of the server 1 according to this embodiment stores a structured model 145 (structure identifier) instead of the structured table 141 . The structured model 145 is a learned model constructed by machine learning, and is a structure identifier for identifying each element in the document image for teacher.

FIG. 12 is an explanatory diagram of the structured model 145. As shown in FIG. It conceptually illustrates how a structured model 145 is generated from a teacher document file (document image), which is unstructured data. Based on FIG. 12, the outline of this embodiment will be described.
As described above, in the present embodiment, the server 1 structures documents using the structured model 145 constructed by machine learning instead of the structured table 141 in which rules are set in advance. For example, the server 1 generates a neural network related to CNN, specifically a neural network related to semantic segmentation, as the structured model 145, and uses it for document structuring processing.

For example, the server 1 learns each element, such as a title and a text, of a document image using a document file for teachers associated with correct values of metadata. For example, as shown in FIG. 12, in a document file for teachers, a tag name corresponding to a correct value of metadata is associated with an area (illustrated by a rectangular frame) corresponding to each element. The server 1 generates the structured model 145 using the teacher data.

The server 1 inputs the document image for teacher to the structured model 145 and acquires the identification result of identifying each element included in the document image as an output. For example, the server 1 acquires the coordinate values of the image area corresponding to each element and the metadata to be given to the elements included in the area as outputs. The server 1 compares the output coordinate values and metadata of the image region with the correct values, and optimizes parameters such as weights used for calculation in the structured model 145 so that the two approximate each other. The server 1 thereby generates the structured model 145 .

When a document image is acquired from the terminal 2, the server 1 structures it using the structured model 145 generated above. Specifically, the server 1 inputs the acquired document image to the structured model 145 and acquires the identification result of identifying each element. The server 1 extracts each element in the document according to the identification result and adds metadata. As a result, the server 1 generates structured data in which the document image is structured. After that, the server 1 performs error detection in the same manner as in the first embodiment, and corrects the erroneous characters.

FIG. 13 is a flow chart showing the procedure for generating the structured model 145 . Based on FIG. 13, the details of the processing for generating the structured model 145 by machine learning will be described.
The control unit 11 of the server 1 generates a structured model 145 for generating a structured model 145. For a teacher document image which is unstructured data, the correct value of the metadata of each element constituting the document is obtained. acquires teacher data associated with (step S201). The control unit 11 generates the structured model 145 using the acquired teacher data (step S202). Specifically, the control unit 11 inputs a document image for teachers into the structured model 145, and identifies an image area corresponding to each element and metadata to be added to the elements included in the area. as output. The control unit 11 compares the obtained identification result with the correct value, optimizes various parameters such as weights so that the two approximate each other, and generates a structured model 145 . The control unit 11 ends the series of processes.

As described above, according to the second embodiment, the structured model 145 constructed by machine learning can be used to structure the document image.

(Embodiment 3)
FIG. 14 is a functional block diagram showing the operation of the server 1 of the form described above. When the control unit 11 executes the program P, the server 1 operates as follows.
Acquisition unit 1401 acquires image data including a character image. A recognition unit 1402 recognizes characters and coordinates corresponding to the character image. The identification unit 1403 refers to a structured table that defines rules for identifying each element that constitutes the image data, or uses a structure classifier that identifies each element that constitutes the image data to identify the character. Identify the structure of each element of the image data that it contains. The character processing unit 1404 detects an error in the character or corrects the erroneous character based on the specified structure.

The third embodiment is as described above, and other aspects are the same as those of the first and second embodiments, so corresponding parts are given the same reference numerals and detailed description thereof will be omitted.

The embodiments disclosed this time are illustrative in all respects and should not be considered restrictive. The scope of the present invention is indicated by the scope of the claims rather than the meaning described above, and is intended to include all modifications within the scope and meaning equivalent to the scope of the claims.

1 server (information processing device)
11 Control Unit 12 Main Storage Unit 13 Communication Unit 14 Auxiliary Storage Unit P Program 141 Structured Table 142 Forward Order Model 143 Reverse Order Model 144 Error Correction Model 145 Structured Model 2 Terminal

Claims (8)

an acquisition unit that acquires image data including a character image;
a recognition unit that recognizes characters and coordinates corresponding to the character image;
Referencing a structured table that defines rules for identifying each element that constitutes the image data, or using a structure identifier that identifies each element that constitutes the image data, identifying the image data including the character. a specifying part that specifies the structure of each element;
a character processing unit that detects an error in the character or corrects the erroneous character based on the identified structure ,
The character processing unit uses a classifier that has already learned a correct sentence among sentences included in the sentence group and a sentence corresponding to the correct sentence and including an erroneous character to determine the character detect or correct errors in
An information processing device characterized by: The information processing apparatus according to claim 1, wherein the recognition unit converts the character image into text data. 3. The information processing apparatus according to claim 1 , wherein the character processing unit detects or corrects an error in the character using one or more classifiers. Claims 1 to 3, wherein the character processing unit detects or corrects an error in the characters using the classifier that has learned the sequence of characters appearing in sentences included in the sentence group. The information processing device according to any one of . The character processing unit rearranges the order of characters appearing in the sentences included in the sentence group in reverse order from the beginning to the end of the sentences with the first classifier that has been trained in the order of the sentences. 5. The information processing apparatus according to claim 4, wherein an error in the character is detected or corrected using the second classifier that has been learned in the order of the sentence. The correct sentence is text data associated with the character image in learning image data including the character image,
The sentence including the erroneous character is text data obtained by converting the character image included in the learning image data by the recognition unit . Information processing equipment. Acquire image data including character images,
recognizing characters and coordinates corresponding to the character image;
Referencing a structured table that defines rules for identifying each element that constitutes the image data, or using a structure identifier that identifies each element that constitutes the image data, identifying the image data including the character. Identify the structure of each element,
A process for detecting an error in the character or correcting the erroneous character based on the identified structure,
Detecting or correcting an error in a character by using a classifier that has already learned a correct sentence among sentences included in the sentence group and a sentence corresponding to the correct sentence and containing an erroneous character. do
An information processing method characterized in that a computer executes processing. Acquire image data including character images,
recognizing characters and coordinates corresponding to the character image;
Referencing a structured table that defines rules for identifying each element that constitutes the image data, or using a structure identifier that identifies each element that constitutes the image data, identifying the image data including the character. Identify the structure of each element,
A process for detecting an error in the character or correcting the erroneous character based on the identified structure,
Detecting or correcting an error in a character by using a classifier that has already learned a correct sentence among sentences included in the sentence group and a sentence corresponding to the correct sentence and containing an erroneous character. do
A program characterized by causing a computer to execute processing.

Images