JP2022032831A - Information processing device and program - Google Patents

Abstract

To provide an information processing device and program for enabling machine learning to make a description subject of a document correspond to a prescribed item without defining the document.SOLUTION: According to an embodiment of the present invention, an information processing device for processing a document by text mapping is provided. The information processing device includes a learned model holding part, a correspondence determination part, and a provision part. The learned model holding part holds a learned model which performs machine learning of a set of description subjects of a document and an item names of description subjects as teacher data. The correspondence determination part is configured to be able to determine which of the prescribed item names each of the description subjects included in a document file corresponds to. The provision part is configured to be able to provide the set of the description subject and the item name determined by the correspondence determination part.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing apparatus and a program.

Many forms are used in organizations such as companies. On the other hand, in these organizations, business is often carried out using a computer, and it is necessary to input the contents of the form as electronic data that can be used by the computer, and this input work may be reduced. It is desired.

For this reason, characters are recognized from the image of the form using OCR (Optical Character Recognition) to reduce the digitization work of the form, in order to improve the OCR system. Techniques have also been proposed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2019-82814

By the way, in general, a form is composed of a combination of an item name and an item content, but in many cases, the item name is omitted and only the item content is described in the form. For example, when the item name is "form type" and the item content is "invoice", the "form type" is omitted in the form and only the character string of "invoice" is described.

In addition, the format of the form is not unified, and it is common that a form having a different format is used depending on the issuer.

Therefore, even if the characters and character strings described in the form can be recognized, it is left to the judgment of the person to determine which item the recognized characters and character strings correspond to (text mapping). Often.

In addition, it is possible to perform automatic extraction by defining the position designation etc. for the form of the format that is relatively often used, but even in this case, it is necessary to define each form format. Yes, even if there is a minor change in the format, it is often necessary to change the definition.

In view of the above circumstances, the present invention has determined to provide an information processing device and a program capable of making the items described in a form correspond to a predetermined item by machine learning without defining the form.

According to one aspect of the present invention, there is provided an information processing apparatus that processes a form by text mapping. This information processing device includes a trained model holding unit, a correspondence determination unit, and a providing unit. The trained model holding unit holds a trained model that has been machine-learned using the set of the items described in the form and the item names of the items as teacher data. The correspondence determination unit is configured to be able to determine which of the predetermined item names each of the items described in the form file corresponds to, based on the trained model. The providing unit is configured to be able to provide a set of the description items and the item names determined by the correspondence determination unit.

According to one aspect of the present invention, it is possible to extract the items described in the form in correspondence with the predetermined items without defining the form in advance.

It is a figure which showed the connection structure of the information processing apparatus 1 which concerns on embodiment of this invention, and other apparatus. It is a figure which showed the outline of the structure of the information processing apparatus 1. It is a block diagram which shows the functional structure of an information processing apparatus 1. It is an activity diagram which shows the flow of operation of an information processing apparatus 1. It is a figure which showed the screen example which shows the determination result. It is a figure for demonstrating an example of learning and judgment. It is a figure for demonstrating an example of learning and judgment. It is a figure for demonstrating an example of learning and judgment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The various features shown in the embodiments shown below can be combined with each other.

By the way, the program for realizing the software appearing in the present embodiment may be provided as a non-temporary recording medium readable by a computer, may be provided as a downloadable form from an external server, or may be provided. It may be provided to start the program on an external computer and realize the function on the client terminal (so-called cloud computing).

Further, in the present embodiment, the "part" may include, for example, a combination of hardware resources implemented by a circuit in a broad sense and information processing of software specifically realized by these hardware resources. .. In addition, various information is handled in this embodiment, and these information are, for example, physical values of signal values representing voltage and current, and signal values as a bit aggregate of a binary number composed of 0 or 1. It is represented by high-low or quantum superposition (so-called qubit), and communication / operation can be executed on a circuit in a broad sense.

Further, a circuit in a broad sense is a circuit realized by at least appropriately combining a circuit, a circuit, a processor, a memory, and the like. That is, an integrated circuit for a specific application (Application Specific Integrated Circuit: ASIC), a programmable logic device (for example, a simple programmable logic device (Simple Programmable Logic Device: SPLD), a composite programmable logic device (Complex Programg)). It includes a programmable gate array (Field Programmable Gate Array: FPGA) and the like.

1. 1. Overall Configuration FIG. 1 is a diagram showing a connection configuration between the information processing apparatus 1 according to the embodiment of the present invention and another apparatus. As shown in the figure, the information processing apparatus 1 is connected to the network 5. The network 5 is, for example, a LAN (Local Area Network) or the Internet in an organization. The information processing device 1 is communicably connected to the information processing terminal 2 and the information processing terminal 4 via the network 5.

The information processing terminal 2 is a terminal used by a person in charge of processing a form, and can upload a form file to the information processing device 1 and acquire items to be described extracted from the form file. Is.

Further, the scanner 3 may be connected to the information processing terminal 2, and the form file printed by the scanner 3 on the paper surface is optically read to generate an imaged form file, and the generated form file is generated. Can be uploaded from the information processing terminal 2 to the information processing device 1. The scanner 3 can also upload the form file directly to the information processing apparatus 1 without going through the information processing terminal 2.

The information processing terminal 2 is a web browser, a personal computer on which dedicated software is operated, or the like. When the dedicated software is used, the communication between the information processing terminal 2 and the information processing apparatus 1 can be performed via API (Application Programming Interface).

The information processing terminal 4 performs the same processing as the information processing terminal 2, but is a smartphone, tablet, or the like equipped with a camera, and a form file is generated by taking a form with the mounted camera.

2. 2. Configuration of Information Processing Device 1 FIG. 2 is a diagram showing an outline of the configuration of information processing device 1. As shown in the figure, the information processing apparatus 1 has a processing unit 11, a storage unit 12, a temporary storage unit 13, an external device connection unit 14, and a communication unit 15, and these components are included. Is electrically connected to the inside of the information processing apparatus 1 via the communication bus 16.

The processing unit 11 is realized by, for example, a central processing unit (CPU), operates according to a predetermined program stored in the storage unit 12, and realizes various functions.

The storage unit 12 is a non-volatile storage medium that stores various information. This is realized by a storage device such as a hard disk drive (Hard Disk Drive: HDD) or a solid state drive (Solid State Drive: SSD). The storage unit 12 can be arranged in another device capable of communicating with the information processing device 1.

The temporary storage unit 13 is a volatile storage medium. This is realized by, for example, a memory such as a random access memory (Random Access Memory: RAM), and temporarily stores information (arguments, arrays, etc.) required when the processing unit 11 operates.

The external device connection unit 14 is a connection unit conforming to a standard such as a universal serial bus (USB) or a high-definition multimedia interface (HDMI), and may be a display device such as a monitor or the like. An input device such as a keyboard can be connected.

The communication unit 15 is, for example, a communication means conforming to a local area network (LAN) standard, and realizes communication between the information processing device 1 and a network 5 such as a local area network or the Internet via the information processing device 1. do.

As the information processing device 1, a computer for a general-purpose server, a personal computer, or the like can be used, and the information processing device 1 can be configured by using a plurality of computers.
3. 3. Functions of Information Processing Device Next, the functions of the information processing device 1 will be described. The information processing apparatus 1 operates according to a program to realize each functional unit described later. This program is a program that operates or functions the computer as the information processing device 1. The information processing device 1 is an information processing device that processes forms.

FIG. 3 is a block diagram showing a functional configuration of the information processing apparatus 1. As shown in the figure, the information processing apparatus 1 includes a reception unit 101, a form data holding unit 102, a character determination unit 103, a learning unit 104, a learned model holding unit 105, and a correspondence determination unit 106. It is provided with a providing unit 107.

The reception unit 101 receives the form file to be processed from the information processing terminal 2 or the information processing terminal 4. The form file may be an image read by the scanner 3 or an image taken by the camera of the information processing terminal 4, or may be in a PDF (Portable Document Form) format including a text (character code).

Further, the reception unit 101 is configured to be able to receive confirmation instructions for the set of the description items and the item names provided by the provision unit 107. The confirmation instruction is an instruction indicating that it has been confirmed that there is no error in the set of the description item and the item name provided by the providing unit 107. Further, the reception unit 101 is configured to be able to receive correction instructions for the set of the description items and the item names provided by the provision unit 107. The correction instruction is to instruct the correction when there is an error in the set of the description item and the item name provided by the providing unit 107. The confirmation instruction and the correction instruction are issued from the information processing terminal 2 and the information processing terminal 4.

The form data holding unit 102 includes information about the form file received by the reception unit 101, text related to the result of character determination by the character determination unit 103 for the form file, and description items determined by the correspondence determination unit 106. The set of the item name and the item name, the set of the corrected description item and the item name included in the correction instruction received by the providing unit 107, and the like are retained. The form data holding unit 102 can also be arranged in an information processing device different from the information processing device 1.

When the form file includes an image, the character determination unit 103 is configured to be able to determine the characters in the image by optical recognition. The character or character string determined by the character determination unit 103 is associated with the form file as text and is held by the form data holding unit 102.

When the reception unit 101 receives the confirmation instruction, the learning unit 104 adds the set of the description items and the item names provided by the providing unit 107 to the teacher data to perform machine learning, and the trained model holding unit 105 holds the learning unit 104. The trained model is configured to be updatable. Further, when the reception unit 101 receives the correction instruction, the learning unit 104 performs machine learning by adding the set of the description item corrected by the correction instruction and the item name to the teacher data, and performs machine learning, and the trained model holding unit The trained model held by 105 is configured to be updatable. The trained model is used by the correspondence determination unit 106 when determining what kind of item each item of the form is. Examples of the trained and trained models will be described later.

The trained model holding unit 105 holds a trained model that has been machine-learned using a set of the items described in the form and the item names of the items as teacher data. The trained model held by the trained model holding unit 105 is updated every time learning is performed by the learning unit 104.

The correspondence determination unit 106 is configured to be able to determine which of the predetermined item names each of the items described in the form file corresponds to, based on the trained model held by the trained model holding unit 105. At this time, the correspondence determination unit 106 may calculate the degree of certainty for the set of the description item and the item name. Conviction is the probability that the set of items and item names is correct.

The providing unit 107 is configured to be able to provide a set of the description items and the item names determined by the correspondence determination unit 106. At this time, the providing unit 107 may provide the conviction for the set together with the set of the description item and the item name determined by the correspondence determination unit 106, and the description item and the item determined by the correspondence determination unit 106. The pair with the name may be expressed by the color set according to the degree of conviction. Further, the providing unit 107 may provide a set of the description items and the item names as a file in CSV (Comma Separated Value) format or the like.

4. Operation of Information Processing Device Next, the operation of the information processing device 1 will be described. FIG. 4 is an activity diagram showing an operation flow of the information processing apparatus 1.

When the form file is uploaded from the information processing terminal 2 or the information processing terminal 4, the information processing apparatus 1 receives the form file (A101) by the reception unit 101 and holds it in the form data holding unit 102. Then, when the received form file is an image, the character determination unit 103 determines the character by OCR processing (A102).

Subsequently, the correspondence determination unit 106 determines the correspondence between the description items included in the form file and the item name (A103), and the provision unit 107 provides the determination result to the information processing terminal 2 or the information processing terminal 4. (A104).

The providing unit 107 provides a determination result by displaying the screen S shown in FIG. 5 on the information processing terminal 2 or the information processing terminal 4, for example. FIG. 5 is a diagram showing a screen example showing the determination result. As shown in the figure, on the screen S, a table T showing a determination result is displayed together with a form image 501 based on the form file. In the table T, for example, "billing date" is displayed as the item name, "August 1, 2020" is displayed as the corresponding description item, and "82.5" is displayed as the certainty of these correspondences. The item "August 1, 2020" corresponds to the item C1 in the form image 501.

Further, the description item C1, the description item C2, and the description item C3 in the form image 501 may be displayed in different colors according to the corresponding certainty degrees.

If there is an error in the set of the item name and the description item, for example, if there is an error in the description item for the item name "billing date", the row containing the item name "billing date" in the table T is inserted. After selecting, by selecting the entry item C2 or the like in the form image, the person in charge of operating the information processing terminal 2 or the information processing terminal 4 shall instruct the correction of the set of the item name and the entry item. Can be done. If correction is not necessary, the person in charge performs an operation of notifying the confirmation instruction. This operation is performed by pressing a confirmation button (not shown).

When the person in charge instructs to correct the set of the item name and the description item, the reception unit 101 receives the correction instruction, and the response determination unit 106 receives the correction instruction, and the response determination unit 106 receives the correction instruction and sets the item name and the description item based on the correction instruction. Is corrected (A105).

After that, the learning unit 104 learns the set of the item name and the description item determined by the correspondence determination unit 106 or the set of the corrected item name and the description item as teacher data, and holds the trained model according to the result. The trained model held by the unit 105 is updated (A106), and the information processing apparatus 1 ends the processing for the form file received by the A101.

5. Example of learning and determination Next, an example of learning by the learning unit 104 and determination by the correspondence determination unit 106 will be described. 6 to 8 are diagrams for explaining an example of learning and determination.

First, the learning unit 104 obtains the feature amount of each description item included in the form file. As the feature amount, for example, N types such as the position where the character or the character string is arranged, the font, the content, the color, and the presence or absence of the peripheral ruled line are required. Specifically, there are two feature quantities for machine learning: the absolute coordinates of the items to be described, the relative coordinates of the items to be described, the position vector position of the synonym of the item name, and the character vector of the items to be described. The above combination. Therefore, the feature amount becomes an N-dimensional value, and if this is expressed in an N-dimensional space, it will be as shown in FIG. FIG. 6 shows an example in which the feature amount F4, the feature amount F5, and the feature amount F6 corresponding to each of the description item C4, the description item C5, and the description item C6 in the form image 502 are arranged in the N-dimensional space. .. The arrangement of the feature amount F4, the feature amount F5, and the feature amount F6 in the N-dimensional space is for explanation, and the learning unit 104 moves to the N-dimensional space of the feature amount F4, the feature amount F5, and the feature amount F6. There is no need to place.

When the learning unit 104 obtains the feature amount of each description item included in the plurality of form files, the learning unit 104 specifies a range in which each item is estimated to be arranged from these feature amounts. When this range is expressed in N-dimensional space, it is as shown in FIG. The range R1 shown in FIG. 7 is a region containing a large amount of features corresponding to the same “billing date” as the feature amount F4, and the range R2 includes a large amount of features corresponding to the same “billing amount” as the feature amount F5. It is an area. Further, the range R3 is a region containing a large amount of features corresponding to the same "bank account" as the feature amount F6. The trained model held by the trained model holding unit 105 has values indicating a range R1, a range R2, and a range R3.

On the other hand, the correspondence determination unit 106 first obtains the feature amount of each description item included in the target form file. Then, the item name corresponding to each description item is determined depending on the range of the trained model in which the obtained feature amount is included. For example, as shown in FIG. 8, the feature amount F7, the feature amount F8, and the feature amount F9 obtained from each of the description item C7, the description item C8, and the description item C9 of the form image 503 are the range R1, the range R2, respectively. When it is included in the range R3, it is determined that the item name of the item C7 corresponding to the feature amount F7 included in the range R1 is "billing date", and the item name C8 corresponding to the feature amount F8 included in the range R2 is determined. It is determined that the item name of the item C9 corresponding to the feature amount F9 included in the range R3 is "bank account".

Further, the correspondence determination unit 106 can determine the degree of certainty depending on the position in the range of the trained model where the feature amount of the described item exists. Specifically, the closer the position where the feature quantity of the description item exists to the center of the range of the trained model, the higher the conviction.

5. Others The present invention may be provided in each of the following embodiments.
The information processing apparatus includes a character determination unit, and the character determination unit is configured to be capable of determining characters in an image by optical recognition when the form file includes an image. Device.
The information processing apparatus includes a reception unit and a learning unit, and the reception unit is configured to be able to receive confirmation instructions for a set of description items and item names provided by the information processing unit. Information processing configured so that when the reception unit receives a confirmation instruction, machine learning is performed by adding a set of the description items and item names provided by the provision unit to the teacher data, and the trained model can be updated. Device.
The information processing apparatus includes a reception unit and a learning unit, and the reception unit is configured to be able to receive correction instructions for a set of description items and item names provided by the information processing unit. When the reception unit receives a correction instruction, the set of the description item corrected by the correction instruction and the item name is added to the teacher data to perform machine learning, and the trained model can be updated. Processing equipment.
In the information processing apparatus, the correspondence determination unit calculates the certainty of the set of the description item and the item name, and the provision unit together with the set of the description item and the item name determined by the correspondence determination unit. An information processing device that provides confidence in a pair.
In the information processing device, the providing unit is an information processing device that expresses a set of a description item and an item name determined by the correspondence determination unit with colors set according to the certainty.
In the information processing apparatus, the feature quantity for the machine learning is one of the absolute coordinates of the description item, the relative coordinates of the description item, the position vector position of the synonym of the item name, and the character vector of the description item. An information processing device that is a combination of two or more.
A program that operates a computer as an information processing device, and is a program that causes the computer to function as the information processing device.
Of course, this is not the case.

According to each of these aspects, since learning is performed every time the form is processed, the information processing device 1 can be used without preparing a large amount of teacher data in advance, and the form is processed. Each time the above is repeated, the learning effect is increased and the certainty of the judgment is improved.

1: Information processing device 2: Information processing terminal 3: Scanner 4: Information processing terminal 5: Network 11: Processing unit 12: Storage unit 13: Temporary storage unit 14: External device connection unit 15: Communication unit 16: Communication bus 101: Reception unit 102: Form data holding unit 103: Character judgment unit 104: Learning unit 105: Learned model holding unit 106: Correspondence judgment unit 107: Providing unit 501: Form image 502: Form image 503: Form image C1: Items to be described C2 : Description item C3: Description item C4: Description item C5: Description item C6: Description item C7: Description item C8: Description item C9: Description item F4: Feature amount F5: Feature amount F6: Feature amount F7: Feature amount F8: Feature Amount F9: Feature amount R1: Range R2: Range R3: Range S: Screen T: Table

Claims (8)

An information processing device that processes forms by text mapping.
It has a trained model holding unit, a correspondence determination unit, and a providing unit.
The trained model holding unit holds a trained model that has been machine-learned using the set of the items described in the form and the item names of the items as teacher data.
The correspondence determination unit is configured to be able to determine which of the predetermined item names each of the items described in the form file corresponds to, based on the learned model.
The providing unit is an information processing device configured to be able to provide a set of a description item determined by the correspondence determination unit and an item name. In the information processing apparatus according to claim 1,
Equipped with a character judgment unit
The character determination unit is an information processing device configured so that when the form file includes an image, the characters in the image can be determined by optical recognition. In the information processing apparatus according to claim 1 or 2.
It has a reception department and a learning department.
The reception unit is configured to be able to receive confirmation instructions for a set of items and item names provided by the provision unit.
When the reception unit receives the confirmation instruction, the learning unit performs machine learning by adding the set of the description items and the item names provided by the provision unit to the teacher data, and makes it possible to update the trained model. Information processing device to be composed. In the information processing apparatus according to claim 1 or 2.
It has a reception department and a learning department.
The reception unit is configured to be able to receive correction instructions for a set of items and item names provided by the provision unit.
When the reception unit receives a correction instruction, the learning unit can perform machine learning by adding a set of a description item corrected by the correction instruction and an item name to the teacher data, and can update the trained model. Information processing device configured in. The information processing apparatus according to any one of claims 1 to 4.
The correspondence determination unit calculates the degree of conviction for the set of the description item and the item name, and obtains the conviction.
The providing unit is an information processing device that provides a set of description items and item names determined by the correspondence determination unit and a degree of certainty for the set. In the information processing apparatus according to claim 5,
The providing unit is an information processing device that expresses a set of description items and item names determined by the correspondence determination unit in colors set according to the certainty. The information processing apparatus according to any one of claims 1 to 6.
The feature quantity for machine learning is a combination of two or more of the absolute coordinates of the items to be described, the relative coordinates of the items to be described, the position vector position of the synonym of the item name, and the character vector of the items to be described. An information processing device. A program that operates a computer as an information processing device.
A program that causes a computer to function as the information processing device according to any one of claims 1 to 7.

Images