JP7211276B2 - Information processing device, information processing method and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, when recognizing a form, it is known to read the character string to be read and recognize its attribute using a "form definition" that registers the position and attributes of the character string to be read on the image of the form. It is

Conventionally, there is known a method of recognizing a form that assumes a form with various layouts or a case where the layout of the form to be processed is unknown. Specifically, for example, conventionally, the item name of the item to be read is defined in advance, the item name is identified from the character string recognized in the form image, and the character string around the identified item name is It is known to extract the value of an item.

In order to newly generate form definition information indicating the definition of the form described above, specialized knowledge regarding form recognition is required. For this reason, conventionally, for example, an engineer interviews the user of the form about the layout and generates form definition information. difficult to do.

SUMMARY OF THE INVENTION It is an object of the present invention to easily generate definition information defining a form.

The disclosed technique acquires first form definition information that defines the positional relationship between the items of the form and the values of the items stored in the storage unit, and extracts a character string from the form image based on the first form definition information. Display control for displaying, on a display device, a screen including an item extraction unit that recognizes a certain item and the value of the item and extracts it as a recognition result, and an input reception field that receives the input of the recognition result and the second form definition information. and determining whether or not the second form definition information stored in the storage unit has been used, and if the second form definition information stored in the storage unit has not been used , the input acceptance Second form definition information is generated based on the information entered in the column and stored in the storage unit, and when the second form definition information stored in the storage unit is used , the second form definition information is and a definition registration unit that generates additional information indicating correction content of definition information and stores the additional information in the storage unit in association with the second form definition information.

Definition information that defines a form can be easily generated.

1 is a diagram showing an example of a system configuration of a form recognition system according to a first embodiment; FIG. It is a figure which shows an example of the hardware constitutions of the server apparatus of 1st embodiment. It is a figure which shows an example of the classification determination database of 1st embodiment. FIG. 4 is a diagram showing types of form definition information stored in the form definition database according to the first embodiment; It is a figure which shows an example of the general-purpose definition information of 1st embodiment. 4 is a diagram showing an example of specific definition information according to the first embodiment; FIG. It is a figure which shows an example of the individual company definition information of 1st embodiment. It is a figure explaining the function of the server apparatus of 1st embodiment. 4 is a sequence diagram for explaining the operation of the form recognition system of the first embodiment; FIG. FIG. 4 is a diagram for explaining information generated by the operation of the form recognition system of the first embodiment; FIG. 9 is a flowchart for explaining processing of a definition registration unit according to the first embodiment; It is a figure which shows an example of the confirmation screen of the recognition result which the display screen production|generation part of 1st embodiment produced|generated. FIG. 11 is a diagram showing another example of a recognition result confirmation screen generated by the display screen generation unit according to the first embodiment; It is a figure explaining correspondence with user input information and individual company definition information. FIG. 11 is a diagram showing another example of a recognition result confirmation screen generated by the display screen generation unit according to the first embodiment; FIG. 10 is a first diagram showing a registration screen for individual company definition information according to the first embodiment; FIG. 11 is a second diagram showing a registration screen for individual company definition information according to the first embodiment; FIG. 11 is a third diagram showing a registration screen for individual company definition information according to the first embodiment; FIG. 11 is a fourth diagram showing a registration screen for individual company definition information according to the first embodiment; FIG. 11 is a fifth diagram showing a registration screen for individual company definition information according to the first embodiment; FIG. 11 is a first diagram showing a registration screen for individual company definition information according to the second embodiment; FIG. 11 is a second diagram showing a registration screen for individual company definition information according to the second embodiment; FIG. 14 is a third diagram showing a registration screen for individual company definition information according to the second embodiment; FIG. 14 is a fourth diagram showing a registration screen for individual company definition information according to the second embodiment.

(First embodiment)
Embodiments will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of the system configuration of a form recognition system.

A form recognition system 100 of this embodiment has a server device 200 , an image forming device 300 and a terminal device 400 . In the form recognition system 100 of this embodiment, the server device 200, the image forming device 300, and the terminal device 400 are connected via a network.

Moreover, the terminal device 400 of this embodiment is connected to the backbone system 500 via a network, for example. The core system 500 is, for example, a system that performs specific processing using the result of recognition of a form by the form recognition system 100 of this embodiment. Specifically, for example, the core system 500 may be an accounting system, an estimate creation system, or the like.

In the form recognition system 100 of the present embodiment, the server device 200 extracts items included in the form and the values of the items from the image data representing the image of the form read by the scanner function of the image forming apparatus 300, and extracts the item values. and item values are associated with each other as text data.

In the following description, an image of a form read by the scanner function of the image forming apparatus 300 is called a form image, and image data representing the form image is called form image data.

In the following description, extracting items included in a form image and the values of the items, converting the items and the values of the items into text data, and associating them with each other is referred to as form recognition.

The server device 200 of this embodiment has a storage unit 210 , a form recognition processing unit 220 and a definition registration unit 290 . The storage unit 210 is provided with an image database 230 , a type determination database 240 , a form definition database 250 and a recognition result database 260 .

The image database 230 stores form image data. The type determination database 240 stores information referred to when determining the type of form. The form definition database 250 stores form definition information that is referenced when the form recognition processing unit 220 performs form recognition. The recognition result database 260 stores information indicating results of form recognition by the form recognition processing unit 220 .

The form recognition processing unit 220 of this embodiment acquires the form image data stored in the image database 230, refers to the type determination database 240, determines the type of the form, and specifies the form definition information to be referred to. . Then, the form recognition processing unit 220 performs form recognition using the specified form definition information in the form definition database 250 and stores the result in the recognition result database 260 . The form definition information includes information indicating the description position of the item name included in the form and information indicating the positional relationship between the item name and the item value.

The definition registration unit 290 of this embodiment causes the terminal device 400 to display an input screen for generating a part of the form definition information among the form definition information stored in the form definition database 250, and Based on this, form definition information is generated. The definition registration unit 290 then stores (registers) the generated form definition information in the form definition database 250 . Here, the definition registration unit 290 of this embodiment stores the result of recognizing the form image data based on the form definition information stored in advance in the form definition database 250 and the input fields for generating new form definition information. are displayed on the input screen together.

In this embodiment, the form definition information prestored in the form definition database 250 includes general definition information and specific definition information.

General-purpose definition information is form definition information that defines the layout of a general form. Specific definition information is form definition information that defines a specific form issued by a specific business operator or the like. Specifically, the specific form is, for example, a commonly used delivery company's slip or the like.

Further, in this embodiment, the form definition information registered by the definition registration unit 290 is called individual company definition information. In other words, the individual company definition information is part of the form definition information stored in the form definition database 250 .

The individual company definition information is form definition information that defines the layout of a form unique to a user who uses the form recognition system 100 .

In this embodiment, by displaying both the recognition result based on the pre-registered general-purpose definition information and the entry fields for the individual company-defined information, the user of the form recognition system 100 can recognize the individual company-defined information. Input can be supported, and individual company definition information can be easily generated.

In the form recognition system 100 of this embodiment, the image forming apparatus 300 is a multifunction machine having a scanner function. The image forming apparatus 300 is equipped with applications for realizing a copy function, a FAX function, a scanner function, etc. By selecting an application corresponding to each function, these functions are realized.

The terminal device 400 of this embodiment is used by a user who uses the form recognition system 100 . Further, the terminal device 400 may display the result of form recognition by the server device 200 .

Note that the user in this embodiment is, for example, a company or the like. More specifically, for example, the user is a business office, company, organization, etc. that has concluded a contract for using the service provided by the form recognition system 100 .

In the example of FIG. 1, the server device 200 is configured to have four databases in the storage unit 210, but the present invention is not limited to this. A part of each database may be provided in an external device of the server device 200, or all of the databases may be provided in an external device.

Also, in the example of FIG. 1, the server device 200 implements the form recognition processing unit 220, but the present invention is not limited to this. The form recognition processing unit 220 may be implemented by a plurality of information processing devices.

In the example of FIG. 1, one image forming device 300 and one terminal device 400 are included in the form recognition system 100. However, the number of image forming devices 300 and terminal devices 400 included in the form recognition system 100 is can be any number.

Next, with reference to FIG. 2, the hardware configuration of the server device 200 of this embodiment will be described. FIG. 2 is a diagram illustrating an example of a hardware configuration of a server device according to the first embodiment;

The server device 200 of this embodiment includes an input device 21, an output device 22, a drive device 23, an auxiliary storage device 24, a memory device 25, an arithmetic processing device 26, and an interface device 27, which are connected to each other via a bus B. It is an information processing device.

The input device 21 is a device for inputting various kinds of information, and may be, for example, a keyboard or a pointing device. The output device 22 is for outputting various kinds of information, and may be, for example, a display. The interface device 27 includes a LAN card and the like, and is used for connecting to a network.

The information processing program is at least part of various programs that control the server device 200 . The information processing program is provided by, for example, distribution of the storage medium 28 or download from the network. The storage medium 28 in which the information processing program is recorded is a storage medium for optically, electrically or magnetically recording information such as a CD-ROM, a flexible disk, a magneto-optical disk, or a storage medium for recording information such as a ROM and a flash memory. Various types of storage media can be used, such as a semiconductor memory that electrically records .

The information processing program is installed in the auxiliary storage device 24 from the storage medium 28 via the drive device 23 when the storage medium 28 recording the information processing program is set in the drive device 23 . An information processing program downloaded from the network is installed in the auxiliary storage device 24 via the interface device 27 .

The auxiliary storage device 24 stores the installed information processing program, as well as necessary files, data, and the like. The memory device 25 reads and stores the information processing program from the auxiliary storage device 24 when the server device 200 is started. The arithmetic processing unit 26 implements various types of processing described later in accordance with the information processing program stored in the memory device 25 .

Since the hardware configuration of the terminal device 400 of this embodiment is the same as that of the server device 200, description thereof will be omitted. The terminal device 400 may be, for example, a tablet-type terminal device, a smartphone, or the like, and has a display operation device realized by a touch panel or the like instead of the input device 21 and the output device 22. Also good.

Next, the type determination database 240 and the form definition database 250 of the server device 200 of this embodiment will be described.

FIG. 3 is a diagram showing an example of the type determination database of the first embodiment. The type determination database 240 of this embodiment has definition ID, definition type, document type, and identification character string as information items. In the type determination database 240, the item "definition ID" is associated with other items, and information including the value of the item "definition ID" and the values of the other items is called type determination information. For a form image containing both character strings of "company R" and "billing", the type of form indicated by the item "document type" is "bill for company R".

The value of the item "definition ID" indicates an identifier for specifying the form definition information.

The value of the item "definition type" indicates the type of form definition information. In this embodiment, specific definition information is included in addition to general-purpose definition information and individual company definition information as types of form definition information. Specific definition information is form definition information that defines a specific form issued by a specific business operator or the like. Specifically, the specific form is, for example, a commonly used delivery company's slip or the like.

The value of the item "document type" indicates the type of form. The types of forms are, for example, bills, quotations, order forms, and the like. The value of the item "identification character string" indicates a character string for identifying the type of form. Specifically, for example, for a form image containing a character string "bill" or "billing", the form type indicated by the item "document type" is "bill".

In the example of FIG. 3, for example, the type of form definition information with definition ID "H01" is general-purpose definition information, and the general-purpose definition information with definition ID "H01" is information defining the layout of "invoice". It is referred to when a character string "billing" or "bill" is extracted from the form image data.

Next, the form definition database 250 of this embodiment will be described with reference to FIGS. 4 to 7. FIG. FIG. 4 is a diagram showing types of form definition information stored in the form definition database of the first embodiment.

General-purpose definition information 251 and specific definition information 252 are stored in the form definition database 250 of this embodiment. Further, individual company definition information 253 generated by the definition registration unit 290 is stored in the form definition database 250 of this embodiment.

In the present embodiment, a priority is given to each of the three types of form definition information stored in the form definition database 250, and when recognizing form image data, the form definition information with the highest priority is selected. The items and the values of the items extracted based on this may be adopted as the recognition result.

Specifically, for example, the general-purpose definition information 251 is form definition information of a form with a highly versatile layout that is used by a plurality of users among forms, so the general-purpose definition information 251 has the lowest priority. do.

Further, the individual company definition information 253 is the form definition information of the form with the layout originally designed by the user among the forms, and is the definition that should be respected the most, so it is given the highest priority.

Further, the specific definition information 252 is form definition information indicating the layout of a specific form that is commonly used among forms, and therefore has an intermediate priority among the three form definition information.

Next, each form definition information stored in the form definition database 250 will be described.

FIG. 5 is a diagram showing an example of general-purpose definition information according to the first embodiment. The general-purpose definition information 251 is generated for each definition ID, and includes item ID, item name, direction, distance, etc. as information items. That is, the general-purpose definition information 251 of this embodiment associates the item "definition ID" with other items, and includes the value of the item "definition ID" and the values of the other items.

The value of the item "item ID" is an identifier for specifying an item included in the form. The value of the item "item name" indicates the name of the item.

Items such as "direction" and "distance" indicate ranges for extracting character strings. For example, if the value of the item "Direction" is "RD" and the value of the item "Distance" is "20 mm", the coordinates of the lower right corner of the rectangle circumscribing the character string that is the item name will be displayed downward. Indicates to extract a character string from within 20 mm.

In other words, items such as "direction" and "distance" in this embodiment are conditions indicating the range from which character strings are extracted, and these conditions are associated with item names. In the example of FIG. 5, the items "direction" and "distance" are listed as the conditions indicating the range from which the character string is extracted, but the conditions may include other items.

Therefore, for example, the character string extracted from the range within 20 mm downward from the lower right coordinate of the rectangle circumscribing the character string of the item name "total" with the item ID "2" is the item name "total". Recognized as the value of the corresponding item.

It is assumed that the general-purpose definition information 251 includes information about items that may be included in the form.

FIG. 6 is a diagram showing an example of specific definition information according to the first embodiment. The specific definition information 252 of this embodiment is registered for each specific form, and includes definition ID, item name, direction, distance, etc. as information items. In the present embodiment shown in FIG. 6, the definition ID, item name, direction, distance, etc. are the same as those of the general definition information 251, so description thereof will be omitted.

Note that the specific definition information 252 of this embodiment may be provided for each specific business operator. In this case, the specific definition information 252 may include a company ID as an information item.

FIG. 7 is a diagram showing an example of individual company definition information according to the first embodiment. The individual company definition information 253 of this embodiment is registered for each user, and includes user ID, definition ID, item name, direction, distance, etc. as information items.

The value of the item "user ID" is an identifier for specifying the user. Note that, in this embodiment, the definition ID, item name, direction, distance, etc. are the same as those of the general definition information 251, so description thereof will be omitted.

In this embodiment, the definition ID of the form definition information whose definition type is individual company definition information may include the user ID.

Further, in the individual company definition information 253, information relating to items in which item names and conditions indicating ranges in which the values of the items are described are registered by the user is stored. Therefore, the item ID included in the individual company definition information 253 may be part of the item ID included in the general definition information 251 .

The example of FIG. 7 shows the individual company definition information 253 registered with the user ID "A". This individual company definition information 253 has a definition ID of "A01" and a document Invoice”.

Next, with reference to FIG. 8, functions of the server device 200 of this embodiment will be described. FIG. 8 is a diagram for explaining the functions of the server device of the first embodiment.

In the server device 200 of the present embodiment, for example, the functions of each unit described later are realized by the arithmetic processing device 26 reading and executing an information processing program stored in the memory device 25, the auxiliary storage device 24, or the like.

The server device 200 of this embodiment has a form recognition processing unit 220 and a definition registration unit 290 . The form recognition processing section 220 has an input reception section 221 , an authentication processing section 222 and a form recognition section 270 .

The input reception unit 221 receives various inputs to the server device 200 . Specifically, the input reception unit 221 receives input of authentication information for logging into the form recognition system 100 . Authentication information is, for example, a user ID and a password. The input reception unit 221 also receives input of form image data transmitted from the image forming apparatus 300, for example.

The authentication processing unit 222 performs authentication based on the authentication information received by the input receiving unit 221 . Note that the authentication processing unit 222 may, for example, transmit the input authentication information to an authentication server provided outside the server device 200 and acquire the result of authentication by the authentication server. The authentication processing unit 222 may pass the form image data received by the input receiving unit 221 to the form recognition unit 270 when the user is authenticated.

The form recognition unit 270 of this embodiment includes an image data storage unit 271, a job list generation unit 272, an image data acquisition unit 273, a preprocessing unit 274, a character recognition unit 275, a definition identification unit 276, an item extraction unit 277, and a display screen. It has a generation unit 278 and an output unit 279 .

When the image data storage unit 271 receives the form image data from the image forming apparatus 300 , the image data storage unit 271 stores the data in the image database 230 .

In the job list generation unit 272, the input reception unit 221 receives the input of the form image data from the image forming apparatus 300, registers the job in the job list, and manages the job list. Details of the job list will be described later.

The image data acquisition unit 273 acquires form image data stored in the image database 230 . The preprocessing unit 274 performs preprocessing for form recognition on the form image data acquired by the image data acquiring unit 273 .

The character recognition unit 275 extracts a character string and information indicating the position where the character string is arranged from the form image data, and holds it as recognition result information. Details of the recognition result information will be described later.

The definition identification unit 276 refers to the type determination database 240 and identifies the definition ID corresponding to the character string included in the recognition result information. In other words, the definition identification unit 276 is an acquisition unit that identifies the form definition information to be referred to when recognizing the form based on the character string extracted from the form image data, and obtains the identified form definition information.

Based on the form definition information specified by the definition specifying part 276 and the recognition result information, the item extracting unit 277 extracts the items included in the form image data and the values of the items, and associates them. Recognition result data is generated in which item IDs and item values are associated with each other. A file in XML format (eXtensible Markup Language) may be generated as the recognition result data. In addition, the item extracting unit 277 extracts the description position of the item in the form and the positional relationship between the item and the value of the item.

The display screen generator 278 uses the recognition result data stored in the recognition result database 260 to generate a recognition result confirmation screen. The item name to be displayed on the recognition result confirmation screen is registered in advance in association with the item ID, and when the recognition result confirmation screen is generated, the item name corresponding to the item ID included in the recognition result data is specified. and display it on the recognition result confirmation screen.

The output unit 279 outputs the recognition result confirmation screen generated by the display screen generation unit 278 . In other words, the output unit 279 causes the web browser of the terminal device 400 to display the recognition result confirmation screen generated by the display screen generation unit 278 .

The definition registration unit 290 of this embodiment has a display control unit 291 , a definition generation unit 292 and a definition update unit 293 .

The display control unit 291 causes the terminal device 400 to display a screen for generating individual company definition information. Specifically, the display control unit 291 generates an XML format file for displaying the screen, and displays the input screen on the web browser of the terminal device 400 .

In this embodiment, the display control unit 291 causes the terminal device 400 to display a screen for generating individual company definition information, but the present invention is not limited to this. A device on which the display control unit 291 displays a screen may be the server device 200 or a device other than the terminal device 400, as long as it has a display function.

The definition generation unit 292 generates individual company definition information based on the information input from the terminal device 400 on the input screen. Specifically, the definition generation unit 292 generates type determination information including the definition ID of the newly generated individual company definition information, and stores it in the type determination database 240 . The definition generation unit 292 also generates individual company definition information based on the input values and stores it in the form definition database 250 .

The definition update unit 293 updates individual company definition information by reflecting corrections to the individual company definition information already registered in the form definition database 250 .

The operation of the form recognition system 100 of this embodiment will be described below with reference to FIG. FIG. 9 is a sequence diagram explaining the operation of the form recognition system of the first embodiment.

In the form recognition system 100, when the image forming apparatus 300 receives a request to start an application for performing form recognition from a user (step S901), the application is started (step S902).

Subsequently, when the image forming apparatus 300 accepts input of authentication information from the user (step S903), the image forming apparatus 300 transmits this authentication information to the server apparatus 200 and requests authentication (step S904).

The server apparatus 200 receives the authentication request, performs authentication by the authentication processing unit 222, and notifies the result to the image forming apparatus 300 (step S905). Here, the operation when the user is authenticated is shown.

Subsequently, the image forming apparatus 300 receives an instruction to scan a form (step S906), scans the form, and acquires form image data (step S907). Subsequently, the image forming apparatus 300 transmits the form image data to the server apparatus 200 (step S908).

In the server apparatus 200, when the input reception unit 221 receives the input of the form image data, the authentication processing unit 222 passes the form image data to the form recognition unit 270 (step S909).

When the form recognition unit 270 of the server device 200 receives the form image data, the job list generation unit 272 registers the job in the job list (step S910). Subsequently, the form recognition unit 270 causes the image data storage unit 271 to store the form image data in the image database 230 (step S911).

Subsequently, the form recognition unit 270 acquires the form image data to be recognized from the image database 230 by the image data acquisition unit 273 (step S912), executes processing for recognizing the form image, and the item extraction unit 277 generates recognition result data (step S913).

Subsequently, the form recognition unit 270 stores the recognition result data generated by the item extraction unit 277 in the recognition result database 260 (step S914).

In the form recognition system 100, when the terminal device 400 receives the browser display instruction input from the user (step S915), the terminal device 400 requests the server device 200 to display the recognition result data (step S916). ).

Also, the terminal device 400 accepts input of authentication information from the user (step S917), and transmits this authentication information to the server device 200 (step S918).

The server device 200 authenticates the user using the authentication processing unit 222 (step S919), and notifies the terminal device 400 of the result (step S920). Here, processing when the user is authenticated is shown.

Subsequently, when the terminal device 400 receives an input of an instruction to view the recognition result of the form image from the user (step S921), the terminal device 400 issues a viewing request to the server device 200 (step S922).

The server device 200 receives the browsing request, acquires the recognition result data stored in the recognition result database 260 (step S923), generates a recognition result confirmation screen, and displays it on the browser of the terminal device 400 (step S923). S924), the user is asked to confirm the recognition result on the displayed recognition result confirmation screen.

Subsequently, when the terminal device 400 receives an instruction to register the company-defined information from the user (step S925), the terminal device 400 transmits a request for registering the company-defined information to the server device 200 (step S926).

When the server device 200 receives this registration request, the display control section 291 of the definition registration section 290 causes the terminal device 400 to display a registration screen for individual company definition information (step S927).

Subsequently, when the terminal device 400 receives the input of the individual company definition information (step S928), the terminal device 400 transmits the input information to the server device 200 (step S929). The server device 200 uses the definition generation unit 292 to generate individual company definition information based on the information transmitted from the terminal device 400 (step S930), stores it in the form definition database 250 (step S931), and ends the process. do. Details of the processing by the definition registration unit 290 will be described later.

Here, with reference to FIG. 10, information generated in the operation of the form recognition system 100 of this embodiment will be described. Specifically, the information generated in the operation of the form recognition system 100 is a job list and recognition result data.

FIG. 10 is a diagram for explaining information generated by the operation of the form recognition system of the first embodiment. FIG. 10A is a diagram showing an example of a job list generated by the job list generating section 272. FIG.

When the form image data is input from the image forming apparatus 300, the server apparatus 200 of the present embodiment holds the form image data and the job ID as a job list in which the form image data and the job ID are associated with each other.

The job list 101 of this embodiment has job ID, user ID, form image file path, recognition result file path, status, and definition ID as information items.

The value of the item "job ID" is an identifier that identifies a job. In other words, the value of the item “job ID” is an identifier for specifying form image data received from the image forming apparatus 300 .

The value of the item “user ID” is an identifier for specifying a user who has logged into the form recognition system 100 .

The value of the item "image file path" is information indicating the location where the form image data is stored. The value of the item "recognition result file path" is information indicating the location where the recognition result data obtained by recognizing the form image is stored.

The value of the item "status" indicates the progress of recognition of the form image. The value of the item "definition ID" indicates the definition ID of the form definition information referred to when recognizing the form image data stored in the location indicated by the value of the item "image file path".

When the server apparatus 200 of the present embodiment receives the form image data from the image forming apparatus 300, the job list generation unit 272 assigns a job ID, associates the user ID obtained as the authentication information, and creates the job list 101. add a record to Then, when the form image data is stored in the image database 230, the job list generator 272 adds information indicating the storage destination as the value of the item "image file path".

Furthermore, when the form definition information to be referred to is specified in recognition of the form image, the job list generation unit 272 adds the definition ID as the value of the item "definition ID". When the job list generating unit 272 completes and the recognition result data is stored in the recognition result database 260, the information indicating the storage destination is added as the value of the item "recognition result file path". "Status" value is Completed.

FIG. 10B shows an example of recognition result data generated by the item extraction unit 277. FIG. In the recognition result data 102 shown in FIG. 10B, the items included in the form image data and the values of the items are extracted and associated, and the item IDs of the items are associated with the values of the items. is written in XML format.

In this embodiment, when the recognition result data 102 is generated and stored in the recognition result database 260 , this storage destination is set as the value of the item “recognition result file path” of the job list 101 .

Next, referring to FIG. 11, processing of the definition registration unit 290 of this embodiment will be described. FIG. 11 is a flowchart for explaining processing of the definition registration unit of the first embodiment. FIG. 11 shows the details of the processing from step S924 to step S931 in FIG.

The server device 200 of this embodiment causes the terminal device 400 to display a confirmation screen of the recognition result of the form by the form recognition unit 270 (step S1101).

Subsequently, the server apparatus 200 determines whether or not a recognition result registration request has been received from the terminal apparatus 400 (step S1102). Specifically, the server device 200 determines whether or not an instruction has been given to store the recognition result data displayed on the confirmation screen in the recognition result database 260 .

In step S1102, if the registration request for the recognition result has not been received, the server device 200 proceeds to step S1106, which will be described later.

In step S1102, when the recognition result registration request is received, the server device 200 determines whether or not the recognition result data has been corrected on the confirmation screen (step S1103). In step S1103, if the recognition result data has not been corrected, the server apparatus 200 terminates the process.

In step S1103, if the recognition result data has been corrected, the definition registration unit 290 causes the definition generation unit 292 to determine whether individual company definition information was used in recognition of the form image data (step S1104). . Specifically, the definition generation unit 292 refers to the job list 101 generated by the job list generation unit 272 and acquires the definition ID associated with the file path of the recognition result data, thereby generating the form image data. The form definition information used for recognition is determined.

In step S1104, if the individual company definition information is not used, the definition registration unit 290 proceeds to step S1107, which will be described later.

In step S1104, if the individual company definition information is used, the definition update unit 293 generates additional information indicating the content of correction, and associates it with the individual company definition information stored in the form definition database 250 ( Step S1105), the process ends.

If the registration request for recognition result data is not accepted in step S1102, the server device 200 determines whether or not the registration request for the form definition (individual company definition information) is accepted from the terminal device 400 (step S1106).

In step S1106, if the registration request for the form definition has not been received, the server device 200 returns to step S1101.

In step S1106, when the form definition registration request is received, the definition registration unit 290 causes the display control unit 291 to display an input screen for inputting individual company definition information on the terminal device 400 (step S1107).

Subsequently, the server device 200 receives input of individual company definition information from the terminal device 400 (step S1108). Subsequently, the definition generation unit 292 generates individual company definition information based on the input information, stores it in the form definition database 250 (step S1109), and terminates the process.

Below, the transition of screens in the terminal device 400 from the recognition result confirmation screen to the input of the company-defined information in this embodiment will be described.

12A is a diagram illustrating an example of a recognition result confirmation screen generated by a display screen generation unit according to the first embodiment; FIG. A screen 120 shown in FIG. 12A is a confirmation screen for allowing the user to confirm the recognition result generated as a result of recognizing the form.

A screen 120 displays a form image 121 read by the image forming apparatus 300 and information 122 in which the items extracted by the form recognition unit 270 and the values of the items are associated with each other. Also displayed on the screen 120 are a registration button 123 and an individual company definition information registration instruction button 124 . The user refers to the form image 121, checks the displayed items and item values, and corrects the item values as necessary. Then, when the registration button 123 is selected, the corrected item value is registered in the recognition result data.

In this embodiment, when the register button 123 is selected, the identification character string of the individual company definition information, which is the key for registering the individual company definition information, is fixed. Specifically, for example, in the example of FIG. 12, when the registration button 123 is selected, the item "billing party" of the information 122 is fixedly set as the identification character string.

In addition, in this embodiment, when the register button 123 is selected, a dialog for setting the identification character string of the company-defined information, which is the key for registering the company-defined information, is displayed to prompt the user. You can set it.

Specifically, for example, when the register button 123 is selected, in addition to the item "billing party", a dialog box is displayed using "billing party's telephone number", "billing party's account number", etc. as identification character string candidates. , and a candidate selected by the user may be set as the identification character string. Also, in this embodiment, a plurality of identification character strings may be set, such as the item "billing party" + "billing party phone number".

Furthermore, in the present embodiment, some character strings may be fixed in the identification character string candidates, and the user may set the identification character string from candidates including the fixed character strings. Specifically, for example, for the item "claim source" that is a candidate for the identification character string, the part of the character string "claim" is fixed, and the candidate for the identification character string that includes the character string "claim" is " The user may be allowed to select an identification character string from "billing source", "billing source's telephone number", or the like.

In this embodiment, for example, when the registration button 123 is selected, the image data representing the form image 121 and the information 122 may be stored in the core system 500 or the like. The user may also download the registered recognition result data via the recognition result confirmation screen and register it in the core system via the operation screen of the core system 500 .

Further, on the screen 120, information 125 indicating form definition information referred to when recognizing the form image 121 may be displayed. In this embodiment, by displaying this information 125, the user can be notified of which type of form definition information was used to recognize the form image 121. FIG.

FIG. 12B is a diagram illustrating another example of the recognition result confirmation screen generated by the display screen generation unit according to the first embodiment. On screen 120A shown in FIG. 12B, in addition to form image 121, information 122, registration button 123, and registration instruction button 124 displayed on screen 120, information 122A input by the user is displayed.

On the screen 120A, for example, the recognition result generated by the display screen generation unit 278 and input fields for allowing the user to input information may be displayed. The information 122A on the screen 120A may be information entered in this entry field.

In this embodiment, the information 122A input by the user on the screen 120A can be registered in association with the individual company definition information.

For example, an accounting system, which is a core system, requires entry of subject codes and department codes, but these information cannot be extracted from forms. Accordingly, such information is manually entered by the user as information 122A. In the following description, the information input by the user on screen 120A will be referred to as user input information.

In this embodiment, by registering the user input information in association with the individual company definition information, together with the result of form recognition using the individual company definition information, the user corresponding to the individual company definition information Input information can be automatically displayed on the recognition result screen. Therefore, it is possible to reduce the trouble of the user inputting information that is not automatically displayed each time a form is recognized using individual company definition information.

In this embodiment, for example, the user input information may be input by selecting from pre-registered codes.

The correspondence between user input information and individual company definition information will be described below with reference to FIG. 12C. FIG. 12C is a diagram illustrating association between user input information and individual company definition information.

The individual company definition information 253A shown in FIG. 12C includes user input information 253B associated with the individual company definition information 253. FIG.

The user input information 253B is stored in the form definition database 250 in association with the individual company definition information 253 based on the definition ID and the user ID.

In the example of FIG. 12C, the user input information 253B is registered in the same table as the individual company definition information 253, but the present invention is not limited to this. The user input information 253B may be stored in a table separate from the individual company definition information 253. FIG.

Next, with reference to FIG. 13, a display example in the case of inputting a correction on the confirmation screen of the present embodiment will be described. FIG. 13 is a diagram illustrating another example of the recognition result confirmation screen generated by the display screen generation unit according to the first embodiment.

A form image 121A, a registration button 123, a registration instruction button 124, and an input field 131 for correction contents are displayed on the screen 130 shown in FIG.

In the form image 121A, the character strings that are candidates for the corrected item values are presented by highlighting them with markers.

In this case, the input field 131 includes a display field 132 that displays a message guiding how to make corrections, and a display field 133 that displays the item selected as the correction target and the value of the item. The input field 131 also includes a button 134 for instructing the addition of candidates for items and item values presented in the form image 121A, a button 135 for notifying the server apparatus 200 of the completion of correction, and a button 135 for canceling the correction. button 136.

The example of FIG. 13 shows a case where the item "billing party" and the item value "R company" are selected from among the items and item values extracted as the result of recognition on the screen 120 of FIG. there is

Therefore, in the display field 133, the item "invoice source" and the item value "R company" are displayed. Also, in the form image 121A, the character strings highlighted by the markers 121-1, 121-2, and 121-3 are presented as possible candidates for the value of the item "invoice".

In this embodiment, when the form image 121A is recognized based on the individual company definition information, additional information reflecting the correction contents input on the screen 130 is generated, and the form image is generated in association with the individual company definition information. Stored in definition database 250 .

Specifically, for example, when the value of the item “Invoice” is corrected to the character string “Hardware Procurement” highlighted by the marker 121-3, the character string “Invoice” in the form image 121A Information indicating the correspondence relationship between the position and the position of the character string "hardware procurement" is used as additional information.

Next, referring to FIGS. 14 to 18, screen transitions in registering individual company definition information according to this embodiment will be described.

FIG. 14 is a first diagram showing a registration screen for individual company definition information according to the first embodiment. A screen 140 shown in FIG. 14 is an example of a registration screen for individual company definition information.

In this embodiment, when the registration instruction button 124 for instructing registration of the individual company definition information is operated on the screen 120 of FIG.

Screen 140 includes display fields 141 , 142 , and 143 . A form image acquired by the server device 200 is displayed in the display field 141 .

A recognition result of the form image displayed in the display column 141 is displayed in the display column 142 . Specifically, the display column 142 displays a list of item names and item values extracted as recognition results of the form image. Note that the recognition results displayed in the display column 142 are recognition results based on general-purpose definition information or specific definition information.

The display field 143 displays an input field for information related to newly generated individual company definition information.

In this embodiment, for example, when an item is selected from the list displayed in the display field 142 on the screen 140, an input field for inputting the definition of the selected item and the value of the item is displayed in the display field 142. be done. That is, the display field 142 of this embodiment is an input reception field for receiving input of individual company definition information.

Further, in this embodiment, when an item is selected from the list displayed in the display field 142 on the screen 140, an input field for information related to individual company definition information is displayed in the display field 143, and the type determination database 240 New individual company definition information and corresponding type determination information are added.

FIG. 15 is a second diagram showing the individual company definition information registration screen of the first embodiment. A screen 150 shown in FIG. 15 shows a state in which the item “sales amount for this month” is selected from the list of items and item values displayed in the display column 142 on the screen 140 .

On the screen 150, a form image is displayed in a display field 141, and an input field 151 for inputting a definition related to the selected item "Sales amount of this month" and the value of the item is displayed in the display field 142. . In other words, in the present embodiment, when selection of an item is accepted in the entry acceptance column (display column 142), the entry column 151 for entering individual company definition information is displayed in part of the entry acceptance column.

An input field 151 displays an input field 152 for defining the positional relationship between the item "sales amount for this month" and the item value, the data type of the item value, and the like. In this embodiment, for example, general-purpose definition information and specific definition information used for recognition of form image data are displayed in the input field 152, and the definitions may be edited. In other words, when inputting a definition from the input field 152, the definition regarding the corresponding item and the value of the item in the general-purpose definition information or the specific definition information may be referred to.

In addition, in the display column 143, an input column 153 for information related to the individual company definition information to be added is displayed. The information related to the individual company definition information may be type determination information corresponding to the added individual company definition information.

In this embodiment, when the icon 154 displayed in the input field 153 is selected, a text box for inputting type determination information is displayed in the input field 153 .

FIG. 16 is a third diagram showing the individual company definition information registration screen of the first embodiment. A screen 160 of FIG. 16 shows a state in which an icon 154 is selected and text boxes 161 , 162 and 163 are displayed in an input field 153 on the screen 150 .

Text boxes 161, 162, and 163 are used to input a combination of the document type and the identification character string included in the type determination information of the individual company-defined information to be added.

Specifically, for example, the text box 161 contains the document type "invoice" and the identification character string "xx", and the text box 162 contains the document type "invoice" and the identification character string "xx". The character string "PEKE" is entered.

In this embodiment, the document type and identification character string input in the input field 153 are associated with the definition type "individual company definition information", a definition ID is assigned, and stored in the type determination database 240 as type determination information. do.

In addition, in this embodiment, when a definition related to an item and an item value is input in the input field 152, the defined content is associated with the definition ID included in the type determination information input in the input field 153. , is stored in the form definition database 250 as individual company definition information.

FIG. 17 is a fourth diagram showing the individual company definition information registration screen of the first embodiment. The screen 170 of FIG. 17 shows a state in which the values of the items selected from the list of items displayed in the display column 142 are extracted based on the newly input individual company definition information on the screen 160 .

In this embodiment, when the re-extraction button 171 displayed below the display column 141 is operated, items are re-extracted from the form image displayed in the display column 141 based on the individual company definition information entered on the screen 160. Extract values.

On the screen 170, the item "sales amount of this month" is selected in the display column 142, and the re-extraction button 171 is operated while the individual company definition information is registered. In this case, on the screen 170, a value 172 extracted based on the new individual company definition information is displayed at the position corresponding to the character string "sales amount of this month" displayed as the item name in the input field 151. FIG.

At this time, since the form image is displayed in the display field 141 on the screen 170, it is possible to use whether or not the items and the item values extracted based on the new individual company definition information match the form image. can be easily discriminated.

FIG. 18 is a fifth diagram showing the individual company definition information registration screen of the first embodiment. In the screen 180 of FIG. 18, the item "billing date" is selected in the list of items displayed in the display column 142, and an input column 181 for entering the definition of the item "billing date" and its value is displayed. ing.

The input field 181 also displays an input field 182 for defining the positional relationship between the item "billing date" and the value of the item, the data type of the value of the item, and the like.

Here, in the input column 151, the value of the item extracted based on the general definition information or the specific definition information is not displayed at the position corresponding to the character string "billing date" displayed as the item name.

On the screen 180, for example, when the re-extraction button 171 is operated after the definition of the item “billing date” and the value of the item is entered in the input field 182, a new character string “billing date” is displayed at the position corresponding to the character string “billing date”. A value extracted based on the individual company definition information is displayed.

As described above, according to the present embodiment, the form image data, the items extracted as a result of recognizing the form image data using the general-purpose definition information or the specific definition information, the item list, and the individual company definition information An input reception field (display field 142) for receiving input is also displayed. Then, in this embodiment, when selection of an item is accepted from the items and the list of items displayed in the input acceptance column, an input column for individual company-defined information regarding the item and the value of the item is displayed.

Therefore, in this embodiment, individual company definition information can be input while referring to recognition results based on general-purpose definition information or specific definition information, and individual company definition information can be easily generated. In other words, according to this embodiment, it is possible to easily generate definition information that defines a form.

(Second embodiment)
A second embodiment will be described below with reference to the drawings. The second embodiment differs from the first embodiment in that the individual company definition information is input without referring to the form definition information stored in advance in the form definition database 250 . Therefore, in the following description of the second embodiment, differences from the first embodiment will be described, and those having the same functional configuration as the first embodiment will be described in the first embodiment. The same reference numerals as the used reference numerals are given, and the description thereof is omitted.

FIG. 19 is a first diagram showing a registration screen for individual company definition information according to the second embodiment. A screen 190 shown in FIG. 19 includes a display field 141 in which form image data is displayed, and a display field 142 in which a list of items and item values extracted as recognition results of the form image data is displayed.

The screen 190 also displays an input field 191 for inputting type determination information for newly generated individual company definition information.

The input field 191 displays text boxes 161, 162, and 163 in which combinations of document types and identification character strings in the type determination information are input. The input field 191 also includes an input field 192 for an extraction parameter referred to when extracting the document type and identification character string from the form image.

The extraction parameters may be, for example, coordinates indicating the position of the character string indicating the document type and the identification character string in the form image.

Further, on the screen 190, for example, when an item is selected from the list displayed in the display column 142, instead of the display column 142, an input column for defining the item and the value of the item is displayed.

FIG. 20 is a second diagram showing a registration screen for individual company definition information according to the second embodiment. A screen 201 shown in FIG. 20 shows a state in which the item "billed amount (including tax)" is selected in the display field 142 of the screen 190 shown in FIG.

On the screen 201, an input field 202 is displayed for defining the item "billing amount (including tax)" and its value. The input field 202 displays, for example, a selection field 203 for selecting a method for extracting the value of the item "claimed amount (including tax)".

For example, when "keyword" is selected in this selection column 203, the input column 204 for the keyword to be extracted, the position of the keyword (character string) to be extracted, and the numerical value that is the value of the item. A selection field 205 for selecting a relationship is displayed.

FIG. 21 is a third diagram showing a registration screen for individual company definition information according to the second embodiment. A screen 201A shown in FIG. 21 shows an example of the input field 202 when "regular expression" is selected in the selection field 203. As shown in FIG.

In the example of FIG. 21, the input field 202 displays an input field 206 for inputting a definition related to "regular expression".

FIG. 22 is a fourth diagram showing a registration screen for individual company definition information according to the second embodiment. A screen 201B shown in FIG. 22 shows an example of the input field 202 when "data type" is selected in the selection field 203. As shown in FIG.

In the example of FIG. 22, the input field 202 displays a selection field 207 and the like for inputting a definition related to "data type".

As described above, in this embodiment, the user can arbitrarily set definitions for items and item values extracted from predefined form definition information.

As described above, according to the present embodiment, since items and item values are extracted from a document image using a plurality of items of document definition information, accuracy of document recognition can be improved.

Although the present invention has been described above based on each embodiment, the present invention is not limited to the requirements shown in the above embodiments. These points can be changed within the scope of the present invention, and can be determined appropriately according to the application form.

REFERENCE SIGNS LIST 100 form recognition system 200 server device 210 storage section 220 form recognition processing section 230 image database 240 type determination database 250 form definition database 260 recognition result database 270 form recognition section 271 image data storage section 272 job list generation section 273 image data acquisition section 274 Preprocessing unit 275 Character recognition unit 276 Definition identification unit 277 Item extraction unit 278 Display screen generation unit 279 Output unit 290 Definition registration unit 291 Display control unit 292 Definition generation unit 293 Definition update unit 300 Image forming device 400 Terminal device 500 Core system

JP 2016-51339 A

Claims (8)

Acquiring first form definition information that defines the positional relationship between the items of the form and the values of the items stored in the storage unit, and extracting the items and the items that are character strings from the form image based on the first form definition information. an item extraction unit that recognizes a value and extracts it as a recognition result;
a display control unit that causes a display device to display a screen including the recognition result and an input reception field for receiving input of the second form definition information;
It is determined whether or not the second form definition information stored in the storage unit is used, and if the second form definition information stored in the storage unit is not used , an input is made to the input reception field. second form definition information is generated based on the stored information and stored in the storage unit, and when the second form definition information stored in the storage unit is used , the second form definition information is an information processing apparatus comprising: a definition registration unit that generates additional information indicating correction content, associates it with the second form definition information, and stores it in the storage unit. The display control unit
2. The information processing apparatus according to claim 1, wherein said form image is also displayed on said screen. The display control unit
displaying a list of the items that are the recognition results and the values of the items in the input reception field;
3. The information processing apparatus according to claim 1, wherein selection of an item in said list is received, and an input field for inputting said selected item and a definition of said item is displayed in a part of said input reception field. The display control unit
displaying a list of the items that are the recognition results and the values of the items in the input reception field;
3. The information processing apparatus according to claim 1, wherein selection of an item in said list is received, and an input field for inputting said selected item and a definition of a value of said item is displayed instead of said input reception field. The definition registration unit
5. The information processing apparatus according to claim 3, wherein the definition of the item and the value of the item entered in the input field is stored in the storage unit as the second form definition information. 6. The information processing apparatus according to any one of claims 1 to 5, wherein said second form definition information is form definition information defining positional relationships between user-specific form items and item values. An information processing method by an information processing device, wherein the information processing device
Acquiring first form definition information that defines the positional relationship between the items of the form and the values of the items stored in the storage unit, and extracting the items and the items that are character strings from the form image based on the first form definition information. a procedure for recognizing a value and extracting it as a recognition result; a procedure for displaying, on a display device, a screen containing the recognition result and an input reception field for receiving input of the second form definition information;
It is determined whether or not the second form definition information stored in the storage unit is used, and if the second form definition information stored in the storage unit is not used , an input is made to the input reception field. second form definition information is generated based on the stored information and stored in the storage unit, and when the second form definition information stored in the storage unit is used , the second form definition information is a step of generating additional information indicating correction details, and storing the additional information in the storage unit in association with the second form definition information;
Information processing method that performs Acquiring first form definition information that defines the positional relationship between the items of the form and the values of the items stored in the storage unit, and extracting the items and the items that are character strings from the form image based on the first form definition information. a process of recognizing a value and extracting it as a recognition result;
a process of displaying, on a display device, a screen including the recognition result and an input reception field for receiving input of the second form definition information;
It is determined whether or not the second form definition information stored in the storage unit is used, and if the second form definition information stored in the storage unit is not used , an input is made to the input reception field. second form definition information is generated based on the stored information and stored in the storage unit, and when the second form definition information stored in the storage unit is used , the second form definition information is a process of generating additional information indicating correction details, storing the additional information in the storage unit in association with the second form definition information;
An information processing program that causes a computer to execute

Images