JP7447928B2 - Information processing device, program, information processing system, information processing method - Google Patents

Description

The present invention relates to an information processing device, a program, an information processing system, and an information processing method.

2. Description of the Related Art Conventionally, form recognition techniques have been known in which item names and item values included in a form image are extracted and associated with each other. Further, it is also possible to create a form definition in advance in which the written position of the character string to be read on the form image is registered, and perform form recognition using the form definition according to the layout of the form.

However, with the conventional technology, there is a possibility that misrecognition of a form may occur depending on the specific method of form definition used for form recognition.

The disclosed technology has been developed in view of the above circumstances, and aims to further improve the accuracy of form recognition.

The disclosed technology receives input of form image data, refers to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and generates form definition information used for form recognition. and determining the item value of a specific item name among the item names included in the form that is the source of the form image data, according to a specifying unit that specifies the form definition information, and a method of specifying the form definition information by the specifying unit. an item value determination unit , the form definition information includes first form definition information and item names and item values of a user-specific form that have a higher priority than the first form definition information. second form definition information defining a positional relationship, and the item value determining unit determines the item value of the specific item name according to how the specifying unit specifies the second form definition information. and the item value determination unit determines the item value of the specific item name when the second form definition information is specified according to a character string included in the form image indicated by the form image data. is determined to be a value extracted based on the second form definition information, and when specified according to the format of the form that is the source of the form image data, the item value of the specific item name is determined as the value extracted from the second form definition information. The information processing device determines the extracted value based on the first form definition information .

According to the disclosed technology, the accuracy of form recognition can be further improved.

1 is a diagram showing an example of a system configuration of a form recognition system. It is a diagram showing an example of the hardware configuration of a server device. It is a diagram showing an example of the hardware configuration of a terminal device. FIG. 3 is a diagram showing an example of a type determination database. FIG. 3 is a diagram showing types of form definition information stored in a form definition database. FIG. 3 is a diagram showing an example of general-purpose definition information. It is a figure showing an example of individual company definition information. FIG. 3 is a diagram showing an example of a format management database. It is a figure explaining the function of a server apparatus. FIG. 3 is a diagram illustrating the functions of a terminal device. FIG. 2 is a sequence diagram illustrating the operation of the form recognition system. FIG. 3 is a diagram showing an example of a job list. It is a first flowchart explaining the processing of a form recognition part. It is a second flowchart explaining the processing of a form recognition part. It is a flowchart explaining the processing of a form management part. FIG. 3 is a first diagram illustrating the effects of this embodiment. FIG. 2 is a second diagram illustrating the effects of this embodiment.

Embodiments will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of the system configuration of a form recognition system.

The form recognition system 100 of this embodiment includes a server device 200, an image forming device 300, and a terminal device 400. In the form recognition system 100 of this embodiment, the server device 200, the image forming device 300, and the terminal device 400 are connected via a network. The form recognition system 100 of this embodiment is an example of an information processing system.

Further, the terminal device 400 of this embodiment is connected to the core system 500 via a network, for example. The core system 500 is, for example, a system that performs specific processing using the result of recognizing a form in the form recognition system 100 of this embodiment. Specifically, for example, the core system 500 may be an accounting system, an estimate creation system, or the like.

In the form recognition system 100 of the present embodiment, the server device 200 extracts item names and item values included in the form from image data indicating the image of the form read by the scanner function of the image forming device 300, and extracts item names and item values included in the form. This is text data in which name and item value are associated with each other. The item name is the name of the item included in the form, and the item value represents the value corresponding to the item specified by the item name.

In the following description, the image of a form read by the scanner function of the image forming apparatus 300 will be referred to as a form image, and the image data representing the form image will be referred to as form image data. Note that the image forming apparatus 300 does not need to be included in the form recognition system 100.

Furthermore, in the following explanation, extracting an item name included in a form image and an item value corresponding to the item name, and associating the item name with the item value will be referred to as form recognition.

The server device 200 of this embodiment includes a storage section 210, a form recognition section 220, and a form management section 290. The storage unit 210 is provided with a form image database 230, a type determination database 240, a form definition database 250, a recognition result database 260, a format management database 265, and a corporate name dictionary database 267.

The form image database 230 stores form image data. The type determination database 240 stores information that is referred to when determining the type of a form. The form definition database 250 stores form definition information used for form recognition by the form recognition unit 220. The form definition information is information that defines the position of item names included in the form and the positional relationship between the item names and item values.

The recognition result database 260 stores form information (recognition result data) that is information indicating the result of form recognition by the form recognition unit 220. The format management database 265 stores format management information for managing the format of forms. The corporate name dictionary database 267 stores dictionary information including corporate names. The dictionary information including a corporate name may be information including a corporate name, a corporate address, a telephone number, etc.

The corporate name dictionary database 267 of this embodiment may be, for example, a publicly available database, or may be created in advance by the administrator of the form recognition system 100.

The form recognition unit 220 of this embodiment acquires form image data stored in the form image database 230 and refers to the type determination database 240 to determine the type of the form. Further, the form recognition unit 220 specifies form definition information used for form recognition, and determines item values corresponding to specific item names included in the form based on the specified form definition information.

Specifically, the form recognition unit 220 recognizes general-purpose definition information (first form definition information) and individual company definition information (second form definition information) included in the form definition information stored in the form definition database 250. ).

The general-purpose definition information in this embodiment is information that defines words corresponding to item names in general forms. The individual company definition information is information that defines words corresponding to item names in a form specific to the user who uses the form recognition system 100 and information to be extracted from the form image. Users who use the form recognition system 100 are businesses, companies, organizations, etc. that have concluded contracts to use the services provided by the form recognition system 100. In the following description of this embodiment, the users of the form recognition system 100 will be referred to as tenants.

In this embodiment, individual company definition information is provided for each tenant, and the form recognition unit 220 recognizes specific item names included in the form depending on how the individual company definition information used for form recognition is specified. Determine item value. The specific item name in this embodiment may be, for example, the item name "issuer" indicating the issuer of the form.

More specifically, the form recognition unit 220 determines whether the individual company definition information is specified based on the format of the form or based on the result of character recognition on form image data. Then, the form recognition unit 220 determines the item value of the item "issuer" of the form to be output as the result of form recognition, according to the result of this determination.

Here, a case where the individual company definition information is specified based on the format of the form and a case where the individual company definition information is specified based on the result of character recognition will be explained.

When specifying the individual company definition information based on the format of the form, the individual company definition information is specified as the individual company definition information of the issuer that is associated with format information similar to the format of the form. Therefore, when identifying company-defined information based on format, even if the issuer is different from the actual issuer, if the layout of the form is similar, it will be identified as company-defined information used for form recognition. there is a possibility.

That is, when the individual company definition information is specified based on the format of the form, the identified individual company definition information may be the individual company definition information of a different issuer than the issuer of the form.

On the other hand, when identifying individual company-defined information based on the result of character recognition, identify individual company-defined information in which the character string included in the form and the publisher associated with the individual company-defined information completely match. do. Therefore, it is unlikely that the individual company definition information specified in the result of character recognition is the individual company definition information of a different issuer than the issuer of the form.

From this, it can be said that the individual company definition information specified based on the result of character recognition is more reliable than the individual company definition information specified based on the format of the form.

In this embodiment, we focus on this point and explain how to determine the item value of the item name "Publisher" when the individual company definition information is specified by the format of the form. be different from the case specified by.

Therefore, in this embodiment, even if the individual company definition information used for form recognition is specified based on the format of the form, misrecognition of the issuer's name can be suppressed. In other words, in this embodiment, it is possible to suppress misrecognition of a form due to the method of specifying form definition information used for form recognition, and improve the accuracy of form recognition.

In the form recognition system 100 of this embodiment, the image forming apparatus 300 is a multifunction device having a scanner function. The image forming apparatus 300 is loaded with applications for realizing a copy function, a FAX function, a scanner function, etc., and these functions are realized by selecting an application corresponding to each function.

The terminal device 400 of this embodiment is mainly used by tenants. Furthermore, a screen or the like may be displayed on the terminal device 400 for confirming the result of form recognition by the server device 200.

In the example of FIG. 1, the server device 200 has a configuration having six databases in the storage unit 210, but the configuration is not limited to this. A part of each database may be provided in an external device of the server device 200, or all of each database may be provided in an external device.

Further, in the example of FIG. 1, the server device 200 implements the form recognition section 220 and the form management section 290, but the present invention is not limited to this. The form recognition unit 220 and the form management unit 290 may be realized by a plurality of information processing devices.

In the example of FIG. 1, the number of image forming apparatuses 300 and terminal apparatuses 400 included in the form recognition system 100 is one each, but the number of image forming apparatuses 300 and terminal apparatuses 400 included in the form recognition system 100 is may be any number.

Next, with reference to FIG. 2, the hardware configuration of the server device 200 of this embodiment will be described. FIG. 2 is a diagram showing an example of the hardware configuration of the server device.

The server device 200 of this embodiment is constructed by a computer, and as shown in FIG. /F (Interface) 208, network I/F 209, data bus B, keyboard 211, pointing device 212, DVD-RW (Digital Versatile Disk Rewritable) drive 214, and media I/F 216.

Among these, the CPU 201 controls the operation of the server device 200 as a whole. The ROM 202 stores programs used to drive the CPU 201, such as IPL. RAM 203 is used as a work area for CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls reading and writing of various data to the HD 204 under the control of the CPU 201. The display 206 is a display unit that displays various information such as a cursor, menu, window, characters, or images.

External device connection I/F 208 is an interface for connecting various external devices. The external device in this case is, for example, a USB (Universal Serial Bus) memory, a printer, or the like. The network I/F 209 is an interface for data communication using the communication network N. The data bus B is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 201 shown in FIG. 2.

Further, the keyboard 211 is a type of input means that includes a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 214 controls reading and writing of various data on the DVD-RW 213, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may be DVD-R or the like. The media I/F 216 controls reading or writing (storage) of data to a recording medium 215 such as a flash memory.

The server device 200 of this embodiment may be, for example, a smartphone, a tablet terminal, a PDA (Personal Digital Assistant), a wearable PC, or the like.

The terminal device 400 of this embodiment is a computer having an arithmetic processing unit and a storage device, and may be, for example, a tablet-type terminal device, a smartphone, or the like. FIG. 3 shows the terminal device 400 of this embodiment, which includes a CPU 401, ROM 402, RAM 403, EEPROM 404, CMOS sensor 405, image sensor I/F 406, acceleration/direction sensor 407, media I/F 409, data bus 410, and GPS receiving section. 411.

Among these, the CPU 401 is an arithmetic processing unit that controls the operation of the entire terminal device 400. The ROM 402 stores the CPU 401 and programs used to drive the CPU 401 such as IPL. RAM 303 is used as a work area for CPU 301. The EEPROM 404 reads or writes various data such as a smartphone (terminal device) program under the control of the CPU 401. ROM 402, RAM 403, and EEPROM 404 are examples of storage devices of terminal device 400.

A CMOS (Complementary Metal Oxide Semiconductor) sensor 405 is a type of built-in imaging means that images a subject (mainly a self-portrait) and obtains image data under the control of the CPU 401. Note that instead of a CMOS sensor, an imaging means such as a CCD (Charge Coupled Device) sensor may be used.

The image sensor I/F 406 is a circuit that controls driving of the CMOS sensor 305. The acceleration/direction sensor 407 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect geomagnetism. A media I/F 409 controls reading or writing (storage) of data to a recording medium 408 such as a flash memory. The data bus 410 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401 shown in FIG. 3. GPS receiving section 411 receives GPS signals from GPS satellites.

The terminal device 400 also includes a long-distance communication circuit 412, an antenna 412a of the long-distance communication circuit 412, a microphone 415, a speaker 416, a sound input/output I/F 417, a display (display unit) 418, an external device connection I/F (Interface ) 419, a short-range communication circuit 420, an antenna 420a of the short-range communication circuit 420, and a touch panel 421.

Among these, the long distance communication circuit 412 is a circuit that communicates with other devices via a communication network. Microphone 415 is a built-in circuit that converts sound into electrical signals. The speaker 416 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and voice. The sound input/output I/F 417 is a circuit that processes input/output of sound signals between the microphone 415 and the speaker 416 under the control of the CPU 401 .

The display 418 is a type of display means such as a liquid crystal or organic EL (Electro Luminescence) that displays images of the subject, various icons, and the like. The external device connection I/F 419 is an interface for connecting various external devices. The near field communication circuit 420 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 421 is a type of input means by which the user operates the terminal device 400 by pressing the display 418. Display 418 is an example of a display unit included in terminal device 400.

Next, the type determination database 240 and form definition database 250 included in the server device 200 of this embodiment will be explained.

FIG. 4 is a diagram showing an example of a type determination database. The type determination database 240 of this embodiment has definition ID, definition type, document type, and identification character string as information items. In the type determination database 240, the item "definition ID" is associated with other items, and information including the value of the item "definition ID" and the values of the other items is called type determination information.

The value of the item "definition ID" indicates identification information for specifying form definition information. The value of the item "definition type" indicates the type of form definition information. In this embodiment, the types of form definition information include general definition information and individual company definition information.

The value of the item "document type" indicates the type of form. Specifically, the types of forms include, for example, invoices, estimates, and order forms.

The value of the item "identification character string" indicates a character string for identifying the type of form. Specifically, for example, for a form image that includes the character string "invoice" or "request", the type of form indicated by the item "document type" is "invoice".

In the example of FIG. 4, for example, the type of form definition information with the definition ID "H01" is general-purpose definition information, and the general-purpose definition information with the definition ID "H01" is information that defines the layout of "invoice", It is referenced when the character string "Claim" or "Bill" is extracted from the form image data.

In addition, in the example of FIG. 4, for example, the type of form definition information with the definition ID "A01" is individual company definition information, and the individual company definition information with the definition ID "A01" is the layout of "Invoice for Company R". This is information that defines the ``invoice'' and is referenced when the character string ``request'' or ``invoice'' is extracted from the form image data.

Next, the form definition database 250 of this embodiment will be described with reference to FIGS. 5 to 7. FIG. 5 is a diagram showing the types of form definition information stored in the form definition database.

The form definition database 250 of this embodiment stores general-purpose definition information 251 and individual company definition information 252.

The general-purpose definition information 251 of this embodiment is form definition information of a form with a highly versatile layout that is used by a plurality of users (tenants) among forms. The general-purpose definition information 251 is form definition information with a lower priority than the individual company definition information 252.

The individual company definition information 252 of this embodiment is form definition information of a form with a layout uniquely designed by the user among forms, and is form definition information exclusive to the user. The individual company definition information 252 is a definition that should be respected and is form definition information with a higher priority than the general purpose definition information 251.

The form recognition unit 220 of this embodiment uses (references to) these two pieces of form definition information to recognize form image data. Therefore, in this embodiment, for example, even if the form image data is of a form whose layout is not defined in the general-purpose definition information 251, if the layout is defined in the individual company definition information 252, the form image data can be recognized.

FIG. 6 is a diagram showing an example of general-purpose definition information. The general-purpose definition information 251 is generated for each definition ID, and includes item ID, item name, direction, distance, etc. as information items. That is, the general-purpose definition information 251 of this embodiment has the item "definition ID" associated with other items, and includes the value of the item "definition ID" and the values of the other items.

The value of the item "item ID" is identification information for specifying the item included in the form. The value of the item "item name" indicates the name of the item.

Items such as "direction" and "distance" indicate the range from which character strings are extracted. For example, if the value of the item "Direction" is "RD" and the value of the item "Distance" is "20mm", move downward from the lower right coordinates of the rectangle circumscribing the character string that is the item name. Indicates that a character string is extracted from a range within 20 mm.

That is, the items "direction", "distance", etc. in this embodiment are conditions indicating the range from which character strings are extracted, and these conditions are associated with item names. In the example of FIG. 5, the items "direction" and "distance" are used as conditions indicating the range from which character strings are extracted, but the conditions may include other items.

Therefore, for example, a character string extracted from a range within 20 mm downward from the lower right coordinates of a rectangle that circumscribes the character string "Total" as the item name of the item ID "2" is called "Total" as the item name. It is recognized as the corresponding item value.

Note that the general-purpose definition information 251 includes information regarding all items included in the form.

FIG. 7 is a diagram showing an example of individual company definition information. The individual company definition information 252 of this embodiment is registered for each tenant definition ID, and includes definition ID, item ID, item name, direction, distance, etc. as information items.

Further, the individual company definition information 252 includes the issuer, telephone number, account number, etc. as information items. The items "issuer", "telephone number", "account number", etc. are items related to the issuer of the form.

Further, information including the items "issuer", "telephone number", "account number" and the values of each item is issuer identification information for specifying the issuer of the form.

The items "definition ID", "item ID", "item name", "direction", "distance, etc." are the same as those in the general-purpose definition information 251, so their explanation will be omitted. The value of the item "issuer" is information that identifies the issuer (tenant) of the form. For example, when the form is a bill, the issuer of the form is synonymous with the billing source of the bill. The values of the items "telephone number" and "account number" indicate the billing party's telephone number and account number.

Note that in this embodiment, the name of the issuer is used as information for identifying the issuer of the form, but the present invention is not limited to this. The information for specifying the issuer of the form is, for example, provided with tenant identification information (tenant ID) for identifying the tenant for each tenant, and the tenant ID may be used as information for specifying the issuer of the form.

In this way, the individual company definition information 252 of this embodiment is associated with the identification information of the issuer of the form.

Note that the identification information of the issuer of the form may be stored in a tenant database for managing the tenant, for example, in association with the tenant ID. In this case, the individual company definition information 252 only needs to be associated with the tenant ID.

The individual company definition information 252 shown in FIG. 7 is an example of the individual company definition information with the definition ID "A01" and the document type "Invoice for Company R" registered by the tenant specified by the issuer "Company R". Yes (see Figure 4). Therefore, it can be seen that the individual company definition information 252 is information that defines the layout of the "Invoice for Company R".

Next, with reference to FIG. 8, the format management database 265 of this embodiment will be explained. FIG. 8 is a diagram showing an example of a format information database.

The format management database 265 of this embodiment is provided for each tenant, and includes issuer identification information and format information as information items. In the format management database 265, the item "Issuer identification information" and the item "Format information" are associated. In the following description, information including the value of the item "Issuer identification information" and the value of the item "Format information" in the format management database 265 may be expressed as format management information.

The item "Issuer identification information" includes the items "Issuer", "Telephone number", and "Account number" regarding the issuer of the form. The value of the item "format information" indicates information indicating the format of the form, and includes the items "maximum rectangular width", "ruled line color", "character information", etc.

In this way, the format management information of this embodiment can be said to be information indicating the layout of forms for each issuer.

In addition, if the identification information of the issuer is stored in the tenant database in association with the tenant ID, the format information only needs to be associated with the tenant ID.

Next, with reference to FIG. 9, the functions of the server device 200 of this embodiment will be described. FIG. 9 is a diagram illustrating the functions of the server device.

In the server device 200 of this embodiment, the functions of each section described below are realized by the CPU 201 reading and executing an information processing program stored in the RAM 203, HD 204, etc., for example.

The server device 200 of this embodiment includes a form recognition section 220, a form management section 290, and a communication section 298. The form recognition section 220 includes an input reception section 221, an authentication processing section 222, and a recognition processing section 270. The form management section 290 includes an operation reception section 291, a display control section 292, and an information update section 293.

In the form recognition unit 220, the input reception unit 221 receives various inputs to the server device 200. Specifically, the input accepting unit 221 accepts input of authentication information for logging into the form recognition system 100. The authentication information is, for example, a tenant ID and password. Further, the input receiving unit 221 receives input of form image data transmitted from the image forming apparatus 300, for example. In addition, the input receiving unit 221 receives various requests and notifications transmitted from the terminal device 400. The input reception unit 221 may accept input to the server device 200 via the communication unit 298, or may accept input directly made to the server device 200.

The authentication processing unit 222 performs authentication based on the authentication information input by the input reception unit 221. Note that the authentication processing unit 222 may, for example, transmit the input authentication information to an authentication server provided outside the server device 200 and obtain the result of authentication by the authentication server. When the user is authenticated, the authentication processing section 222 may pass the form image data received by the input reception section 221 to the recognition processing section 270.

The recognition processing unit 270 of this embodiment includes an image data storage unit 271, a job list generation unit 272, an image data acquisition unit 273, a preprocessing unit 274, a character recognition unit 275, a definition identification unit 276, an item extraction unit 277, and an item value It includes a determining section 278, a display screen generating section 279, and an output section 280.

When the input reception unit 221 receives form image data from the image forming apparatus 300 via the communication unit 298, the image data storage unit 271 stores the form image data in the form image database 230.

When the input reception unit 221 receives input of form image data from the image forming apparatus 300 via the communication unit 298, the job list generation unit 272 registers the job in the job list and manages the job list. Details of the job list will be described later.

The image data acquisition unit 273 acquires form image data stored in the form image database 230. The preprocessing unit 274 performs preprocessing for performing form recognition on the form image data acquired by the image data acquisition unit 273.

The character recognition unit 275 extracts a character string and information indicating the position where the character string is placed from the form image data, and holds the extracted character string as reading result data. Details of the read result data will be described later.

The definition specifying unit 276 specifies form definition information to be referred to during form recognition based on the character string included in the read result data, and acquires the specified form definition information from the form definition database 250.

The item extracting unit 277 extracts item names and item values included in the form image data based on the reading result data and the form definition information specified by the definition specifying unit 276.

The item value determining unit 278 determines the item value to be associated with the item name extracted by the item extracting unit 277 according to the method of specifying the form definition information by the definition specifying unit 276, and extracts the item name and the determined item value. Correlate with. More specifically, the item value determination unit 278 determines the item value of the issuer of the form depending on how the individual company definition information is specified.

Furthermore, the item value determining unit 278 stores recognition result data in which item names and item values are associated with each other in the recognition result database 260. Note that the recognition result data of this embodiment includes information indicating the issuer of the form written on the form (publisher information).

The display screen generation unit 279 uses the recognition result data stored in the recognition result database 260 to generate a recognition result confirmation screen.

The output unit 280 outputs the recognition result confirmation screen generated by the display screen generation unit 279 via the communication unit 298. In other words, the output unit 280 displays the recognition result confirmation screen generated by the display screen generation unit 279 on the web browser of the terminal device 400 via the communication unit 298.

The form management section 290 of this embodiment includes an operation reception section 291, a display control section 292, and an information update section 293.

The operation reception unit 291 receives, via the communication unit 298, an operation input on a screen displayed on the web browser of the terminal device 400. More specifically, in this embodiment, the communication unit 298 receives information indicating an operation from the terminal device 400, and the operation reception unit 291 receives input of the information received by the communication unit 298.

The display control unit 292 instructs the terminal device 400 to display a screen via the communication unit 298. Specifically, the display control unit 292 causes the terminal device 400 to display a confirmation screen for recognition result data, an input screen for generating individual company definition information, and the like. Further, the display control unit 292 generates an XML format file for displaying a screen, and displays the screen on the web browser of the terminal device 400 via the communication unit 298.

Note that the device on which the display control unit 292 displays the screen may be the server device 200 or a device other than the terminal device 400, and may be any display device that has a display function.

The information update unit 293 generates individual company definition information based on the information input from the terminal device 400 and stores it in the form definition database 250.

The communication unit 298 of this embodiment is a function realized by, for example, the network I/F 209, and transmits and receives data between the server device 200 and an external device. Specifically, for example, the communication unit 298 transmits a notification instructing the terminal device 400 to display the screen under the control of the display control unit 292.

Next, with reference to FIG. 10, the functions of the terminal device 400 of this embodiment will be described. FIG. 10 is a diagram illustrating the functions of the terminal device.

The functions of each section described below are realized by the CPU 401 of the terminal device 400 reading and executing a program stored in the ROM 402 or the like.

The terminal device 400 of this embodiment includes an input reception section 431, a display control section 432, and a communication section 433.

The input receiving unit 431 receives input on the screen displayed on the web browser of the terminal device 400 by the display control unit 432. Specifically, the input receiving unit 431 receives an operation for editing the recognition result on the recognition result confirmation screen displayed on the web browser of the terminal device 400.

The display control unit 432 controls the display on the display 418 of the terminal device 400. Specifically, the display control unit 432 causes the display 418 (display unit) to display a screen according to an instruction from the server device 200.

The communication unit 433 is a function realized by the long-distance communication circuit 412, the antenna 412a of the long-distance communication circuit 412, and the like, and transmits and receives data between the terminal device 400 and other devices.

The operation of the form recognition system 100 of this embodiment will be described below with reference to FIG. 11. FIG. 11 is a sequence diagram illustrating the operation of the form recognition system.

In the form recognition system 100, when the image forming apparatus 300 receives a request from a user to start an application for performing form recognition (step S1101), it starts this application (step S1102).

Subsequently, the image forming apparatus 300 receives the input of authentication information from the user (step S1103), and transmits this authentication information to the server apparatus 200 to issue an authentication request (step S1104).

Upon receiving the authentication request, the server apparatus 200 performs authentication using the authentication processing unit 222, and notifies the image forming apparatus 300 of the result (step S1105). Note that here, the operation when the user is authenticated is shown.

Subsequently, the image forming apparatus 300 receives an instruction to scan the form (step S1106), scans the form, and obtains form image data (step S1107). Subsequently, the image forming apparatus 300 transmits the form image data to the server apparatus 200 (step S1108).

In the server device 200, when the input receiving unit 221 receives the input of form image data, the authentication processing unit 222 passes this form image data to the recognition processing unit 270 (step S1109). Note that in the example of FIG. 11, the form image data is passed to the recognition processing section 270 via the authentication processing section 222, but the present invention is not limited to this. The form image data may be passed to the recognition processing section 270 without going through the authentication processing section 222.

When the recognition processing unit 270 of the server device 200 receives the form image data, the job list generation unit 272 registers the job in the job list (step S1110). Subsequently, the recognition processing unit 270 causes the image data storage unit 271 to store the form image data in the form image database 230 (step S1111).

Next, the recognition processing unit 270 uses the image data acquisition unit 273 to acquire form image data to be recognized from the form image database 230 (step S1112), and executes processing to recognize the form image (form recognition). (Step S1113). Details of the process in step S1113 will be described later.

Subsequently, the recognition processing unit 270 stores the recognition result data generated by the item extraction unit 277 in the recognition result database 260 (step S1114).

In the form recognition system 100, when the terminal device 400 receives an input of a browser display instruction from the user through the input receiving unit 431 (step S1115), the terminal device 400 sends the recognition result to the server device 200 via the communication unit 433. A display request is made to the browser for displaying the data (step S1116).

Furthermore, the terminal device 400 receives an input of authentication information from the user through the input reception unit 431 (step S1117), and transmits this authentication information to the server device 200 through the communication unit 433 (step S1118).

The server device 200 authenticates the user using the authentication processing unit 222 (step S1119), notifies the terminal device 400 of the result, and causes the terminal device 400 to display a job list list (step S1120). Note that here, the processing when the user is authenticated is shown.

Next, when the terminal device 400 receives the selection of a job from the job list through the input reception unit 431 (step S1121), the communication unit 433 transmits the selection to the server as an instruction to view the recognition result of the form image. A viewing request is made to the device 200 (step S1122).

In response to the viewing request, the server device 200 uses the recognition processing unit 270 to acquire the recognition result data stored in the recognition result database 260 (step S1123), generates a recognition result confirmation screen, and displays the recognition result data on the terminal device 400. The recognition result is displayed on the browser (step S1124), and the user is asked to confirm the recognition result on the displayed recognition result confirmation screen.

Subsequently, when the terminal device 400 receives an input of an instruction to register individual company definition information through the input reception unit 431 (step S1125), the terminal device 400 sends a registration request for individual company definition information to the server device 200 via the communication unit 433. Transmit (step S1126). In other words, this registration request is a request to display an input screen for inputting individual company definition information.

When the input reception unit 221 receives this display request, the server device 200 causes the display control unit 292 of the form management unit 290 to display an input screen for individual company definition information on the terminal device 400 (step S1127).

Subsequently, when the terminal device 400 receives the input of the individual company definition information (step S1128), the terminal device 400 transmits the input information to the server device 200 (step S1129). The form management unit 290 in the server device 200 uses the information update unit 293 to generate individual company definition information based on the information transmitted from the terminal device 400 (step S1130), and stores it in the form definition database 250 (step S1131). ), the process ends.

Here, with reference to FIG. 12, the job list generated by the job list generation unit 272 of this embodiment will be described. FIG. 12 is a diagram showing an example of a job list.

When the server apparatus 200 of this embodiment receives form image data from the image forming apparatus 300, the job list generation unit 272 stores the form image data and job ID as a job list in association with each other.

The job list 101 of this embodiment has information items such as job ID, issuer, form image file path, recognition result file path, status, and definition ID.

The value of the item "Job ID" is identification information that identifies the job. In other words, the value of the item "job ID" is identification information for identifying the form image data received from the image forming apparatus 300.

The value of the item “issuer” is information for identifying the user who has logged in to the form recognition system 100.

The value of the item "image file path" is information indicating the location where the form image data is stored. The value of the item "recognition result file path" is information indicating the location where recognition result data resulting from recognition of a form image is stored.

The value of the item "Status" indicates the progress of form image recognition. In other words, the value of the item "status" indicates the state of the job.

In this embodiment, there are four values for the item "status": unprocessed state, draft saved state, confirmed state, and externally outputted state. Each state is as follows.

- Unprocessed state (state 1) A state in which recognition result data has just been acquired by form recognition, and the recognition results have not been confirmed.

・Draft save state (state 2) A state in which work such as confirmation of recognition result data is in progress, and before recognition result data is finalized.

- Confirmed state (state 3) A state in which work such as confirmation of recognition result data has been completed and it is stored in the recognition result database 260 as confirmed information.

- External output completed state (state 4) A state in which the output of recognition result data to the core system 500 that cooperates with the form recognition system 100 has been completed.

The value of the item "definition ID" indicates the definition ID of the form definition information used to generate the recognition result data. In other words, the value of the item "definition ID" is identification information that specifies the form definition information used for form recognition.

When the server device 200 of this embodiment receives form image data from the image forming device 300, the job list generation unit 272 assigns a job ID, associates the issuer obtained as authentication information, and creates the job list 101. Add records. Then, when the form image data is stored in the form image database 230, the job list generation unit 272 adds information indicating the storage location as the value of the item "image file path".

Furthermore, when the recognition result data is stored in the recognition result database 260, the job list generation unit 272 adds information indicating the storage location as the value of the item "recognition result file path" and changes the value of the item "status" to the value of the item "recognition result file path". Set to confirmed state.

In this embodiment, for example, a job list generated by the job list generation unit 272 may be displayed. In this case, the user can check the progress of form image recognition via the job list screen displayed based on the job list.

Next, with reference to FIG. 13, the processing of the recognition processing section 270 of this embodiment will be described. FIG. 14 is a first flowchart illustrating the processing of the form recognition unit. FIG. 13 shows details of the process of step S1113 in FIG. 11. Further, FIG. 13 shows processing when a plurality of form image data is received from the image forming apparatus 300.

In the recognition processing unit 270 of this embodiment, when the image data acquisition unit 273 acquires form image data from the form image database 230 (step S1301), the preprocessing unit 274 converts the format of the form image data into a PNG format. (Step S1302).

Next, the preprocessing unit 274 selects one page of form image data (step S1303) and performs preprocessing (step S1304). Specifically, the preprocessing is, for example, a correction process for aligning the orientation of the form image.

Next, the recognition processing unit 270 uses the character recognition unit 275 to cut out an area where characters are formed (hereinafter referred to as a character field) from the form image (step S1305). Subsequently, the recognition processing unit 270 uses the character recognition unit 275 to cut out the characters in the cut out character field and perform character recognition (step S1306). At this time, the character recognition unit 275 also acquires the coordinates indicating the position of the extracted character.

Subsequently, the recognition processing unit 270 generates and holds reading result data based on the positional relationship of each character recognized by the character recognition unit 275 (step S1307). Specifically, based on the coordinates of each recognized character, the recognition processing unit 270 recognizes characters whose distance from each other is within a preset threshold as one character string, and forms a character string. The reading result data is generated by associating the data with information such as coordinates indicating the position of the area.

Subsequently, the recognition processing unit 270 uses the definition specifying unit 276, the item extracting unit 277, and the item value determining unit 278 to extract the item name and item value from the reading result data (step S1308). Details of the process in step S1308 will be described later.

Subsequently, the recognition processing unit 270 determines whether or not the item name and item value have been successfully extracted (step S1309). Specifically, the recognition processing unit 270 determines that the extraction of the item name and item value has been successful when data that associates the item name and the item value is temporarily held. good.

If extraction of the item name and item value fails in step S1309, the recognition processing unit 270 notifies the terminal device 400 or the image forming apparatus 300 of the error (step S1310), and proceeds to step S1313, which will be described later.

If the item name and item value are successfully extracted in step S1309, the recognition processing unit 270 generates recognition result data in which the extracted item name and item value are associated and merged (step S1311).

Next, the recognition processing unit 270 causes the job list generation unit 272 to register the definition ID of the form definition information used to generate the recognition result data and the storage location of the recognition result data in the job list 101 (step S1313).

Subsequently, the recognition processing unit 270 determines whether the processing has been executed up to the last page (step S1313). In step S1313, if the process has not been executed up to the last page, the recognition processing unit 270 returns to step S1303.

In step S1313, when the process has been executed up to the last page, the recognition processing unit 270 causes the display screen generation unit 279 to display on the terminal device 400 a recognition result confirmation screen generated using the recognition result data (step S1314), the process ends.

Note that the recognition processing unit 270 may display the recognition result data after receiving a request to view the recognition result data from the terminal device 400.

Next, with reference to FIG. 14, the processing of the definition specifying unit 276, item extracting unit 277, and item value determining unit 278 of this embodiment will be described. FIG. 14 is a second flowchart illustrating the processing of the form recognition unit. FIG. 14 shows details of the process of step S1308 in FIG. 13.

In the recognition processing unit 270 of this embodiment, the definition specifying unit 276 specifies individual company definition information to be used for form recognition from among the individual company definition information stored in the form definition database 250 from the result of character recognition (step S1401).

Specifically, the definition specifying unit 276 determines whether two or more item values of the issuer identification information associated with the individual company definition information among the individual company definition information stored in the form definition database 250 are readable. Identify individual company-defined information that completely matches the character string included in the result data. In other words, the definition specifying unit 276 makes sure that the character string included in the reading result data and the item values of two or more of the items "issuer", "telephone number", and "account number" are completely different from each other. Identify matching company definition information.

Subsequently, the recognition processing unit 270 determines whether the definition identification unit 276 has identified individual company definition information (step S1402). If the individual company definition information is specified in step S1402, the recognition processing unit 270 proceeds to step S1408 and step S1410, which will be described later. At this time, the recognition processing unit 270 temporarily holds information indicating that the individual company definition information has been specified based on the result of character recognition.

If the individual company definition information is not specified in step S1402, the recognition processing unit 270 causes the definition identification unit 276 to specify the individual company definition information based on the format of the form (step S1403).

Specifically, the definition specifying unit 276 refers to the format management database 265 and determines whether the width of the maximum rectangle and the color of the ruled line match the width of the maximum rectangle and the color of the ruled line obtained from the reading result data, and if the character The format information for which the matching rate of is equal to or higher than the threshold value is identified. Next, the item value determining unit 278 identifies the identification information of the issuer that corresponds to the specified format information, and refers to the form definition database 250 to identify the individual company definition information that is associated with the identification information of the issuer. do.

Subsequently, the recognition processing unit 270 determines whether individual company definition information has been identified (step S1404). If the individual company definition information is specified in step S1404, the recognition processing unit 270 proceeds to step S1408 and step S1410, which will be described later. At this time, the recognition processing unit 270 temporarily holds information indicating that the individual company definition information has been specified based on the format of the form.

If the individual company definition information is not specified in step S1404, the recognition processing unit 270 uses the definition identification unit 276 to refer to the form definition database 250 and obtains general definition information (step S1405).

Specifically, the definition specifying unit 276 determines the document type of the form from the character string included in the read result data, and acquires general-purpose definition information corresponding to the determined document type from the form definition database 250.

Note that the case where the individual company definition information is not specified in step S1402 and step S1404 means that the corresponding individual company definition information has not been created and the corresponding individual company definition information is not stored in the form definition database 250. shows.

Subsequently, the recognition processing unit 270 causes the item extraction unit 277 to extract the item name and item value using the general-purpose definition information acquired in step S1405 (step S1406). Subsequently, the recognition processing unit 270 retains data in which the extracted item values and item names are associated with each other (step S1407), and ends the process.

When the individual company definition information is specified in step S1402 or step S1404, the recognition processing unit 270 uses the definition specifying unit 276 to acquire the specified individual company definition information from the form definition database 250 (step S1408). Next, in the recognition processing unit 270, the item extraction unit 277 extracts the item value of the item name "Issuer" using the acquired individual company definition information (step S1409), and proceeds to step S1412, which will be described later.

Furthermore, the recognition processing unit 270 uses the definition specifying unit 276 to specify general-purpose definition information and acquires it from the form definition database 250 (step S1410). Next, in the recognition processing unit 270, the item extraction unit 277 extracts the item value of the item name "publisher" using the acquired general-purpose definition information (step S1411), and proceeds to step S1412, which will be described later.

Subsequently, the recognition processing unit 270 determines, by the item value determination unit 278, whether the individual company definition information is specified based on the format of the form (step S1412). In other words, the item value determination unit 278 determines whether the recognition processing unit 270 holds information indicating that the individual company definition information has been specified based on the format of the form.

In step S1412, if the individual company definition information is not specified based on the format, the item value determining unit 278 determines the item value of the item name "Publisher" to be the item value extracted in step S1409, and other items Extract the item value of the name using the individual company definition information. The item value determining unit 278 then holds data that associates item names and item values (step S1413), and ends the process.

Note that in step S1412, the case where the individual company definition information is not specified based on the format is the case where the individual company definition information is specified based on the result of character recognition.

In step S1412, if the individual company definition information is specified based on the format, the recognition processing unit 270 causes the item value determination unit 278 to add the publisher extracted using the individual company definition information to the corporate name dictionary database 267. It is determined whether it is registered (step S1414).

Specifically, the item value determining unit 278 determines whether a corporate name that matches the name of the issuer extracted in step S1409 is stored in the corporate name dictionary database 267.

In step S1414, if the issuer extracted using the individual company definition information is stored in the corporate name dictionary database 267, the recognition processing unit 270 proceeds to step S1413.

If the issuer extracted using the individual company definition information is stored in the corporate name dictionary database 267, this indicates that the reliability of the individual company definition information identified in step S1408 is relatively high. Therefore, in this embodiment, although the individual company definition information is specified based on the format in this case, it is assumed that sufficient reliability is ensured when used for form recognition, and this individual company definition information is specified based on the format. Extract item values for other item names using company-defined information.

Further, in step S1414, if the publisher extracted using the individual company definition information is not stored in the corporate name dictionary database 267, the item value determination unit 278 sets the item value of the item name "Publisher" to The item values extracted using the general-purpose definition information are determined, and the item values of other item names are determined to be the item values extracted using the individual company definition information. Then, the item value determination unit 278 temporarily holds data in which the item name and the item value are associated with each other (step S1415), and ends the process.

In this way, in this embodiment, when the individual company-defined information used for form recognition is specified based on the result of character recognition, the item value of the item name "Issuer" is extracted based on the individual company-defined information. The item value is determined. In addition, in this embodiment, the individual company definition information used for form recognition is specified based on the format, and the item value of the item name "Issuer" extracted using the individual company definition information is the corporate name. If it is stored in the dictionary database 267, the item value of the item name "Publisher" is determined to be the item value extracted based on the individual company definition information.

Furthermore, in this embodiment, the individual company definition information used for form recognition is specified based on the format, and the item value of the item name "Issuer" extracted using the individual company definition information is the corporate name. If it is not stored in the dictionary database 267, the item value of the item name "Publisher" is determined to be the item value extracted based on the general-purpose definition information.

In other words, in this embodiment, when it is estimated that the specified individual company definition information has sufficient reliability when used for form recognition, this individual company definition information is used to identify all items. The name and item value are extracted and used as recognition result data.

In addition, in this embodiment, if the general-purpose definition information is estimated to be more reliable than the specified individual company-defined information, the item value of the item "Publisher" is changed using the general-purpose definition information. The item value is determined based on the extracted item value. Note that in this case, item values of other item names are extracted using individual company definition information that has a higher priority than general definition information.

As described above, according to the present embodiment, it is possible to suppress the occurrence of misrecognition of a form due to the way in which individual company definition information is specified when performing form recognition, and it is possible to improve the accuracy of form recognition.

Next, with reference to FIG. 15, registration of management information by the form management section 290 will be described. FIG. 15 is a flowchart illustrating the processing of the form management section. FIG. 15 shows details of the processing from step S1121 to step S1131 in FIG. 11.

When the operation reception unit 291 receives an instruction to view recognition result data from the terminal device 400, the form management unit 290 of the present embodiment causes the display control unit 292 to display a recognition result data confirmation screen on the terminal device 400. (Step S1501).

Subsequently, the form management unit 290 determines whether the operation reception unit 291 has received a registration request for recognition result data from the terminal device 400 (step S1502). Specifically, the server device 200 determines whether an instruction has been given to store the recognition result data displayed on the confirmation screen in the recognition result database 260.

In step S1502, if the registration request is not accepted, the form management unit 290 proceeds to step S1507, which will be described later.

If the registration request is received in step S1502, the form management unit 290 determines whether or not the recognition result data has been modified (step S1503).

In step S1503, if no correction has been made, the form management unit 290 ends the process.

If a modification has been made in step S1503, the form management unit 290 determines whether the individual company definition information 252 is referred to when generating this recognition result data (step S1504). Specifically, the form management unit 290 determines whether or not there is individual company definition information 252 associated with the issuer included in the read result data.

In step S1504, if the individual company definition information is not referenced, the form management unit 290 proceeds to step S1508, which will be described later.

If the individual company definition information is referenced in step S1504, the form management unit 290 determines whether the revised location is the issuer (step S1505).

In step S1505, if the revised part is not the issuer, the form management unit 290 causes the information update unit 293 to update the individual company definition information according to the revised content (step S1506), and ends the process. The case where the corrected part is not the issuer is, for example, a case where the details part in the recognition result data is corrected.

Note that in this embodiment, the recognition result data may be registered after the processing by the form management unit 290 is completed. Specifically, the form management unit 290 may store the recognition result data in the recognition result database 260 after the process of FIG. 15 is completed.

In step S1505, if the corrected part is the issuer, the form management unit 290 causes the display control unit 292 to display an input screen for the identification information of the issuer on the terminal device 400 (step S1507).

Subsequently, the form management unit 290 receives input of information on the input screen displayed on the terminal device 400 through the operation reception unit 291 (step S1508).

Subsequently, the form management unit 290 causes the information update unit 293 to update the identification information of the issuer in the format management database 265 and the identification information of the issuer associated with the individual company definition information 252 in the form definition database 250 ( overwrite) (step S1509), and the process ends.

If the recognition result registration request is not accepted in step S1502, the form management unit 290 determines whether the operation reception unit 291 has received a form definition registration instruction (step S1510).

In step S1510, if the instruction to register the form definition is not accepted, the form management unit 290 returns to step S1501.

In step S1501, when an instruction to register a form definition is received, the form management unit 290 causes the display control unit 292 to display an input screen for the individual company definition information 252 on the terminal device 400 (step S1511).

Subsequently, the form management unit 290 receives input of individual company definition information on the input screen through the operation reception unit 291 (step S1512).

Subsequently, the form management section 290 causes the information update section 293 to store the input company definition information 252 in the form definition database 250 (step S1513), and ends the process.

In this embodiment, through the above-described processing, newly created individual company definition information can be stored in the form definition database 250, and issuer identification information can be corrected.

Next, the effects of this embodiment will be described with reference to FIGS. 16 and 17. FIG. 16 is a first diagram illustrating the effects of this embodiment.

FIG. 16 shows an example of a recognition result confirmation screen when the present embodiment is not applied and form recognition is performed using individual company definition information specified based on the form format.

A screen 501 shown in FIG. 16 is an example of a recognition result confirmation screen displayed on the terminal device 400.

Screen 501 includes display field 502, display field 503, and display components 508 and 509. In the display field 502, a form image indicated by the form image data is displayed. In the example of FIG. 16, the form is an invoice, and includes an area 504 indicating a detailed part in the form image.

In the form image displayed in the display field 502, the invoice number, closing date, total invoice amount, etc. are arranged in the upper left. An area 502a indicating the billing source (issuer) is arranged at the upper right. Further, in the display field 502, the detailed portion of the form image is displayed in a table format.

The display column 503 displays recognition result data as a result of the document recognition unit 220 performing document recognition on the document image data indicating the document image displayed in the display column 502.

Display field 503 includes display fields 505, 506, and 507. In the display column 505, the value of the item "Status" of the job list 101 is displayed. In the display column 506, billing source information including the billing source is displayed. The billing source information is part of the recognition result data.

In the display column 507, detailed information is displayed. Specifically, the display field 507 displays a list of item values included in the detailed information. Display field 507 includes display fields 511, 512, 513, 514, 515, and 516.

In the display column 511, the item value of the item name "product name" displayed in the area 504 indicating the detailed part in the form image is displayed. In the display column 512, the item value of the item name "product code" displayed in the area 504 is displayed. In the display field 513, the item value of the item name "Product Price" displayed in the area 504 is displayed.

In the display field 514, the item value of the item name "unit price" displayed in the area 504 is displayed. In the display field 515, the item value of the item name "Quantity" displayed in the area 504 is displayed. In the display field 516, the item value of the item name "Order Number" displayed in the area 504 is displayed.

In this way, on the screen 501, item values corresponding to each item name in the details part of the form image are extracted and displayed in display columns 511, 512, 513, 514, 515, and 516.

A display component 508 is a display component for changing the status value of the job list 101 from an unprocessed state to a draft saved state. When the display component 508 is operated, the display field 505 changes from "unprocessed" to "save draft".

A display component 509 is a display component for setting the status value of the job list 101 to a confirmed state. On the screen 501, when the display component 509 is operated, the status value of the job list 101 becomes a confirmed state.

Here, the billing source shown in FIG. 16 will be explained. The display field 506 of the screen 501 shown in FIG. 16 includes a display field 506a.

The display field 506a displays the item value of the item "billing source" extracted using the individual company definition information specified based on the format of the form image displayed in the display field 502.

In the display field 506a, "A Corporation" is displayed as the item value of the billing party. On the other hand, in the area 502a indicating the billing source in the form image displayed in the display field 502, "R Corporation" is displayed, and the two do not match.

For this reason, in the example of FIG. 16, the billing source's item value is incorrectly recognized during form recognition.

This misrecognition is due to the fact that the format indicated by the format information corresponding to the billing party "A Corporation" is similar to the format of the form image displayed in the display field 502.

In other words, in FIG. 16, the billing party "A Corporation" is identified from the format of the form image displayed in the display field 502, and the bill is recognized using the individual company definition information corresponding to the billing party "A Corporation". This is an example in which the billing party was misrecognized as a result of this.

In contrast, a case where this embodiment is applied will be described with reference to FIG. 17. FIG. 17 is a second diagram illustrating the effects of this embodiment.

In the example of FIG. 17 as well, it is assumed that the individual company definition information of the issuer "A Corporation" is specified based on the format of the form image displayed in the display field 502.

In this case, in this embodiment, since the individual company definition information is specified based on the format, the item value "A Corporation" of the item name "Billing source" extracted using the individual company definition information is Determine whether it is stored in .

Here, it is assumed that the item value "A Corporation" is not stored in the corporate name dictionary database 267. In that case, in this embodiment, the item value of the item name "Billing source" is extracted from the form image data using the general definition information acquired from the form definition database 250 together with the individual company definition information, and the item value of the item name "Billing source" is extracted from the form image data. This is associated with the recognition result data.

As a result, the item value "R Corporation" extracted using the general-purpose definition information is displayed in the display field 506b of the screen 501A in association with the item name "Billing source."

In this embodiment, in this way, the item value of the issuer of a form is determined according to the method of specifying the individual company-defined information used for form recognition. The occurrence of misrecognition can be suppressed.

Each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" as used herein refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function explained above. This includes devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules.

Additionally, the devices described in the embodiments are merely illustrative of one of a plurality of computing environments for implementing the embodiments disclosed herein.

In some embodiments, server apparatus 200 includes multiple computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network, shared memory, etc., to perform the processes disclosed herein. Similarly, server apparatus 200 may include multiple computing devices configured to communicate with each other.

Further, the server device 200 can be configured to share the disclosed processing steps in various combinations. For example, processes executed by server device 200 may be executed by other server devices. Similarly, the functions of server device 200 can be performed by other server devices. Further, each element of the server device and other server devices may be combined into one server device, or may be divided into a plurality of devices.

Moreover, the database in the specification may be generated by the learning effect of machine learning. Furthermore, by using machine learning to classify keywords and account items that may be included in transaction descriptions, there is no need to use a correspondence table.

Here, machine learning is a technology that allows computers to acquire human-like learning abilities, and computers autonomously generate algorithms necessary for decisions such as data identification from learning data that has been captured in advance. This refers to a technology that applies this to new data to make predictions. The learning method for machine learning may be supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, or deep learning, or it may be a learning method that combines these learning methods. It doesn't matter what learning method you use.

Although the present invention has been described above based on each embodiment, the present invention is not limited to the requirements shown in the above embodiments. These points can be changed without detracting from the gist of the present invention, and can be determined appropriately depending on the application thereof.

100 Form recognition system 200 Server device 210 Storage unit 220 Form recognition unit 230 Form image database 240 Type determination database 250 Form definition database 260 Recognition result database 265 Format management database 267 Corporate name dictionary database 270 Recognition processing unit 290 Form management unit 300 Image formation Device 400 Terminal device 500 Core system

Japanese Patent Application Publication No. 2004-158030 Japanese Patent Application Publication No. 2016-51339

Claims (8)

A specific unit that accepts input of form image data, refers to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and identifies form definition information to be used for form recognition. and,
an item value determining unit that determines an item value of a specific item name among item names included in the form that is the source of the form image data, depending on how the specifying unit specifies the form definition information; have,
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The item value determining unit is
determining the item value of the specific item name depending on how the second form definition information is specified by the specifying unit;
The item value determining unit is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
an information processing device that determines the item value of the specific item name to be a value extracted based on the first form definition information when the item value is specified according to the format of the form that is the source of the form image data; . A specific unit that accepts input of form image data, refers to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and identifies form definition information to be used for form recognition. and,
an item value determining unit that determines an item value of a specific item name among item names included in the form that is the source of the form image data, depending on how the specifying unit specifies the form definition information; have,
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The item value determining unit is
determining the item value of the specific item name depending on how the second form definition information is specified by the specifying unit;
The item value determining unit is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in a predetermined dictionary database. In this case, the item value of the specific item name is determined to be a value extracted based on the second form definition information,
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in the predetermined dictionary database. If there is no item, the information processing device determines the item value of the specific item name to be a value extracted based on the first form definition information . Specific processing that accepts input of form image data, refers to a storage unit that stores multiple pieces of form definition information that defines the positional relationships of item names and item values included in the form, and identifies form definition information to be used for form recognition . and,
information processing for determining an item value of a specific item name among item names included in the form that is the source of the form image data, depending on how the form definition information is specified by the identification process; A program that is executed by a device,
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The determination process is
determining the item value of the specific item name according to how the second form definition information is specified by the specifying process;
The determination process is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
A program that determines an item value of the specific item name to a value extracted based on the first form definition information when specified according to a format of a form that is the source of the form image data. Specific processing that accepts input of form image data, refers to a storage unit that stores multiple pieces of form definition information that defines the positional relationships of item names and item values included in the form, and identifies form definition information to be used for form recognition . and,
information processing for determining an item value of a specific item name among item names included in the form that is the source of the form image data, depending on how the form definition information is specified by the identification process; A program that is executed by a device,
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The determination process is
determining the item value of the specific item name according to how the second form definition information is specified by the specifying process;
The determination process is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in a predetermined dictionary database. In this case, the item value of the specific item name is determined to be a value extracted based on the second form definition information,
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in the predetermined dictionary database. If there is no item, the program determines the item value of the specific item name to a value extracted based on the first form definition information . An information processing system in which a terminal device and an information processing device are connected via a network,
The information processing device includes:
A specific unit that accepts input of form image data, refers to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and identifies form definition information to be used for form recognition. and,
an item value determining unit that determines the item value of a specific item name among item names included in the form that is the source of the form image data, depending on how the specifying unit specifies the form definition information;
an output unit that transmits recognition result data in which the item name determined by the item value determining unit and the item value are associated with each other to the terminal device;
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The item value determining unit is
determining the item value of the specific item name depending on how the second form definition information is specified by the specifying unit;
The item value determining unit is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
determining the item value of the specific item name to be a value extracted based on the first form definition information when specified according to the format of the form that is the source of the form image data;
The terminal device is
a communication unit that receives the recognition result data from the information processing device;
An information processing system, comprising: a display control unit that displays the recognition result data on a display device. An information processing system in which a terminal device and an information processing device are connected via a network,
The information processing device includes:
A specific unit that accepts input of form image data, refers to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and identifies form definition information to be used for form recognition. and,
an item value determining unit that determines the item value of a specific item name among item names included in the form that is the source of the form image data, depending on how the specifying unit specifies the form definition information;
an output unit that transmits recognition result data in which the item name determined by the item value determining unit and the item value are associated with each other to the terminal device,
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The item value determining unit is
determining the item value of the specific item name depending on how the second form definition information is specified by the specifying unit;
The item value determining unit is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in a predetermined dictionary database. In this case, the item value of the specific item name is determined to be a value extracted based on the second form definition information,
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in the predetermined dictionary database. If not, determining the item value of the specific item name to a value extracted based on the first form definition information,
The terminal device is
a communication unit that receives the recognition result data from the information processing device;
An information processing system, comprising: a display control unit that displays the recognition result data on a display device. An information processing method using an information processing system in which a terminal device and an information processing device are connected via a network,
The information processing device
A specific procedure for accepting input of form image data, referring to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and identifying form definition information to be used for form recognition . and,
a determining step of determining an item value of a specific item name among the item names included in the form that is the source of the form image data, depending on how the form definition information is specified by the specifying step ;
performing a transmission procedure of transmitting recognition result data in which the determined item name and the item value are associated with each other to the terminal device ;
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The determination procedure is
determining the item value of the specific item name according to how the second form definition information is specified by the specifying procedure;
The determination procedure is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
determining the item value of the specific item name to be a value extracted based on the first form definition information when specified according to the format of the form that is the source of the form image data;
The terminal device is
receiving the recognition result data from the information processing device;
An information processing method that displays the recognition result data on a display device. An information processing method using an information processing system in which a terminal device and an information processing device are connected via a network,
The information processing device
A specific procedure for accepting input of form image data, referring to a storage unit storing a plurality of form definition information that defines the positional relationships of item names and item values included in the form, and identifying form definition information to be used for form recognition. and,
a determining step of determining an item value of a specific item name among the item names included in the form that is the source of the form image data, depending on how the form definition information is specified by the specifying step ;
performing a transmission procedure of transmitting recognition result data in which the determined item name and the item value are associated with each other to the terminal device ;
The form definition information includes first form definition information and a second form definition that defines the positional relationship between item names and item values of a user-specific form, which has a higher priority than the first form definition information. including information and;
The determination procedure is
determining the item value of the specific item name according to how the second form definition information is specified by the specifying procedure;
The determination procedure is
The second form definition information is
determining the item value of the specific item name to be a value extracted based on the second form definition information when specified according to a character string included in a form image indicated by the form image data;
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in a predetermined dictionary database. In this case, the item value of the specific item name is determined to be a value extracted based on the second form definition information,
The item value of the specific item name specified according to the format of the form that is the source of the form image data and extracted based on the second form definition information is stored in the predetermined dictionary database. If not, determining the item value of the specific item name to a value extracted based on the first form definition information,
The terminal device is
receiving the recognition result data from the information processing device;
An information processing method that displays the recognition result data on a display device.

Images