JP7304604B1 - How to support data entry for forms - Google Patents

Abstract

Kind Code: A1 An object of the present invention is to provide a method for further improving work efficiency in inputting data for a form such as a health checkup form. In the method, a control unit of a server terminal recognizes a plurality of input items included in the image data and input values of each of the plurality of input items by analyzing the image data of the form, displays the recognized input items and the input values as input data, receives a request for extracting the reference data of the form from the user terminal, extracts the reference data based on the matching degree of the input items of the form in response to the extraction request, and displays the extracted one or more reference data. [Selection drawing] Fig. 6

Description

The present invention mainly relates to a method of assisting data input such as health examination forms.

2. Description of the Related Art In recent years, an automatic data input method using OCR technology has become popular in order to improve work efficiency of data input for forms such as health checkups.

For example, Japanese Patent Laid-Open No. 2002-200003 discloses a technique that enables accurate storage and management of data by recognizing data written on a medical checkup sheet by processing such as OCR.

Japanese Patent Application Laid-Open No. 2022-64870

However, the technique disclosed in Patent Document 1 displays the character recognition result and recognition rate by OCR for input items, and stores the character change result by the user. Workload challenges can still arise as data entry is required for items. Moreover, in particular, since many formats are provided as forms for health checkups, there is also the problem that the accuracy of the recognition rate for linking character strings recognized as input values for input items is low.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a technique for further improving work efficiency in inputting data for a form such as a medical examination form.

According to one aspect of the present invention, there is provided a method for assisting data entry of a form, wherein a control unit of a server terminal analyzes image data of a form to determine a plurality of input items included in the image data and the plurality of input items. Recognizing the input value of each input item, displaying the recognized input item and the input value as input data, receiving a request to extract the reference data of the form from the user terminal, and responding to the extraction request, Reference data is extracted based on the matching degree of the input items of the form, and the extracted one or more reference data is displayed.

According to the present invention, it is possible to provide a technique for further improving work efficiency when inputting data for a form such as a health checkup form.

1 is a block configuration diagram showing a data input support system according to a first embodiment of the present invention; FIG. 2 is a functional block configuration diagram showing a server terminal 100 of FIG. 1; FIG. 2 is a functional block configuration diagram showing a user terminal 200 of FIG. 1; FIG. 4 is a diagram showing an example of user data stored in the server 100; FIG. 4 is a diagram showing an example of form data stored in the server 100; FIG. It is a flow chart which shows an example of processing of a data entry support method concerning a first embodiment of the present invention. 9 is a flow chart showing another example of processing of the data input support method according to the first embodiment of the present invention; FIG. 4 is a diagram illustrating an example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention; FIG. 7 is a diagram illustrating another example of a user interface screen displayed on the user terminal according to the first embodiment of the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below do not unduly limit the scope of the invention described in the claims. Moreover, not all the constituent elements shown in the embodiments are essential constituent elements of the present invention.

<Configuration>
FIG. 1 is a block configuration diagram showing a data input support system according to the first embodiment of the present invention. For example, the system 1 performs character recognition on a form such as a health checkup sheet photographed or scanned by a user by image processing such as OCR, stores the data in a storage unit as input data, and further performs character recognition processing based on reference data. It includes a server terminal 100 that executes data input by a user and stores it as final input data, and user terminals 200A and 200B for users who perform the above-described photographing of forms such as employee health checkups. For convenience of explanation, each terminal is described as one or a specific number, but each number is not limited.

The server terminal 100, user terminals 200A and 200B, and company terminals 300A and 300B are each connected via a network NW1. The network NW includes the Internet, an intranet, a wireless LAN (Local Area Network), a WAN (Wide Area Network), and the like.

The server terminal 100 may be, for example, a general-purpose computer such as a workstation or personal computer, or may be logically realized by cloud computing.

The user terminals 200A and 200B are, for example, information processing devices such as personal computers and tablet terminals, but may be configured by smartphones, mobile phones, PDAs, and the like. Further, the user terminals 200A and 200B can also be connected to a scanner for scanning forms such as health checkups via a network such as the Internet or short-range wireless communication.

In this embodiment, the system 1 includes a server terminal 100 and user terminals 200A and 200B, and the user of each terminal uses each terminal to operate the server terminal 100. However, the server terminal 100 may be configured as a stand-alone, and the server terminal itself may be provided with a function for direct operation by each user. Hereinafter, the user terminals 200A and 200B will be collectively referred to as the user terminal 200 as required.

FIG. 2 is a functional block configuration diagram of the server terminal 100 of FIG. The server terminal 100 includes a communication section 110 , a storage section 120 and a control section 130 .

The communication unit 110 is a communication interface for communicating with the user terminal 200 via the network NW1, and communication is performed according to a communication protocol such as TCP/IP (Transmission Control Protocol/Internet Protocol).

The storage unit 120 stores programs, input data, and the like for executing various control processes and functions in the control unit 130, and is composed of a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. be. The storage unit 120 also includes a user data storage unit 121 that stores various data related to a user, and a form data storage unit 122 that stores various data related to a form such as a health checkup form provided by a user. have. A database (not shown) storing various data may be constructed outside the storage unit 120 or the server terminal 100 .

The control unit 130 controls the overall operation of the server terminal 100 by executing a program stored in the storage unit 120, and is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like. be done. The functions of the control unit 130 include an information reception unit 131 that receives information from various terminals, an image processing unit 132 that performs image processing such as OCR on image data related to forms such as health checkup sheets, and a user and a form data processing unit 133 that refers to and processes reference data data including forms such as provided health checkup forms. The information reception unit 131, the image processing unit 132, and the form data processing unit 133 are activated by a program stored in the storage unit 120 and executed by the server terminal 100, which is a computer (electronic calculator).

Information reception unit 131 receives information from user terminal 200 via communication unit 110 . For example, the user terminal 200 receives image data related to a form generated by photographing a form such as a health checkup form.

The image processing unit 132 performs image processing such as OCR on the image data of the form received from the user terminal 200, and performs character recognition.

The form data processing unit 133 refers to the form data related to the form received from the user terminal 200 and the reference data stored in the form data storage unit 122, and performs predetermined processing.

The control unit 130 can also have a screen generation unit (not shown), and generates screen information to be displayed via the user interface of the user terminal 200 upon request. For example, using image and text data (not shown) stored in the storage unit 120 as materials, various images and text are arranged in predetermined areas of the user interface based on predetermined layout rules to generate a user interface. do. Processing related to the image generator can also be executed by a GPU (Graphics Processing Unit).

FIG. 3 is a functional block configuration diagram showing the user terminal 200 of FIG. The user terminal 200 includes a communication section 210 , a display operation section 220 , a storage section 230 and a control section 240 .

The communication unit 210 is a communication interface for communicating with the server terminal 100 via the network NW, and communication is performed according to a communication protocol such as TCP/IP.

The display operation unit 220 is a user interface used by the user to input instructions and display text, images, etc. according to input data from the control unit 240, and the user terminal 200 is configured by a personal computer. When the user terminal 200 is a smart phone or a tablet terminal, it is composed of a touch panel or the like. The display operation unit 220 is activated by a control program stored in the storage unit 230 and executed by the user terminal 200, which is a computer (electronic calculator).

The storage unit 230 stores programs, input data, and the like for executing various control processes and functions in the control unit 240, and is composed of RAM, ROM, and the like. Further, the storage unit 230 temporarily stores communication contents with the server terminal 100 .

The control unit 240 controls the overall operation of the user terminal 200 by executing programs stored in the storage unit 230, and is composed of a CPU, a GPU, and the like.

Camera 250 photographs a form such as a medical checkup form, and control unit 240 stores the photographed form as image data in storage unit 230 and/or transmits the image data to server terminal 100 via communication unit 210 . The user terminal 200 can also be connected to a scanner (not shown) as an external device via the communication unit 210. The scanner scans a form such as a medical checkup sheet, and the user terminal 200 transmits the scanned image data. It is also possible for the user terminal 200 to receive the image data and transmit the image data to the server terminal 100 .

FIG. 4 is a diagram showing an example of user data stored in the server 100. As shown in FIG.

User data 1000 shown in FIG. 4 stores various data related to the user. FIG. 4 shows an example of one user (the user identified by the user ID "10001") for convenience of explanation, but information of a plurality of users can be stored. As various data related to users, for example, basic information about users (for example, company name, address, name of person in charge, department name, contact information, etc.), report data such as health checkup slips (each employee corresponding to the health checkup slip, etc.) employee ID, image data generated by image processing the form, structured input data including each input item and input value included in the form), but not limited to this.

FIG. 5 is a diagram showing an example of form data stored in the server 100. As shown in FIG.

The form data 2000 stores various data related to the form provided by the user. In particular, with respect to health checkup forms, where there are health checkup forms created in multiple patterns of formats, the health checkup forms provided by the user are accumulated as form data in each format, and later another health checkup form, etc. is stored. When character recognition is performed on a form, it can be managed so that it can be referred to as reference data during data structuring processing, including linking character strings as recognized input values to input items. FIG. 5 shows an example of one form (a form identified by the form ID "20001") for convenience of explanation, but information related to a plurality of forms can be stored. Here, it is possible to store the form data as reference data in units of forms, or to reduce the capacity of the storage area, it is also possible to store the form data in units of form formats. In addition, since the form data is managed regardless of a specific user or a user belonging to a specific company, it is preferable that the form data be stored in a format in which personal information is deleted or not displayed. Various data related to a form, such as image data generated by photographing the form, and input items included in the form generated by character recognition by performing image processing such as OCR on the image data. and input values, including but not limited to structured input data. In addition, as the form data, each form based on the form stored as the form data, including the form data including the reference data necessary for executing the character recognition at the time of the OCR processing of the form such as the health checkup sheet. It is also possible to store a learning model generated by learning input items and position coordinates of input values and character recognition results included in the data.

FIG. 6 is a flow chart showing an example of processing of the data input support method according to the first embodiment of the present invention.

First, as pre-processing in step S101, a user such as a person in charge of a company sends a health checkup sheet of an employee of the company to the camera built in the user terminal 200, or to the user terminal 200 via the Internet or short-range wireless communication. A scanner connected via a network such as a scanner takes an image (or scans) and transmits (uploads) it to the server terminal 100 as image data. The information reception unit 131 of the control unit 130 of the server terminal 100 receives the image data of the medical checkup sheet transmitted from the user terminal 200 via the communication unit 110, and the form data processing unit 133 receives the received image data. , the image data transmitted from the user terminal 200 is stored in the user data storage unit 121 of the storage unit 120 together with data such as the employee ID corresponding to the image data transmitted from the user terminal 200 as form data of the user data.

Next, as the process of step S101, the user requests reading of the form data via the user interface screen displayed on the user terminal 200, and the form data processing unit 133 of the control unit 130 of the server terminal 100 In response to a read request from the terminal 200, the user data storage unit 121 of the storage unit 120 is referenced, and the relevant form data stored in the user data storage unit 121 is read.

For example, as shown in FIG. 8, the user requests reading of the file via the user interface screen displayed on the user terminal 200 and instructs to read the image data of the transmitted health checkup sheet.

Subsequently, as the process of step S102, the form data processing unit 133 of the control unit 130 of the server terminal 100 refers to the user data storage unit 121 of the storage unit 120, reads the requested image data, and 132 executes image processing by OCR on the read image data. For example, the image processing unit 132 determines a predetermined upper left position of the read image data of the medical checkup form as the vertex, sets the vertex as the zero coordinate, and sets the X axis in the horizontal and vertical directions with the zero coordinate as the base point. , a rectangular area defined by coordinate points on the Y-axis (for example, the area described as "height" and "177.0" on the image data of the health checkup sheet shown in FIG. 9) as a character recognition target area by OCR. Character recognition processing is executed in the character recognition target area corresponding to each input item and input value on the image data. In addition, during this character recognition, by machine learning based on the above learning model, the relationship (relative coordinate ) can also be determined. FIG. 9 shows the results of character recognition processing in each character recognition target area. An input item and an input value corresponding to the input item are recognized as characters, such as "177.0". By OCR processing, for example, a rectangle of "height" is searched, the relative distance from the rectangle and the coordinates of the input value are estimated, and based on the estimation, the character string (input The input value (or value) of "177.0" can be obtained from the item or key) and determined as the input value of "177.0" corresponding to the input item of "height". The form data processing unit 133 uses the input items and input values of the recognized form as input data, and converts the form data (input data). Here, the form data processing unit 133 uses the form data stored in the user data storage unit 121 as reference data, which will be described later, when the user or another user performs input processing for another health checkup form. It can also be stored in the form data storage unit 122 so that it is possible to refer to whether the input values corresponding to the items and the input items match. The form data processing unit 133 also causes the user terminal 200 to display the input data as the character recognition result, as shown in FIG. Here, there are input items and/or input values that are not recognized by OCR, and in the input values, those recognized in a format different from the format (numerical values, etc.) set in advance based on the rule base ( For example, the form data processing unit 133 determines that an input item “visual acuity” includes “(0.70)” and a character string other than a numerical value) as an error, and the user terminal 200 can be identified and notified of an alert. In addition, the form data processing unit 133 masks and displays input items and input values that have been correctly recognized and determined to have been input, and displays only input items and input values that are determined to be in error. can also

When input data is normally generated as a result of character recognition, the user can save the structured input data as final data without inputting corrections, etc., to complete a series of processes. However, as described above, if there are unrecognized input items and/or input values, or if an error is notified, enter additional data or correct it by manual or automatic input However, such manual input requires time and effort. In such a case, the user refers to the form data having the input data of the health checkup form in a format similar to the health checkup form read this time as reference data, and based on the reference data, the input items in the form data and corresponding input items. In the sense that the input value to be read is acquired as structured data, it is possible to acquire the input data of the health checkup sheet to be read this time while improving the accuracy of character recognition.

As described above, as the process of step S103, for example, as shown in FIG. 10, the user selects "reference search" on the user interface screen displayed on the user terminal 200, thereby requesting extraction of reference data. , information reception unit 131 of control unit 130 of server terminal 100 receives a reference data extraction request via communication unit 110 .

Subsequently, as the process of step S104, the form data processing unit 133 of the control unit 130 of the server terminal 100 searches for the form data of the health checkup form similar to the input data of the health checkup form to be read this time. Extract multiple form data as reference data. As shown in FIG. 11, as a reference list, reference data corresponding to the form ID are included in the input items (Ref) included in the reference data and the input items (Key ) has a high matching ratio (matching rate) or a large number of matching input items (keys), and/or the number of input items included in the form to be compared as the format of the health checkup form. Those having a close positional relationship (image data of each form are vectorized, and as a result of comparison, those having a short distance) are extracted. Here, when extracting the reference data, it is also possible to extract the reference data for each page of the health checkup sheet to be read this time. For example, the health checkup that is an option in the health checkup may have substantial content on subsequent pages, and in such a case, similar forms can be extracted for each page. In the example of FIG. 11, the form with the form ID "820" is extracted as the form with the highest matching rate of the input items and the shortest distance between the images (closest format). It is also possible to allow the user to input the distance between images (“D threshold” in the example of FIG. 11), and extract image data by automatic selection of image data based on the distance.

Subsequently, as the process of step S105, as shown in FIG. 12, the user selects the desired reference data among the plurality of reference data displayed in the list format in order of distance on the user interface screen displayed on the user terminal 200. When (for example, the most similar) reference data is selected (reference data identified by the form ID “820”), the information reception unit 131 of the control unit 130 of the server terminal 100 receives the reference data via the communication unit 110. Accept selection requests.

FIG. 7 is a flow chart showing another example of the processing of the data input support method according to the first embodiment of the present invention, in particular, the analysis processing of the form data of the health checkup slip to be read based on the reference data. and input support processing.

As the process of step S201, the image processing unit 132 of the control unit 130 of the server terminal 100 refers to the reference data extracted based on the reference data selection request, and reads the form data of the health checkup slip to be read. , performs image analysis processing by OCR. As described above, the input data of the health checkup sheet to be read and the input data of the reference data have a high matching rate. It is possible to perform character recognition with comparatively high accuracy.

Subsequently, as the process of step S202, the form data processing unit 133 of the control unit 130 of the server terminal 100 converts the image analysis result into character-recognized image data, input items, and input values of the health checkup sheet to be read. to be displayed as structured data. As shown in FIG. 13, for each of the character-recognized input items and input values of the medical checkup sheet, the input items and the input values are identified by enclosing them in rectangles of different colors, thereby improving visibility. , the user can easily grasp the contents of each item. Also, as shown in FIG. 14, a list of input values can be displayed as structured data. In FIG. 14, each input item (key (inspection item)) is displayed so as to be associated with an input value (value). Here, input values that are normally recognized as characters and input values that are recognized in a format different from other input values (input values determined to have an error in the input format) are identified in different ways, such as by different colors. can also be identified by Thereby, the user can easily grasp the input value to be confirmed and respond to it. Also, the display order of the input items and input values shown in the example of FIG. 14 is changed in the form data shown in the example of FIG. It can also be displayed based on the display order of the input values.

Subsequently, as the process of step S203, the user performs an operation to input a correct input value for the input value determined as an error on the user interface screen displayed on the user terminal 200, and the control unit of the server terminal 100 An information reception unit 131 of 130 receives information (input value) input by the user via the communication unit 110 . For example, as shown in FIG. 15, in the input item "visual acuity", an error is displayed because the input value is a character string other than a numerical value/unit. An input value can be updated by performing an input operation. The form data processing unit 133 of the control unit 130 of the server terminal 100 stores the received input value in the user data storage unit 121 of the storage unit 120 so as to be updated as the input data of the form data.

Here, as shown in FIG. 16, in FIG. 15, the corresponding input items and input values of the image data corresponding to the input areas for displaying the input items and input values to be input by the user are indicated by rectangles. By overlapping and enclosing, the user can refer to the image data of the form and the input area mutually. etc., can also be displayed in an identifiable manner. Also, as shown in FIG. 17, when the user selects an area such as a rectangle in which input items and input values are displayed in the image data by, for example, pointing the mouse on the user terminal, the input items are superimposed on the image data. And it is also possible to enlarge and display the comment to inform the input value information and the error.

As described above, according to the present embodiment, the user manually inputs a form such as a health checkup sheet, and utilizes the manually accumulated form data to continue the work, thereby allowing the input to be performed later. Work labor can be saved. In addition, compared to systems that generate machine learning models based on huge amounts of form data and perform character recognition through machine learning, it is possible to save labor in terms of storage capacity and calculation processing costs, and from the perspective of economical system resources. You can also get the effect from
As an example of the present embodiment, the method of supporting the data entry of a health checkup form has been described as an example of a form having a wide variety of formats. However, this embodiment can also be applied.

Although the embodiments according to the invention have been described above, they can be implemented in various other forms, and can be implemented with various omissions, substitutions, and modifications. These embodiments, modifications, omissions, substitutions and changes are included within the technical scope of the claims and their equivalents.

1 system 100 server terminal, 110 communication unit, 120 storage unit, 130 control unit, 200 user terminal, NW1 network

Claims (8)

A method for assisting input of form data as structured data for each item ,
The control part of the server terminal is
recognizing a plurality of input items included in the image data and respective input values corresponding to the plurality of input items by analyzing the image data of the form;
displaying the recognized input items and input values as structured input data;
Receiving a request for extracting reference data of the form from the user terminal,
extracting reference data based on a matching rate of a plurality of input items of the form and reference data and a distance of the input items in response to the extraction request;
displaying the plurality of extracted reference data in order of the matching rate and the distance ;
receiving a request to select one reference data from the extracted reference data;
Method. 2. The method of claim 1 , wherein
The form is a health checkup form, and the image data is analyzed based on the input items and input values included in the selected reference data, and the plurality of input items and the plurality of input items included in the image data are analyzed. A method of recognizing each input value of an input item as structured data . 3. The method of claim 2, wherein
A method in which the control unit notifies as an error when the input format of the input value included in the reference data and the input value included in the image data do not match as the recognition result. 3. The method of claim 2, wherein
The control unit
A method of displaying input items and input values recognized based on the reference data as input data. 3. The method of claim 2, wherein
The control unit
A method of accepting further input for the input value recognized based on the reference data. 2. The method of claim 1, wherein
The control unit
A method of masking and displaying the recognized input items and input values. 2. The method of claim 1, wherein
recognizing a plurality of input items included in the image data and respective input values corresponding to the plurality of input items based on positional coordinates in the image of each of the input items and the input values;
A method, wherein the input items and input values are displayed as input data as structured data in the order recognized based on the position coordinates. 2. The method of claim 1, wherein
A method of superimposing the displayed input items and input values based on a position selected by a user on a user terminal.

Images