JP2016126558A - Entry system and entry method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an entry system in which display which indicates that there is an entry column which is not noted, in an entry sheet, is not overlooked.SOLUTION: The invention relates to an entry system for displaying an entry sheet formed of plural entry columns to a user through a display panel, for making a user note on the respective entry columns. The entry system comprises: a detection unit (S302) for detecting an entry column on which a user is noting current; a determination unit (S308) for determining whether or not, there is an entry column which is not noted, when the entry column which a user is noting current, is detected; and a display control unit (S317) for, when there is the entry column which is not noted, in a view field of a user through the display panel, specifying an attention area which a user pays attention during noting to the detected entry column (S313), then, displaying a warning image which indicates the fact that, there is the entry column which is not noted, so as to be superposed to at least a part of the specified attention area, on the display panel.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for supporting entry in an entry field of an entry form, and particularly to a technique for preventing omission of entry in an entry field.

2. Description of the Related Art In recent years, an entry system that can provide information related to entry in an entry field to a user when filling in an entry form has been used.
For example, in Patent Document 1, when there is an unfilled entry field on an entry form entered with an electronic pen,
A technique for highlighting an unfilled column of an entry form on a display device of a reception terminal that has accepted entry data is disclosed.

As a result, when there is an omission in the entry field of the entry form, the user can visually recognize the unfilled area in which the omission has occurred and can prevent omission of entry.

JP 2004-127197 A

However, in the above entry system, it is highlighted that there is an entry in the entry field on the entry form. For example, if the user is gazing at the entry field, the entry field is not filled. Even if is highlighted, it may not be noticed and may be missed.
The present invention has been made in view of the above-described problems, and provides an entry system and an entry method that can prevent a display indicating that there is an unfilled field on an entry form from being overlooked. The purpose is to do.

In order to achieve the above object, an entry system according to an embodiment of the present invention is an entry system that allows an entry form composed of a plurality of entry fields to be entered in each entry field while showing the user through a display panel. A detecting means for detecting an entry field being filled in; a determination means for determining whether or not an unfilled entry field remains when a currently entered entry field is detected; and an entry in which the decision means is not filled in When it is determined that the field remains, the attention area that the user is paying attention to during the entry in the detected entry field is identified in the user's field of view through the display panel, and at least one of the identified attention areas is identified. Display control means for displaying on the display panel a warning image indicating that an unfilled entry field remains so as to be superimposed on the part, wherein the attention area is located before the detected entry field. It can be a display area in the display panel. In addition, when an entry field currently being entered is detected, the display panel includes a line-of-sight detection unit that detects an area in which the user's line of sight is facing, and the attention area is an area detected by the line-of-sight detection unit. Can be.

In addition, the plurality of entry fields are ordered, and the unfilled field that is subject to determination of the presence or absence of the unfilled entry field by the judging means is an unfilled field that is higher in order than the detected entry field. Can be.
Further, the warning image may be an image showing an unfilled column in an order closest to the detected order of the filled-in column among the unfilled columns in order higher than the detected filled-in column.
In addition, when it is determined that the unfilled entry field remains, the determination unit further determines whether or not the total number of unfilled entry fields is equal to or less than a threshold value, and the display control unit If the total number is less than or equal to the threshold value, instead of the warning image showing the unfilled column in the order closest to the detected order of the filled-in column, the warning image showing all the unfilled columns in the higher order than the detected entry column Can be displayed so as to be superimposed on at least a part of the attention area.

  In addition, when the determination means determines that there is no unfilled entry field, it further determines whether or not there is an unfilled entry field among the entry fields that are lower in order than the detected entry field. The display control means detects when there is an unfilled entry field whose order is lower than the detected entry field, and all the entry fields whose order is lower than the detected entry field are not filled. A warning image indicating an unfilled entry field in an order closest to the detected order of the entry fields is superimposed on at least a part of the attention area among the unfilled entry fields lower in order than the entered field. Can be displayed.

  Further, the display control means, if there is an unfilled entry field that is lower in order than the detected entry field, and there is one unfilled entry field, the display control means displays the unfilled entry field. The warning image shown can be displayed so as to be superimposed on at least a part of the attention area. The warning image may be an image indicating the total number of unfilled entry fields that are higher in order than the detected entry field.

  An entry method according to an embodiment of the present invention is an entry method in which an entry form composed of a plurality of entry fields is entered in each entry field while showing the user through the display panel, and the entry field currently being filled in. A detection step for detecting, a determination step for determining whether or not an unfilled entry field remains when a currently entered entry field is detected, and the determination step determines that an unfilled entry field remains In such a case, in the user's field of view through the display panel, the attention area focused by the user during the entry in the detected entry field is identified and superimposed on at least a part of the identified attention area. And a display control step of displaying on the display panel a warning image indicating that an unfilled entry field remains.

  By having the above configuration, it is determined whether or not there is an unfilled entry field while filling in the entry field on the entry form. If an unfilled entry field remains, the user's view through the display panel In the above, the attention area that the user is paying attention to during the above entry is identified, and a warning image indicating that there is an unfilled entry field is displayed so as to overlap at least a part of the identified attention area. Can easily notice the display of the warning image, so that a display indicating that there is an unfilled column during entry can be avoided.

It is the schematic of an entry system. 3 is a functional block diagram showing a configuration of a display terminal 10. FIG. It is a flowchart which shows operation | movement of the entry paper entry assistance process which the display terminal 10 performs. It is a table which shows the specific example of the operation | movement by the entry process assistance process of FIG. It is a figure which shows the layout of the entry column of an entry form 4 shows a warning image displayed on the display panel 110 at time T5 in FIG. It is the schematic of the entry system of a modification. It is a functional block diagram which shows the structure of 10 A of display terminals. It is a flowchart which shows the modification of operation | movement of the entry paper entry assistance process which 10A of display terminals perform. The example of a display of the warning image displayed on 10 A of display terminals is shown. It is the schematic of the writing system of another modification. It is a functional block diagram which shows the structure of the entry system of FIG. It is a figure which shows the operation | movement of the entry paper entry assistance process which PC20 performs. It is a flowchart which shows the operation | movement of the warning image display process which HMD30 performs. It is a flowchart which shows the modification of the operation | movement of the entry paper entry assistance process of FIG. It is a flowchart which shows another modification of the operation | movement of the entry paper entry assistance process of FIG. It is a table which shows the specific example of the operation | movement by the entry process assistance process of FIG. It is a table which shows another specific example of the operation | movement by the entry process assistance process of FIG.

(Embodiment)
[1] Configuration of entry system FIG. 1A is a schematic diagram of an entry system according to the present embodiment. The entry system 1 includes a placement table 2 on which the entry sheet S is placed, a flat display terminal 10 having a photographing unit 109 and a display panel 110, and a placement surface of the placement table 2 that is fixed to the placement table 2. It is comprised from the supporting member 3 which supports the display terminal 10 so that the main surface of the display terminal 10 may become substantially parallel.

The imaging unit 109 is a main surface (the main surface on which the display panel 109 is disposed) of the display terminal 10 on the side facing the mounting surface so that the entry sheet S mounted on the mounting surface of the mounting table 2 can be captured. The optical axis of the photographing lens of the photographing unit 109 intersects the mounting surface substantially perpendicularly so that the photographed image is not distorted (main surface of the display terminal 10 on the opposite side) (see FIG. 1B). It is configured as follows.
FIG. 2 is a functional block diagram showing the configuration of the display terminal 10. The display terminal 10 is a so-called computer, and includes a CPU 100, a wireless communication unit 101, a ROM 102, a RAM 103, an entry position detection unit 104, an entry status management unit 105, a warning image creation unit 106, a display control unit 107, and an entry form information storage unit. 108, a photographing unit 109, a display panel 110, and the like.

  The wireless communication unit 101 wirelessly communicates with a wireless router (not shown) and connects to a network such as a LAN (Local Area Network) via the wireless router. The ROM 102 stores a program for controlling the entire display terminal 10, a program for executing an entry form entry support process, which will be described later, and the like. The RAM 103 is used as a work area when the CPU 100 executes a program.

The entry position detecting unit 104 performs an entry form (hereinafter referred to as “entry completed”) in an entry form composed of a plurality of ordered entry fields while executing an entry form entry support process described later. Column)).
Specifically, the entry field during entry is detected as follows.
The entry position detection unit 104 detects the position of the pen tip of the pen used for entry from the image data of the entry sheet photographed by the photographing unit 109, and enters whether the detection position is in the entry column. The determination is made with reference to the coordinate information indicating the coordinate range of the display area of the entry column for each number of the entry form stored in the form information storage unit 108. The detection of the position of the pen tip is performed by, for example, storing image data of a photographed image of the pen nib portion of the pen used for filling in the entry sheet in the entry sheet information storage unit 108 in advance and taking an image taken by the imaging unit 109. This can be done by analyzing the image data and extracting the area where the stored image data and the characteristic part match from the photographed image area.

When the detected position is in any entry field, the entry position detecting unit 104 determines whether or not the entry is currently made in the entry field, by using the image data of the entry sheet continuously photographed. It is determined by taking the difference in the number of pixels, and when the number of pixels is changed in the entry column, it is determined that the entry is currently made in the entry column.
Here, a mark serving as a reference point for the coordinate position is printed in advance on the entry sheet, and the entry sheet is placed at a position where the mark can be photographed. For example, it is assumed that the mark is printed at the position indicated by the symbol A on the entry sheet shown in FIG. Assume that the position of each entry field on the entry form is expressed as a relative coordinate position with respect to the reference point, and the detection position is detected as a relative coordinate position with respect to the reference point.

The entry status management unit 105 stores the coordinates stored in the entry form information storage unit 108 every time the entry position detection unit 104 detects the position of the entry field currently filled in after the entry form entry support process described later is started. With reference to the information, the number in the entry field is specified, and the entry status flag of the number is set to “entered” in the entry status table.
Here, the “entry status table” is a table showing the correspondence between the entry column number of the entry form and the entry status flag, and is stored in the entry form information storage unit 108. The “entry status flag” is a flag indicating whether the entry column is “not filled” or “completed”.

The entry status management unit 105 further refers to the entry status flag in each entry column of the entry status table to determine whether or not there is an unfilled entry column in the entry column whose entry column number is higher than that in the entry entry column. .
The warning image creation unit 106 reads out the original image data of the warning image stored in the entry form information storage unit 108, and displays a warning image (filled-in entry column) for display on the display panel 110 based on the original image data. A warning image indicating that there is an unfilled entry field (missing entry) in the entry field with a higher entry field number is created. Here, as shown in FIG. 6, it is assumed that a warning image indicating a message “There is an entry omission in the previous entry field” is created. In the figure, for the convenience of explanation, the display of the image of the user's hand holding the pen is omitted.

  The display control unit 107 causes the display panel 110 to display an image of the entry sheet photographed by the photographing unit 109, and when there is an unfilled entry column in the entry column whose entry column number is higher than the entry entry column. An area in which a warning image indicating that there is an unfilled field in the upper entry field is to be displayed on the display panel 110 is displayed in the user's field of view through the display panel 110 while the user is filling in the filled-in entry field. The warning image generated by the warning image generation unit 106 is displayed in the determined area, based on the attention area currently focused on.

Specifically, the display control unit 107 identifies the display area in the entry entry column as the attention area described above, and displays a warning image for an area that can be superimposed on at least a part of the attention area. It is determined as a power region, and a warning image is displayed on the display panel 110 so as to be superimposed on at least a part of the attention region in the determined region.
For example, as shown in FIG. 6, the display control unit 107 displays the distance from the left end of the filling-in entry column (entry column of the entry column number 6) to the left end of the rectangular balloon-shaped warning image of a predetermined size, and The distance from the right edge of the completed entry column to the right edge of the warning image is the same distance (for example, a distance of 1/4 of the horizontal width of the entry column), and the distance from the ruled line above the entry entry column An area in which the vertical distance to the lower end of the warning image (the tip of the balloon) is a predetermined distance (for example, a distance of 1/6 of the distance between the upper and lower ruled lines in the entry field currently being entered) The image can be determined as an area to be displayed, and the warning image can be displayed so as to be superimposed on a part of the attention area (display area of entry column number 6) in the determined area.

The entry sheet information storage unit 108 stores coordinate information indicating the coordinate range of the position of each entry column of the entry sheet, an entry status table, original image data of a warning image, image data of a pen tip portion, and the like.
The imaging unit 109 captures an entry sheet placed on the placement surface of the placement table 2 during execution of an entry form entry support process, which will be described later, and obtains, for example, a frame image of the entry sheet at 30 to 50 per second. To do. The display panel 110 displays an image of an entry sheet imaged by the imaging unit 109 while executing an entry form entry support process to be described later, and the entry column number is higher than the entry field currently being entered by the entry status management unit 105. When it is determined that there is an unfilled entry field, a warning image is displayed in the area determined by the display control unit 107.

[2] Entry Form Entry Support Process FIG. 3 is a flowchart showing the operation of the entry form entry assistance process performed by the display terminal 10. The display terminal 10 sets the display flag indicating the display state of the warning image to OFF indicating that the warning image is not displayed, and sets the entry status flag of each entry column number of the entry status table to “not filled”. (Step S301), the occurrence of entry in the entry field of the entry form is monitored, and when entry in the entry field is detected (Step 302: YES), the entry field number (X) in which entry has been detected is entered in the entry field. The entry field number is specified (step S303), and the entry status flag corresponding to X is set to “entered” in the entry status table (step S304).

Next, if X is not 1 (step S305: NO), the display terminal 10 sets the variable Y to X (step S306), then decrements the value of the variable Y by 1 (step S307), and enters the entry status table. In step S308, it is determined whether or not the entry field of entry field number Y is unfilled with reference to the entry status flag corresponding to entry field number Y (step S308).
Then, when the entry field of the entry field number Y is not blank (filled) (step S308: NO), the display terminal 10 determines whether Y is 1 (step S309).

If Y is not 1 (step S309: NO), the display terminal 10 proceeds to the process of step S307, and if Y is 1 (step S309: YES), it is determined whether or not the display flag is on. (Step S310).
When the display flag is on (step S310: YES), the warning image being displayed on the display panel 110 is not displayed and the display flag is set to off (step S311), and all entries are made with reference to the entry status table. It is determined whether or not the column has been entered (step S312).

If the display flag is not on (off) (step S310: NO), the display terminal 10 proceeds to the process of step S312.
If the entry field of entry field number Y is not filled in step S308 (step S308: YES), the display terminal 10 identifies the display area of the entry field being entered as the attention area and detects the attention area. Based on the area, an area on the display panel 110 to display the warning image is determined (step S313). If the display flag is not on (off) (step S314: NO), the display flag is set to on ( In step S315), the warning image created in the area determined by creating the warning image is displayed so as to be superimposed on at least a part of the attention area (step S317).

  In step S314, when the display flag is on (step S314: YES), the display terminal 10 hides the warning image being displayed on the display panel 110 (step S316), and proceeds to the process of step 317. . In step S305, when X is 1 (step S305: YES), the display terminal 10 proceeds to the process of step 310.

  FIG. 4 is a table showing a specific example of the operation by the entry processing support process. The specific example of the figure shows an operation example when the entry fields of the entry form are ordered from No. 1 to No. 10 and each entry field is arranged in a layout as shown in FIG. T1 to T10 in FIG. 4 indicate entry times in the entry column and indicate that time has passed in ascending order of numbers. The circle in the figure indicates that the corresponding entry field has been entered, and the x mark indicates that the corresponding entry field has not been entered.

For example, at time T 1, the entry column number 1 is “entered” and the other entry columns are “unfilled”, and at time T 5, entry column numbers 1 to 4 are indicated. The No. and No. 6 entry fields are “entered”, and the other entry fields are “not filled”.
In the specific example of the figure, the entry column numbers are 1st, 2nd, 3rd, 4th, 6th, 5th, 7th, 9th, 8th, 10th in order. It has been shown.

The bottom column of the table in the figure is a column indicating whether or not a warning image is displayed at the corresponding time. “Yes” indicates that a warning image is displayed, and “No” indicates that a warning image is displayed. Indicates not to be done.
At time T1, entry to the entry column with the entry column number 1 is detected in the entry processing support process of FIG. 3 (step S302: YES), X = 1 (step S305: YES), and a warning image Is not displayed (step S310: NO, step S312: NO).

  At time T2, in the entry process support process of FIG. 3, entry to the entry field whose entry field number is 2 is detected (step S302: YES), X = 2 (step S305: NO), and entry is in progress. Since the entry column number 1 which is higher than the entry column number 2 is not yet entered, no warning image is displayed (step S308: NO, step S309: YES, step S310: NO, step S312: NO).

At each time point of time T3 and T4, the entry column number (entry column number 3 at time T3, entry column number 4 at time T4) higher than the entry column number (entry time T3) at the time T3 and T4, respectively. Then, at entry column numbers 1 and 2 and time T4, entry column numbers 1 to 3) are not filled in, so no warning image is displayed.
At time T5, the entry in the entry column number 6 is detected in the entry process support process of FIG. 3 (step S302: YES), X = 6 (step S305: NO), and entry is in progress. Since entry field number 5 higher than entry field number 6 has not been entered (step S308: YES), a warning image is displayed so as to be superimposed on at least a part of the attention area in the area determined in step 313 (step 308). S313, Step S314: NO, Step S315, Step S317).

  At time T6, in the entry process support process of FIG. 3, entry to the entry field whose entry field number is 5 is detected (step S302: YES), and X = 5 (step S305: NO). ), There are no unfilled columns in the column numbers 1 to 4 higher than the currently filled column number 5 (step S308: NO). Further, since the warning image is displayed at time T5 (step S310: YES), the currently displayed warning image is hidden (steps S311 and S312).

At time T7, as in the case of time T2, the entry column numbers (entry column numbers 1 to 6) higher than the entry column number 7 being entered are not yet filled, so that no warning image is displayed.
At time T8, as in time T5, entry field number 8 higher than entry entry field number 9 is not yet entered (step S308: YES), so a warning image is displayed in the area determined in step 313. Is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: NO, step S315, step S317).

At time T9, as in the case of time T6, the entry in the entry column number 8 that has not been entered is entered, and the entry column numbers 1 to 7 higher than the entry column number 8 being entered are not yet entered. Since there is no entry field, the warning image displayed at time T8 is hidden.
At time T10, since the entry fields for all entry field numbers have been entered (step S312: YES), the entry process support process ends.

  FIG. 6 shows a warning image G1 displayed on the display panel 110 at time T5. As shown in the figure, at the time T5, the warning image G1 is displayed so as to be superimposed on the upper end portion (part) of the attention area (display area of the entry field (entry field of entry field number 6)). Has been. Although not shown in the figure, at the time T8 as well as at the time T5, the warning image is displayed at the upper end of the attention area (the display area in the entry field (entry field number 9 entry field)). Part).

As described above, in the entry system according to the present embodiment, when filling in the entry form, if the entry column that is higher in order than the entry entry column is not filled in, there is an unfilled column. The warning image to be displayed is displayed so as to be superimposed on at least a part of the attention area in which the user is paying attention in the user's field of view through the display panel 110, so that the user can easily notice the warning image. , It is possible not to miss a display indicating that there is an unfilled column during entry.
(Modification)
As described above, the present invention has been described based on the embodiment. However, the present invention is not limited to the above-described embodiment, and the following modifications can be implemented.

(1) In the above embodiment, in the entry form entry support process, the display area of the entry field being filled in is identified as the attention area, but the area on the display panel 110 where the user's line of sight is facing May be detected using a line-of-sight sensor to identify the region as a region of interest.
FIG. 7 is a schematic diagram of an entry system according to this modification. As shown in the figure, the entry system 1A according to the present modification is different from the entry system 1 according to the above-described embodiment in that the display terminal 10A includes a line-of-sight sensor 111. FIG. 8 is a functional block diagram showing the configuration of the display terminal 10A.

Hereinafter, the difference between the entry system 1A according to the present modification and the entry system 1 according to the above-described embodiment will be described with reference to FIG. 7 and FIG. Reference numerals are assigned and description is omitted.
The line-of-sight sensor 111 includes a light emitting element that irradiates infrared light toward the user's eye to acquire a Purkinje image, and an image sensor that captures an area including the user's eye, and uses a Purkinje image. The gaze direction of the user is measured by the method, and the measurement result is output to the gaze area calculation unit 1001.

The gaze area calculation unit 1001 calculates and obtains an area where the user's line of sight is directed on the display panel 110 based on the measurement result input from the line-of-sight sensor 111.
FIG. 9 is a flowchart showing a modification of the operation of the entry form entry support process performed by the display terminal 10A. About the same processing content as the entry paper entry support processing of FIG. 3 of said embodiment, the same step number is provided and description is abbreviate | omitted, and a difference is demonstrated below.

  When the entry field of entry field number Y is not filled in step S308 (step S308: YES), display terminal 10A has the user's line of sight on display panel 110 based on the current measurement result of line-of-sight sensor 111. The calculated area is identified as the attention area, and the area on which the warning image is to be displayed on the display panel 110 is determined based on the attention area (step S901). Specifically, the display terminal 10A determines an area that can be superimposed on the entire calculated area as an area where a warning image is to be displayed. In step S902, the warning image is displayed on the determined area so as to be superimposed on the entire attention area (the entire calculated area).

For example, as illustrated in FIG. 10, the display terminal 10 </ b> A displays an area in which a rectangular balloon-shaped warning image G <b> 2 having a predetermined size is superimposed on the entire calculated area (area indicated by a dotted circle) at the upper right end. It can be determined as an area to display an image. In the figure, as in the case of FIG. 6, the display of the image of the user's hand holding the pen is omitted.
As described above, in the entry system according to the present modification, it is indicated that there is an unfilled field when the fill-in field in the order higher than the filled-in entry field is unfilled during filling in the fill-in form. The warning image is displayed on the display panel so that it overlaps the entire area where the user's line of sight is facing, so that the user can easily notice the warning image, and do not miss that there is an unfilled field during filling Can be.

  (2) The configuration of the entry system is not limited to the configuration of the above-described embodiment and modification (1). As shown in FIG. 11, the entry system comprises a mounting table 2, a PC (Personal Computer) 20, a head mount. It is good also as comprising from the plate-shaped pressure sensor 40 connected with the display (HMD) 30 and PC20. In the entry system 1B according to the present modification, the user can fix the entry sheet S at a predetermined position on the pressure sensor 40 arranged on the mounting table 2 with a fixing member T (for example, drafting tape). The coordinates of the contact position where the pen is brought into contact with the entry sheet S are detected by the pressure sensor 40 to detect the entry position on the entry sheet S. Then, the PC 20 specifies the entry column number during entry based on the detection result, and transmits it to the head mounted display 30.

  FIG. 12 is a functional block diagram showing the configuration of the entry system 1B according to this modification. Hereinafter, the functional configuration of the entry system 1B will be described with reference to FIG. The PC 20 includes a CPU 200, a communication interface (I / F) unit 201, a ROM 202, a RAM 203, an entry column specifying unit 204, a warning image display adjustment unit 205, an entry form information storage unit 206, an input operation unit 207, a display unit 208, and the like. Composed.

The communication I / F unit 201 is an interface for connecting to a LAN such as a LAN card or a LAN board. The ROM 202 stores a program for controlling the pressure sensor 40, a program for executing an entry form entry support process, which will be described later, and the like.
The RAM 203 is used as a work area when the CPU 200 executes the program.
The entry field specifying unit 204 performs the entry field entry in the entry form composed of a plurality of entry fields that are ordered based on the detection result of the pressure sensor 40 while executing the entry form entry support process described later. Identify the entry column number.

  Specifically, each time the entry field specifying unit 204 acquires the position coordinates of the position where the pen writing pressure is detected from the pressure sensor 40, the entry field specifying unit 204 stores the entry form stored in the entry form information storage unit 206. Referring to the entry field position information indicating the coordinate range of the position coordinates of each entry field number entry field (hereinafter referred to as “position coordinate range”), the acquired position coordinates are the positions of any entry fields on the entry form. When entering the coordinate range, it is detected that there is an entry in the entry column, and the entry column number of the entry column in which entry is detected is specified as the entry column number in the entry column.

Further, each time the entry field identification unit 204 identifies the entry field number of the entry field being entered, the entry field flag of the entry field number is entered in the entry situation table stored in the entry form information storage unit 206. Set to finished.
The warning image display adjustment unit 205 refers to the entry status table every time the entry column specifying unit 204 identifies the entry column number of the entry column being entered, and the entry column number is assigned to the entry column higher than the entry column being entered. It is determined whether or not there is an unfilled entry field. If there is an unfilled entry field, the specified entry field number and a warning image display instruction are transmitted to the HMD 30 via the communication I / F unit 201. . Further, as will be described in detail in the description of the entry form entry support process described later, the warning image display adjustment unit 205 sends a warning image non-display instruction via the communication I / F unit 201 according to the display flag setting state. Send to HMD30.

The entry form information storage unit 206 stores entry column position information, an entry status table, and the like.
The input operation unit 207 includes an input device such as a keyboard and a mouse, and receives an operation instruction from the user through the input device. For example, an instruction to start a program for executing an entry form entry support process described later is received.
The display unit 208 includes a liquid crystal display or the like, and displays a screen or the like for receiving a program start instruction or an end instruction.

The HMD 30 is a glasses-type terminal worn on the user's head, and includes a display panel (glasses-type display panel 308) in the glasses portion. The HMD 30 includes a CPU 300, a wireless communication unit 301, a ROM 302, a RAM 303, a storage unit 304, a warning image creation unit 305, a display control unit 306, a photographing unit 307, a glasses-type display panel 308, an input operation unit 309, and the like.
The wireless communication unit 301 wirelessly communicates with a wireless router (not shown), connects to the LAN via the wireless router, and communicates with the PC 20. The ROM 302 stores a program for controlling the entire HMD 30, a program for executing a warning image display process started when executing an entry form entry support process described later, and the like. The RAM 303 is used as a work area when the CPU 300 executes the program.

  The storage unit 304 stores an entry field specifying table indicating the correspondence between entry field verification marks and entry field numbers added to each entry field of the entry form, original image data of a warning image, and the like. In each entry field of the entry form S used in the entry system 1B of the present modification, a mark for specifying the display area of each entry field from the captured image (for example, the shape, color, pattern, etc. for each entry field is mutually different). Different marks) are added in advance to predetermined positions (for example, the positions of the upper left, upper right, lower left, and lower right ends of the rectangular entry field).

The warning image creation unit 305 reads the original image data of the warning image stored in the storage unit 304 and creates a warning image to be displayed on the glasses-type display panel 308 based on the original image data.
The display control unit 306 causes the glasses-type display panel 308 to display an image of the entry sheet photographed by the photographing unit 307. Further, when the display control unit 306 receives the entry column number of the entry entry column and the warning image display instruction from the PC 20, the entry control unit 306 refers to the entry column identification table and identifies the entry column verification mark corresponding to the entry column number. Further, the spectacle-type display panel 308 is identified by analyzing the image data of the image of the entry sheet photographed by the photographing unit 307 at this time point and identifying the position of the mark that matches the identified entry field comparison mark. The display area of the entry column in the photographed image displayed in is specified.

  The display control unit 306 then pays attention to the display area of the entry entry column specified in the same manner as in the above embodiment in the user's field of view through the glasses-type display panel 308. A region that can be superimposed on at least a part of the region of interest is determined as a region to display a warning image, and the warning image is determined to be at least a part of the region of interest. The image is displayed on the glasses-type display panel 308 so as to be superimposed.

In addition, when receiving a warning image non-display instruction from the PC 20, the display control unit 306 hides the currently displayed warning image.
The imaging unit 307 captures a scene viewed by the wearer of the HMD 30. Here, during execution of an entry form entry support process to be described later, the entry form S placed on the pressure sensor 40 is photographed and, for example, a frame image of the entry form S at 30 per second is acquired. The glasses-type display panel 308 displays an image of the entry sheet S photographed by the photographing unit 307 during execution of an entry form entry support process, which will be described later, and receives a warning image display instruction from the PC 20. A warning image is superimposed on the image of the entry sheet S. The input operation unit 309 receives an operation instruction from the user. For example, for example, an instruction to start a program that executes a warning image display process described later is accepted.

FIG. 13 is a diagram showing an operation of an entry form entry support process performed by the PC 20. The same processing content as the operation of the entry form entry support processing of the above embodiment of FIG. 3 is given the same step number and description thereof is omitted, and the difference will be mainly described below.
In step S310, when the display flag is on (step S310: YES), the PC 20 transmits a warning image non-display instruction to the HMD 30 and sets the display flag to off (step S1301).

If the display flag is on in step S314 (step S314: YES), the PC 20 transmits a warning image non-display instruction to the HMD 30 (step S1302), and the entry field number and warning image specified in step S303. Is displayed to the HMD 30 (step S1303).
Further, in step S312, the PC 20 transmits a processing end instruction to the HMD 30 when all the entry fields have been completed (step S312: YES) (step S1304).

FIG. 14 is a flowchart showing an operation of warning image display processing performed by the HMD 30. When the HMD 30 receives the entry column number of the entry entry column and the warning image display instruction from the PC 20 (step S1401: YES), the HMD 30 displays a display area of the entry entry entry column in the photographed image displayed on the glasses-type display panel 308. It is identified (step S1402).
Next, the HMD 30 identifies the identified display area as the attention area, determines an area on the glasses-type display panel 308 to display the warning image based on the attention area (step S1403), creates and determines the warning image. The warning image created in the area is displayed so as to be superimposed on at least a part of the attention area (step S1404).

When the HMD 30 receives a warning image non-display instruction from the PC 20 (step S1405: YES), the HMD 30 hides the currently displayed warning image (step S1406) and determines whether or not a process end instruction has been received from the PC 20. (Step S1407).
When the process end instruction has not been received (step S1407: NO), the HMD 30 proceeds to the process of step S1401, and when the process end instruction has been received (step S1407: YES), the warning image display process is performed. finish.

(3) In addition, in the configuration of the entry system according to the modified example (2), the entry position on the entry form of the user may be detected by using a publicly known electronic pen instead of the pressure sensor.
Specifically, an entry sheet on which a dot pattern shown in FIGS. 3 and 4 of the patent document (Japanese Patent Laid-Open No. 2013-145586) is printed as an entry sheet is described in FIG. 5 and paragraphs 0036 to 0045 of the patent document. The dot pattern printed by the electronic pen of the configuration is photographed, the coordinate data of the entry position on the entry sheet is calculated from the image data of the photographed dot pattern, and the coordinate data is built in the electronic pen. It is good also as transmitting to PC20 via the communication unit which is, and performing the operation | movement similar to a modification (2).

The entry system is configured so that the coordinate data is directly transmitted from the communication unit to the HMD 30 without using the PC 20, and the HMD 30 performs the same operation as the entry form entry support process of the modification (2). It is good to do.
(4) In addition, in the writing systems of the modification examples (2) and (3), the HMD 30 is configured to include a line-of-sight sensor. It is good also as detecting the area | region which has faced using a gaze sensor, and specifying the said area | region as an attention area. Specifically, instead of the processing in step S1403 in FIG. 14, the same processing as in steps S901 and S902 in FIG. 9 may be performed. In the case of this modification, the glasses-type display panel 306 may be a translucent display panel, and the configuration of the imaging unit 307 in the HMD 30 may be omitted.

  (5) The operation of the entry form entry support process of the above embodiment may be modified as shown in FIG. 15 to change the display form of the warning image. In the figure, the same processing contents as those of the entry form filling support processing of FIG. 3 of the above embodiment are given the same step numbers and description thereof is omitted, and differences will be described below. After executing the processing of step S315 or step S316, the display terminal 10 refers to the entry status table and counts the total number (N) of unfilled columns whose entry column number (order) is higher than X (step S1501). , N is less than or equal to a threshold value (step S1502).

  If N is not less than or equal to the threshold value (step S1502: NO), the entry column number Y (unfilled entry column number that is higher in order than X and closest to X in the area determined in the process of step S313). A warning image (for example, if Y is 5, entry field 5) indicating that the entry field is unfilled (missed entry) is missing entry. Is generated based on the original image data of the warning image and the information of the unfilled column number Y higher in order than X, and the generated warning image is superimposed on at least a part of the attention area. (Step S1503).

  If N is less than or equal to the threshold value (step S1502: YES), the display terminal 10 warns all unfilled columns (filled omission columns) in order higher than X in the area determined in the process of step S313. An image (for example, a warning image indicating a message that “entry field 5) and 6) are missing” when the threshold value is 2 and the entry field number in the unfilled field is 5, 6) Is generated based on the original image data and the information in the unfilled column higher in order than X, and the generated warning image is displayed so as to be superimposed on at least a part of the attention area (step S1504).

  As a result, in the entry form entry support processing according to the present modification, when filling in the entry form, if the entry field that is higher in order than the entry entry field is not entered, the entry field is specified. Since the warning image is displayed so as to be superimposed on at least a part of the attention area in which the user is paying attention within the user's field of view through the display panel 110, the user can easily notice the warning image display. It is possible to know the position of the blank field quickly.

Note that the present modification may be modified such that the process of step S1501, step S1502, and step S1504 of the present modification is eliminated and the process of step S315 or step 316 is performed, and then the process of step S1503 is performed. Good.
(6) The operation of the entry form entry support process of the above embodiment may be modified as shown in FIG. 16 to change the display form of the warning image. In the figure, the same processing contents as those of the entry form filling support processing of FIG. 3 of the above embodiment are given the same step numbers and description thereof is omitted, and differences will be described below.

  After executing the process of step S304, the display terminal 10 refers to the entry status table to determine whether or not there is an unfilled field whose entry column number (order) is higher than X (step S1601). In this case (step S1601: YES), after executing the processing of step S306 to step S308 and step S313 to step S316, a warning image indicating that the entry column of entry column number Y is not filled (missed entry) ( For example, if Y is 5, entry field 5) is missing. Is generated based on the original image data of the warning image and the information of the unfilled column number Y, and the generated warning image is displayed so as to be superimposed on at least part of the attention area (step S1602).

  If there is no unfilled column whose order is higher than X (step S1601: NO), the display terminal 10 refers to the fill status table to determine whether there is a blank field whose order is lower than X. Judgment is made (step S1603), and if there is any (step S1603: YES), whether all the entry fields in the lower order than X are unfilled fields and whether the number of unfilled fields is 2 or more. Is determined (step S1604).

  If the determination result in step S1604 is negative (step S1604: NO), the display terminal 10 sets the variable Y to X (step S1605) and then increments the value of the variable Y by 1 (step S1606). In the entry status table, the entry status flag corresponding to the entry column number Y is referred to and it is determined whether or not the entry column of the entry column number Y is not filled (step S1607).

When the entry column of entry column number Y is not filled (step S1607: YES), the display terminal 10 proceeds to the process of step S313, and when the entry column of entry column number Y is not blank ( Step S1607: NO), the process proceeds to Step S1606.
If the determination result of step S1604 is affirmative (step S1604: YES), the display terminal 10 determines whether or not the display flag is on (step S1608). If the display flag is on (step S1608: YES), the display terminal 10 hides the warning image being displayed on the display panel 110 and sets the display flag to off (step S1609). If is not on (step S1608: NO), the process proceeds to step S302.

  Further, in step S1603, when there is no unfilled column lower in order than X (step S1603: NO), the display terminal 10 determines whether or not the display flag is on (step S1610), and the display flag Is on (step S1610: YES), the warning image being displayed on the display panel 110 is not displayed and the display flag is set to off (step S1611). If the display flag is not on (step S1610). : NO), the process is terminated.

FIG. 17 is a table showing a specific example of the operation by the entry process support process. The specific example of the figure shows an operation example when the entry fields of the entry form are ordered from No. 1 to No. 10 and each entry field is arranged in a layout as shown in FIG. Since the reference numerals in the figure are the same as those in the table in FIG.
In the specific example of the figure, the entry field numbers are 1, 3, 5, 8, 8, 2, 9, 4, 6, 10, 10, and 7 in the order of entry. It has been shown.

Also, the bottom column of the table in the figure is a column indicating the unfilled column number Y indicated by the warning image at the corresponding time. “Number” indicates the Y number, “None” indicates the warning image. Indicates that they are not displayed.
At time T1, entry to the entry column with the entry column number 1 is detected in the entry processing support process of FIG. 16 (step S302: YES), X = 1, and no warning image is displayed (step S1601). : NO, step S1603: YES, step S1604: YES, step S1608: NO).

  At time T2, in the entry processing support process of FIG. 16, entry to the entry column number 3 is detected (step S302: YES), X = 3, and the entry column number 3 is entered in order. Since there is a high-level unfilled column (unfilled column of entry column number 2) (step 1601: YES), Y = 2, and the entry column of entry column number 2 is missing in the area determined in step 313. A warning image indicating this is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: NO, step S315, step S1602).

  At time T3, entry in the entry column number 5 is detected in the entry process support process of FIG. 16 (step S302: YES), and X = 5, and the entry column number 5 is entered in order. Since there is an upper unfilled column (unfilled column of entry column numbers 2 and 4) (step 1601: YES), Y is 4 which is closest to X (entry column number 5) and is determined in step 313 A warning image indicating that the entry field of entry field number 4 is omission is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: YES, step S316, step S1602). .

  At time T4, an entry in the entry column number 8 is detected in the entry process support process of FIG. 16 (step S302: YES), and X = 8. Since there is a high-level unfilled column (unfilled column of entry column numbers 2, 4, 6, and 7) (step 1601: YES), Y becomes 7 that is closest to X (entry column number 8), A warning image indicating that the entry field of entry field number 7 is missing in the area determined in step 313 is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: YES, step S316). Step S1602).

  At time T5, an entry in the entry field with the entry field number 2 is detected in the entry process support process of FIG. 16 (step S302: YES), and X = 2. Then, there is no unfilled column in the higher order than the entered column number 2 and the unfilled columns in the lower order than the entered column number 2 (unfilled columns of the entered column numbers 4, 6, 7, 9, 10). ) (Step S1601: NO, step S1603: YES, step S1604: NO), Y is 4 which is closest in order to X (entry column number 2), and the entry column number 4 is displayed in the area determined in step 313. Is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: YES, step S316, step S1602).

  At time T6, the entry in the entry column number 9 is detected in the entry process support process of FIG. 16 (step S302: YES), X = 9, and the entry column number 9 is entered in order. Since there is an upper unfilled column (unfilled columns of entry column numbers 4, 6, and 7) (step 1601: YES), Y becomes 7 that is closest to X (entry column number 9), and step 313 Is displayed in such a manner that a warning image indicating that the entry field of entry field number 7 is omission is superimposed on at least a part of the attention area (step S313, step S314: YES, step S316, step S1602).

  At time T7, an entry in the entry field with the entry field number 4 is detected in the entry process support process of FIG. 16 (step S302: YES), and X = 4. And, there is no unfilled column in the order higher than the entry column number 4 during entry, and there are unfilled columns (the entry columns of the entry column numbers 6, 7, 10) lower in order than the entry column number 4 being entered. (Step S1601: NO, Step S1603: YES, Step S1604: NO), Y is 6 which is closest in order to X (entry column number 4), and the entry column of entry column number 6 is in the area determined in step 313. A warning image indicating omission is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: YES, step S316, step S1602).

  At time T8, the entry in the entry column number 6 is detected in the entry process support process of FIG. 16 (step S302: YES), and X = 6. And, there is no unfilled column higher in order than the filled-in column number 6 and there are unfilled columns (unfilled columns of the filled-in column numbers 7 and 10) in the lower order than the filled-in column number 6 (steps). S1601: NO, step S1603: YES, step S1604: NO), Y is 7 which is closest to X (entry column number 6), and the entry column of entry column number 7 is missing in the area determined in step 313 Is displayed so as to be superimposed on at least a part of the attention area (step S313, step S314: YES, step S316, step S1602).

  At time T9, the entry in the entry column number 10 is detected in the entry process support process of FIG. 16 (step S302: YES), and X = 10. Since there is an upper unfilled field (unfilled field of the filled field number 7) (step 1601: YES), Y = 7, and the filled field of the filled field number 7 is missing in the area determined in step 313. A warning image indicating this is displayed so as to overlap at least a part of the attention area (step S313, step S314: YES, step S316, step S1602).

  At time T10, entry in the entry column number 7 is detected in the entry process support process of FIG. 16 (step S302: YES), and all entry columns of entry column numbers 1 to 10 are entered. As a result (step S1601: NO, step S1603: NO), the currently displayed warning image is hidden, and the entry support process ends (step S1610: YES, step S1611).

In this way, in this modification, even when there is an unfilled field that is lower in order than the filled-in entry field, a warning image that identifies the unfilled field is displayed, so the user can Therefore, even when the information is entered randomly, the user can be notified of the omission of entry by the warning image, and the occurrence of omission can be prevented.
In addition, as shown in a table of a specific example different from FIG. 17 shown in FIG. 18, when the entry order in the entry field follows the order of the entry field, the warning image has one unfilled field. (Step S1601: NO, step S1603: YES, step S1604: YES, step S1608: NO), the warning image is displayed unnecessarily excessively when the user fills in the order of entry. Can be prevented.

  (7) In the above embodiment, a warning image indicating that there is an unfilled entry field (missing entry) in the entry field whose entry field number is higher than the entry field being entered is displayed on the display panel 110. However, instead of the warning image, a warning image indicating the total number of unfilled columns whose entry column numbers are higher than the entry column being entered may be created and displayed on the display panel 110. Similarly, in the modifications (1) to (3), the warning image of this modification may be displayed.

(8) In the above embodiment and modifications (1) to (3), a warning image is displayed on the display panel when there is an unfilled entry field in the entry field that is higher in order than the entry field during entry. But
A warning image may be displayed on the display panel when there is an unfilled field regardless of whether the order is higher or lower than the currently entered field.

  The present invention relates to a technique for supporting entry in an entry field of an entry form, and in particular, can be used as a technique for preventing omission of entry in an entry field.

1, 1A, 1B Writing system 2 Mounting table 3 Support member 10, 10A Display terminal 20 PC
30 HMD
40 Pressure sensor 100 CPU
101 Wireless communication unit 102 ROM
103 RAM
104 entry position detection unit 105 entry status management unit 106 warning image creation unit 107 display control unit 108 entry form information storage unit 109 photographing unit 110 display panel

Claims (10)

An entry system that allows an entry form composed of a plurality of entry fields to be entered in each entry field while showing the user through a display panel.
A detection means for detecting the entry field currently being entered,
A determination means for determining whether or not an unfilled entry field remains when a currently entered entry field is detected;
When the determination means determines that an unfilled entry field remains, the attention area that the user is paying attention to during the entry to the detected entry field is identified in the user's field of view through the display panel. Display control means for displaying on the display panel a warning image indicating that an unfilled entry field remains so as to overlap at least a part of the identified attention area;
An entry system characterized by comprising: The entry system according to claim 1, wherein the attention area is a display area in the display panel of the entry field detected. A line-of-sight detecting means for detecting an area in which the line of sight of the user is facing in the display panel when the entry field currently being filled is detected;
The entry system according to claim 1, wherein the attention area is an area detected by the line-of-sight detection unit. The plurality of entry fields are ordered,
4. The unfilled column that is a target for determining whether or not the unfilled entry column remains by the judging means is an unfilled column that is higher in order than the detected entry column. Kani's entry system. The said warning image is an image which shows the unfilled column of the order nearest to the order of the detected said entry column among the unfilled columns of order higher than the detected said entry column. Description entry system. When it is determined that the unfilled entry field remains, the determination unit further determines whether or not the total number of unfilled entry fields is equal to or less than a threshold value.
When the total number of unfilled entry fields is equal to or less than the threshold value, the display control means starts from the detected entry field instead of the warning image indicating the unfilled field in the order closest to the detected order of the entry fields. 6. The filling system according to claim 5, wherein warning images indicating all unfilled columns at the top are displayed so as to be superimposed on at least a part of the attention area. When the determination means determines that there is no unfilled entry field, it further determines whether or not there is an unfilled entry field among the entry fields lower in order than the detected entry field,
If there is an unfilled entry field whose order is lower than the detected entry field, and all the entry fields whose order is lower than the detected entry field are not filled, the display control means detects the detected entry field. A warning image indicating an unfilled entry field in the order closest to the detected order of the entry fields among the unfilled entry fields lower in order than the entry field is displayed so as to be superimposed on at least a part of the attention area. The entry system according to claim 5, wherein: The display control means, when there is an unfilled entry field lower in order than the detected entry field and there is one unfilled entry field, a warning indicating the unfilled entry field The entry system according to claim 7, wherein an image is displayed so as to be superimposed on at least a part of the region of interest. The entry system according to claim 4, wherein the warning image is an image indicating a total number of unfilled entry fields that are higher in order than the detected entry field. An entry method that allows an entry form composed of a plurality of entry fields to be entered in each entry field while showing the user through a display panel,
A detection step to detect the entry field currently being filled in;
A determination step for determining whether or not an unfilled entry field remains when a currently entered entry field is detected;
When the determination step determines that an unfilled entry field remains, in the user's field of view through the display panel, the attention area that the user is paying attention to during the entry to the detected entry field is identified. A display control step for displaying on the display panel a warning image indicating that an unfilled entry field remains so as to be superimposed on at least a part of the identified attention area;
An entry method characterized by including:

Images