JP2005135135A - Sight line input device for electronic form - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sight line input device for electronic form that is adaptive various formats and styles of electronic forms and extremely excellent in operability. <P>SOLUTION: The sight line input device for electronic form is equipped with a sight line detecting means for detecting a sight line position based upon movements of the eyeballs, a decision input means for decision input at the sight line position, and a display means of displaying a form display screen, a character input picture, and the sight line position. One of a plurality of input areas displayed in the form display screen is indicated with the sight line position and inputting a decision by the decision input means, and then the display means displays a character input screen comprising input key elements corresponding to information kinds to be inputted according to property information of the selected input area. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a line-of-sight input device for an electronic form for detecting a line-of-sight position based on the movement of an eyeball and inputting characters or the like into an input area of the electronic form.

  In the past, persons with disabilities and injuries who were particularly unable to use their hands could not prepare documents without the cooperation of third parties. This is a major obstacle in filling in various documents required by medical institutions. Even if you do not fill out the document, your communication method will be limited. In view of this, there has been proposed a technique that enables communication of intention by using eye movement, that is, a line of sight, which is relatively difficult to cause an obstacle.

  For example, a. Means for detecting an electrooculogram signal generated by movement of a human eye; b. Means for converting the electrooculogram signal into a directivity control signal representing the direction of eye movement; c. There has been proposed a line-of-sight controller including means for interfacing a directivity control signal to a device to be controlled (see Patent Document 1).

  Also, for example, a line of sight having eye-gaze detection means for calculating an eyeball rotation amount by performing image processing on an eyeball image obtained from an imaging means for capturing eyeball images for both the left and right eyes, and calculating a gaze position from the calculated eyeball rotation amount In a device with a detection function, a. Blink determination means for determining a blink state; b. When it is determined by the determination means that the state of either the left or right eye is a blinking state and the other eye is not a blinking state, the input operation is performed using the gaze position calculated by the line-of-sight detection means as the input instruction position. An apparatus with a line-of-sight detection function has been proposed that includes a confirmation signal output means for outputting a confirmation signal for execution (see Patent Document 2).

  Further, for example, in a line-of-sight input device that inputs characters and the like according to the position of the line of sight by detecting the position of the line of sight on the display screen according to the movement of the eyeball of the observer observing the display screen. . A character area display means for displaying a character area in which a part of a character group is displayed among a plurality of types of character groups in the line-of-sight input area; b. A character group scrolling means for displaying a new character in the character region when the position of the line of sight is moved to the specific region; c. There has been proposed a line-of-sight input device including an input unit that inputs a character at a line-of-sight position when the line-of-sight position is on a character group in a character area (see Patent Document 3).

  As described above, in the prior art, even if an operation by line-of-sight input can be performed, character input cannot be performed, or even if a character input can be performed by line-of-sight input, complicated operation cannot be performed. In addition, various forms and forms of electronic forms cannot be handled, or even if they can be handled, the operation becomes extremely difficult. As for electronic forms, several systems have been proposed with regard to their formats and forms, creation processing, submission via a network, and the like (see Patent Documents 4 and 5). Also, in the electronic form input, personal authentication is an indispensable condition and convenience is often obtained thereby, but the conventional gaze input system cannot perform personal authentication. For personal authentication only, a system based on iris authentication has been proposed (see Patent Document 6).

JP-A-6-95797 JP 2002-7053 A JP 2000-20196 A JP 2001-265910 A JP 2002-74245 A JP 2001-167252 A

  The present invention solves the above-described problems, and an object of the present invention is to provide a line-of-sight input device for an electronic form that can cope with various formats and forms in the electronic form and has extremely good operability.

The line-of-sight input device for electronic forms according to claim 1 of the present invention includes a line-of-sight detecting means for detecting a line-of-sight position based on a movement of an eyeball, a decision input means for performing a decision input at the line-of-sight position, and a form display screen And a character input screen and a display means for displaying the line-of-sight position, wherein one of a plurality of input areas displayed on the form display screen is indicated by the line-of-sight position and selected by the decision input means Then, the display means displays a character input screen composed of input key elements corresponding to the information type to be input from the attribute information of the selected input area.
According to a second aspect of the present invention, there is provided a line-of-sight input device for an electronic form according to the first aspect of the present invention, wherein the line-of-sight detection means is based on a captured image obtained by capturing a movement of an eyeball with a camera. Gaze detection means for detecting a gaze position, wherein the camera is provided with iris authentication means for performing personal authentication based on a captured image obtained by imaging an iris (iris) in the eyeball.

According to the method for manufacturing a paper container set according to claim 1 of the present invention, when one of the plurality of input areas displayed on the form display screen is indicated by the line-of-sight position and selected by the decision input means, the display is displayed. The means displays a character input screen composed of input key elements corresponding to the information type to be input from the attribute information of the selected input area. That is, it is possible to display a character input screen in a form suitable for the selected input area. Therefore, it is possible to provide a line-of-sight input device for electronic forms that can cope with various forms and forms of electronic forms and has extremely good operability.
According to the electronic form line-of-sight input device according to claim 2 of the present invention, the iris (iris) is imaged by the camera of the line-of-sight detection means, and the identity authentication is performed by the iris authentication means. Accordingly, the user authentication can be performed with a lean configuration and good operability.

  The line-of-sight input device for an electronic form according to the present invention has dedicated software for combining the line-of-sight input system and the electronic form system and making it possible to suitably input in the entry field. Since the line-of-sight input system and the electronic form system are generally realized as a computer system, they can be configured to operate in one computer system (hardware). An interface may be provided between the two computer systems, and the line-of-sight input system may be configured as an input terminal of the electronic form system. In any configuration, the display of the viewpoint position and the like in the line-of-sight input system is combined with the display of the electronic form image and the like in the electronic form system.

  The line-of-sight input system portion of the electronic form line-of-sight input device includes line-of-sight detection means for detecting the line-of-sight position based on the movement of the eyeball and decision input means for making a decision input at the line-of-sight position. The line-of-sight detection means includes, for example, a video camera that captures particularly the eye part of the face, and an image processing unit that performs processing for detecting eye movement, that is, the line-of-sight direction based on the captured image obtained by the video camera. Consists of. When the line-of-sight direction is in the line-of-sight input area (usually the entire display screen) of a display device such as a liquid crystal display, the intersection of the line-of-sight direction (straight line) and the line-of-sight input area (plane), that is, the line of sight displayed on the display device The coordinates of the position are calculated by the line-of-sight input system. The line-of-sight position displayed on the display device is a position for designating a specific position on the screen when performing line-of-sight input. That is, it performs the same function as a mouse cursor.

  Further, the decision input means is means for performing an input in conjunction with a portion pointed by the line-of-sight position (pointer) to activate and activate a function existing at the position, and performs the same function as a mouse button. Specifically, for example, after the line-of-sight position is introduced to a specific position on the screen, the operation of “biting” is performed, and the switch is turned on and the determination input is performed. In addition, a system in which the switch is turned on and the determination input is performed by repeating the operation of “blinking” a predetermined number of times can be adopted. In addition, it is possible to adopt a method in which the switch is turned on and the decision input is performed when the magnitude of the “speech” emitted is a certain value or more.

  The electronic form system part in the electronic form gaze input device is a process for providing the electronic form requested by the user, and the user fills in the electronic form with the above-mentioned gaze input system part. Processing for providing a GUI (graphical user interface) environment, processing for registering and saving the filled electronic form, and the like. All of these processes can be configured so that the electronic form system part can perform it alone. However, in general, the electronic form server system in the background and the electronic form system as a client are configured to perform a coordinated process. That is, when a computer system downloads data and programs prepared in the server system via a network, the computer system performs a process in cooperation with the server system to function as an electronic form system.

  The electronic form is described in a markup language with a tag, for example, HTML (HyperText Markup Language), XML (Extensible Markup Language), TeX, or the like. The electronic form is composed of design data and input data. The design data corresponds to an unfilled document in a normal paper application form. The input data corresponds to items handwritten by the user. Design data is a field that indicates frame data including document ruled lines, characters, shaded images, etc., input positions (input fields) of items to be input by the user, and attributes described in a tagged format that controls character display Includes invisible data including data and management information for the electronic form. A tag is prepared for each necessary input such as a name tag and an address tag. The input data is data corresponding to the input items input by the user based on the display of the design data, and the input data also includes character data and attribute data.

  The GUI provides an environment in which a user fills in an electronic form by displaying a form display screen, a character input screen, and a line-of-sight position output by the line-of-sight input system described above on a display device. The display device for the form display screen and the display device for the character input screen may be separated, but the form display screen and the character input screen may be displayed simultaneously on one display device. The GUI updates necessary screens based on an input operation performed by the user using the line-of-sight input system while looking at the display device.

  The form display screen is a screen that displays an electronic form. Initially, only the design data, that is, an unfilled electronic form is displayed. When input is made in the input field of the electronic form and the input data is generated, the screen is updated to reflect the input content in the input field. For example, the input character string is added (overwritten) to the input field.

  The character input screen is a screen composed of input key elements corresponding to the type of information to be input. The user selects one of the input fields (input areas) in the electronic form displayed on the form display screen. In the line-of-sight input device for electronic forms of the present invention, this selection is performed by entering the line-of-sight position in the input field and performing a decision input. The input field is accompanied by attribute information of the input field. The attribute information is described in a tagged format as described above. An information type to be input is specified from the attribute information, and a screen composed of input key elements corresponding to the specified information type is displayed on the character input screen.

For example, when the attribute information is “zip code”, the information type to be input is “numerical value”, and the screen composed of the input key elements is “ten-key”. The numeric keypad can be, for example, * − +. It may contain keys such as Enter.
When the attribute information is “name”, the information type to be input is “kanji”, and the screen composed of input key elements is “kana input keyboard” or “romaji input keyboard”. This keyboard layout is not related to the standard keyboard layout, and only the minimum number of keys required for input can be optimally arranged to improve operability.
When the attribute information is “reading”, the type of information to be input is “half-width katakana”, and the screen composed of input key elements is “kana input keyboard”. Of course, this keyboard layout is not related to the standard keyboard layout, and only the minimum number of keys necessary for input can be optimally arranged to improve operability.
When the attribute information is “mail address”, the type of information to be input is “single-byte alphanumeric characters”, and the screen composed of input key elements is “single-byte alphanumeric input keyboard”. Of course, this keyboard layout is not related to the standard keyboard layout, and only the minimum number of keys necessary for input can be optimally arranged to improve operability.

Next, description will be made with reference to the explanatory view of the line-of-sight input device for electronic forms of the present invention shown in FIG. In FIG. 1, 1 is a line-of-sight input system, 2 is form data, 3 is dedicated software, 4a and 4b are line-of-sight tracers, 5a and 5b are decision input sections, 6 is a display section, 7 is a form display screen, and 8 is character input. A screen 9 is an input form display screen.
The line-of-sight input system 1 schematically shows a main part of a computer system. The storage unit stores form data 2 and dedicated software 3. The form data 2 is electronic form data and is composed of design data and input data. The dedicated software 2 is software for making it possible to suitably enter the entry field. For example, the character input screen 8 suitable for the selected input field in the form data 2 is displayed, input processing for the input field is performed, and input data is generated when input is performed.

The line-of-sight tracers 4a and 4b detect the movement of the eyeball of the person who inputs characters to the electronic form and output data corresponding to the line-of-sight position to the line-of-sight input system 1. The line-of-sight input system 1 inputs the data, calculates the line-of-sight position on the display unit 6, and displays the line-of-sight position as a pointer (cursor).
The decision input units 5a and 5b are decision input means for linking with a portion pointed to by the line-of-sight position (pointer) and performing an input to activate and activate a function existing at that position. For example, the decision is made by moving the eyeball and placing the line-of-sight position in one of the input fields in the electronic form displayed on the form display screen. Then, the function existing in the input field selected by the determination is activated, and the character input screen 8 suitable for the input field is displayed.
The display unit 6 is a display device such as a liquid crystal display, and displays form display screens 7 and 9, a character input screen 8, a line-of-sight position, and the like.

Next, the operation of the line-of-sight input device for electronic forms according to the present invention will be described with reference to FIGS. FIG. 2 is a flowchart showing a process of input field line-of-sight input in the electronic form line-of-sight input device of the present invention.
First, in step S1 (menu screen display) in FIG. 2, the electronic form line-of-sight input device (client system) is activated and connected to the server system via the network. Then, data such as a menu screen and a processing program is transferred from the server system. The electronic form gaze input device downloads them. A menu screen is displayed on the display unit 6 (not shown).

Next, in step S2 (display desired electronic form), the desired electronic form is selected on the menu screen. This selection is performed by line-of-sight input using a line-of-sight tracer and a decision input unit. Thereby, the line-of-sight input system 1 of the line-of-sight input device for electronic forms downloads the design data and the like of the electronic form and stores it in the form data 2 and the like, and displays the electronic form on the form display screen 7. The form display screen 7 of the display unit 6 in FIG. 1 shows the screen in that state.
The form display screen 7 is provided with not only an electronic form but also a control button (not shown) such as an end button for ending the input field line-of-sight input in the selected electronic form.

Next, in step S3 (moving the line-of-sight position), as shown in FIG. 1, the user turns his / her line of sight to the reading field, which is one of the input fields in the electronic form displayed on the form display screen 7. The line-of-sight tracer 4a detects the movement of the user's eyeball and outputs data corresponding to the line-of-sight position to the line-of-sight input system 1, and the line-of-sight input system 1 inputs the data and calculates the coordinates on the display unit 6 The line-of-sight position (pointer) is displayed in the coordinates. As a result, the line-of-sight position moves to the reading field.
Next, in step S4 (selecting an entry frame or the like), the user performs a decision input by operating the decision input unit 5a while keeping the line-of-sight position in the reading field.

  Next, in step S5 (display a character input screen), the line-of-sight input system 1 displays a character input screen 8 composed of input key elements corresponding to the information type to be input from the attribute information of the selected input area. In this case, since the attribute information is “phonetic”, the type of information to be input is “half-width katakana”, and the screen composed of input key elements is “kana input keyboard”. As described above, this keyboard layout is not related to the standard keyboard layout, and only the minimum number of keys required for input can be optimally arranged to improve operability.

  Next, in step S <b> 6 (moving the line-of-sight position), the user turns his line of sight to the “c” key which is one of the input keys on the kana input keyboard displayed on the character input screen 8. The line-of-sight tracer 4a detects the movement of the user's eyeball and outputs data corresponding to the line-of-sight position to the line-of-sight input system 1, and the line-of-sight input system 1 inputs the data and calculates the coordinates on the display unit 6 The line-of-sight position (pointer) is displayed in the coordinates. As a result, the line-of-sight position moves to the “c” key.

  Next, in step S7 (selecting a character key or the like), the user operates the decision input unit 5a and makes a decision input while keeping the line-of-sight position in the “c” key. With this input, the line-of-sight input system 1 performs processing to update the input data of the electronic form. Then, the updated input data is reflected in the display of the electronic form on the home display screen. In this case, the character “ha” corresponding to the selected “ha” key is displayed in the reading field of the electronic form.

When all characters in the selected input field in the selected electronic form have been input, the all character input end button (for example, the Enter key) which is one of the control buttons provided on the character input screen 8 in this step. ) By gaze input. With this input, the line-of-sight input system 1 updates the input data of the electronic form. In this case, “Hayashi Taro” is displayed in the reading field of the electronic form as shown on the home display screen 9.
In addition, when input to all entry frames in the selected electronic form is completed, the entry of all entry frame end buttons (for example, the Esc key), which is one of the control buttons provided on the character input screen 8 in this step, is line-of-sight input. Select by.

Next, in step S8 (all character input is ended), it is determined whether or not the all character input end button is selected. When not selected (no), that is, when all the characters “Hayashi Taro” are not input in the reading field, the process returns to step S6 and the subsequent steps are repeated. When it is selected (yes), the process proceeds to step S9.
Next, in step S9 (end all entry frames), it is determined whether or not the all entry frame end button is selected. When not selected (no), the process returns to step S3 and the subsequent steps are repeated.

In step S9, when the all entry frame end button is selected (yes), the line-of-sight input system 1 stores the input data generated in the above-described step in the form data 2 or the like. Then, returning to step S1, the subsequent steps are repeated.
The menu screen has a transfer button (or registration button) for transferring at least the generated input data to the server system in the form data 2 composed of design data and input data. When this transfer button is selected, the completed electronic form is transferred to the server system and registered in the server system.

  Next, a modified example of the line-of-sight input device for electronic forms of the present invention will be described. In this modified example, the electronic form line-of-sight input device includes an iris authentication means and has a function of performing personal authentication. In the above-described line-of-sight input device for electronic forms, a method of detecting a line-of-sight position based on a captured image obtained by capturing an eye movement with a camera is applied as a line-of-sight detection unit. And the iris (iris) of an eyeball is imaged with the camera of the said gaze detection means, and a captured image is obtained. The line-of-sight input system 1 incorporates a processing program for iris authentication means, and the iris authentication means authenticates the person based on the captured image.

  The line-of-sight input device for electronic forms according to this modification can be applied to various processes including identity authentication. An example of application will be described. This personal authentication is performed after the menu screen is displayed in step S1 described above and before the desired electronic form is displayed in step S2. When the user authentication is performed, the server system refers to registered data related to the user. Then, if there is data to be entered in the input field in the desired electronic form and exists in the registered data, the data is generated in the form of input data. Further, when the server system transfers a desired electronic form to the electronic form line-of-sight input device (client system), the input data is transferred together with the design data.

  As a result, the number of entry fields in the electronic form is displayed on the display unit 6 of the electronic form line-of-sight input device in an already-entered state. Of course, it is not necessary to input in the entry field. This significantly reduces the load when a person with a disability or a disabled person who is inconvenient to use their hands inputs.

It is explanatory drawing of the gaze input device for electronic forms of this invention. It is a flowchart which shows the process of the gaze input in the gaze input device for electronic forms of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Line-of-sight input system 2 Form data 3 Dedicated software 4a, 4b Line-of-sight tracer 5a, 5b Decision input part 6 Display part 7 Form display screen 8 Character input screen 9 Form display screen already input

Claims (2)

  Gaze detection means for detecting a gaze position based on the movement of the eyeball, decision input means for performing a decision input at the gaze position, a form display screen, a character input screen, and display means for displaying the gaze position When one of the plurality of input areas displayed on the form display screen is selected by indicating with the line-of-sight position and performing determination input with the determination input means, the display means determines from the attribute information of the selected input area A line-of-sight input device for an electronic form, characterized by displaying a character input screen composed of input key elements corresponding to the type of information to be input. The line-of-sight input device for an electronic form according to claim 1, wherein the line-of-sight detection means is a line-of-sight detection means for detecting a line-of-sight position based on a captured image obtained by capturing a movement of an eyeball with a camera. An eye-gaze input device for electronic forms, comprising iris authentication means for performing personal authentication based on a captured image obtained by imaging an iris (iris) in an eyeball.

Images