JP2001350578A - Device and method for input control by visual line and recording medium with control program to be inputted by visual line recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent an intention transmission device from malfunctioning by eliminating states wherein the gazing position of the eyeball can not be detected owing to movement of a face when a severely handicapped person including an ALS(amyotrophic lateral sclerosis) patient is using the intention transmission device and to provide a controller which is small-sized and excellent in economy. SOLUTION: A CCD camera is used instead of infrared as a method used to read in the face of a user as data. The amount of image data on the face (not only the position of the eyeball, but also the position and the direction of the face, and visual line direction) is increased. Further, a monitor is previously divided and which part the user is viewing is recognized and stored as an image (dictionary); when the user actually uses the device, which key the user is viewing is decided by selecting image data from the dictionary to eliminate malfunction, thereby making the device small-sized and excellent in economy.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for a seriously handicapped person or the like to easily and accurately control a communication device using a programmed computer.

[0002]

2. Description of the Related Art With the increase of the elderly population, many elderly patients in hospitals and homes are hospitalized and receive home care.
Among them, amyotrophic lateral sclerosis (ALS), which is an intractable disease
It is common to see that patients with speech impairment such as language disorders or the like often have trouble communicating with nurses, caregivers, etc., and at present, discomfort and stress remain on both sides. In particular, ALS patients at the end of life are generally said to move the area near their eyes until the latest.

Conventionally, in a device for controlling a communication device based on a line of sight, a gaze position of an eyeball is detected and controlled using infrared rays. However, when the user moves the face while using the communication device, the gaze position of the eyeball may not be detected, and the communication device may not be controlled. For this reason, there is a problem that the intention communication device is erroneously operated.

Further, a conventional device for controlling a communication device based on a line of sight is relatively large and expensive. Furthermore, it is difficult to adjust and control a device that controls a communication device based on a line of sight, and there is a need for a device that allows an ALS patient or the like to easily and accurately control the communication device.

[0005]

SUMMARY OF THE INVENTION The present invention eliminates a situation in which an ALS patient or the like uses a communication device to detect a gaze position of an eyeball due to a movement of a face, and does not operate the communication device incorrectly. To do. Further, the control device is made smaller and more economical.

[0006]

Therefore, in the present invention, there is provided a device which can be easily and accurately controlled without erroneous operation when using a communication device utilizing a line of sight. Further, the apparatus was small in size and excellent in economy.

More specifically, a CCD camera is used as a method for reading a user's face as data from infrared rays. In addition, the image data amount of the face (not only the position of the eyeball but also the position, orientation, gaze direction, etc. of the face) was increased. Furthermore, the monitor is divided in advance so that the user can see which part the user is looking at, and convert the data into an image (dictionary). At the time of actual use, the user can select the image data from the dictionary which key the user is looking at. In this way, erroneous operation is eliminated, miniaturization is realized, and excellent economy is achieved.

A specific procedure for easily and accurately inputting and controlling an instruction based on a line of sight of a communication device is as follows. Divide the monitor of the communication device in advance and ask the user to see each of the divisions, and
The user's image (face position, orientation, line-of-sight direction) is read and recognized by the D camera and converted into data (dictionary).
At the time of use of the communication device, the image data of the user is read by the CCD camera, and when the user actually operates, the image of the key on the monitor is compared with a dictionary which has been converted into image data. A similar image data is selected from the dictionary, and a conversation recorded in a location corresponding to the selected image data and an operation procedure are executed.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described based on examples. FIG. 1 schematically shows an input control device based on a line of sight in the present invention. 1 is a computer, 2 is a control BO
X and 3 are CCD cameras. When operating the communication device, the image data read by the CCD camera 3 is controlled by the control BO.
Through X2, the computer 1 processes data to operate the communication device.

FIG. 2 shows an arrangement of a communication device on a monitor. Keys from keys 1 to 12 are input when a conversation, an operation procedure, and the like displayed on the monitor are input, and the sound is obtained by staring at the key (moving the key), holding or blinking (determining the key). Is to occur. Further, the user can operate the communication device.

An embodiment will be specifically described. First, as preparation, the computer 1 converts the image into a data dictionary (dictionary) so that the computer 1 determines which part of the monitor the user is looking at. The procedure is to have the user look at any point on the communication device monitor for several seconds,
In each case, an image (information on the face position, orientation, line-of-sight direction, etc.) when the user sees an arbitrary point is read by the CCD camera 3 and is recognized by the computer 1 via the control BOX 2, Are stored in the computer 1 as a dictionary in advance. FIG. 3 shows an arbitrary point on a screen used when creating a dictionary. The CCD
The details of the method of reading from the camera 3 are as follows. When the monitor of the communication device is divided and the user sees an arbitrary point (27 × 17, the number of arbitrary points can be changed) divided in the X direction and the Y direction. Read the image.

Next, the user reads which part of the monitor the user is looking at. As a procedure, when actually operating the communication device, the user looks at the conversation (including pictograms and pictograms) displayed on the monitor, and looks at the key on which the procedure for operation and the like have been input. An image (information on the face position, direction, line of sight, etc.) when the user looks at the key is read by the CCD camera 3 and the control box 2
Through the computer 1.

Next, it is determined which part the user is actually looking at. To operate the communication device,
The user looking at the key is read by the CCD camera 3 and the image recognized by the computer 1 and the image data of the user stored in the computer 1 in advance as a dictionary (information on the face position, direction, line of sight, etc.) ) Is selected and a similar image is selected from the dictionary (determination).

Next, the user operates the key at the position where the user is looking. Keys at locations (on the monitor of the communication device) corresponding to the image data selected from the dictionary stored in the computer 1 in advance are input with a procedure for operating a conversation that has already been recorded. The operation of the key is a movement operation by staring at the key, and a fixed operation by holding or blinking.

Next, the key of the place where the user is staring is executed. The key in which the recorded conversation, the procedure for operation, and the like are input is executed by a moving operation and a fixing operation. When the conversation key is determined, the voice of the conversation determined by the user is generated, and when the key such as the operation procedure is determined, operations such as procedures necessary for the user to perform the determined operation are performed. .

The user can move the colored cursor (for example, a frame surrounding the key) freely by looking at the key on the monitor of the communication device. Confirmation is
It operates by staring at a key on the monitor of the communication device and holding or blinking for several seconds.

The object (face) is tracked by the CCD camera 3 in order to prevent an erroneous operation caused by a situation where the user's face moves and the gaze position of the eyeball cannot be detected. In the process of reading information such as a face by the CCD camera 3, a frame for reading data is provided every time near an object (face) so that the user's face moves and no erroneous operation is performed.

The input control apparatus and method based on the line of sight according to the present invention and the recording medium storing the control program to be input by the line of sight according to the present invention have been described above. Of course, it can of course be used more effectively.

[0019]

The present invention has the following effects by using the above-described method and apparatus.

The use of a CCD camera instead of infrared rays for reading face data has made it possible to reduce the size of the line-of-sight input control device. Furthermore, the above-mentioned apparatus which was conventionally expensive and was difficult to purchase has become excellent in economical efficiency and easy to purchase.

In addition, it has been difficult for a severely disabled person including an ALS patient to adjust and control a device for controlling a communication device based on his / her line of sight. Can be controlled easily and accurately.

The amount of data is increased not only in the position of the eyeball but also in the position, orientation, line of sight, etc. of the face. Further, the user is made to recognize in advance what part the user is looking at, and the data is converted into an image (dictionary). At the time of use, by selecting image data from the dictionary to determine which key the user is looking at, it is possible to eliminate the erroneous operation which has conventionally been a problem.

Conventionally, the face was moved and the gaze position of the eyeball could not be detected. However, by adding a function of tracking an object (information such as a face) by a CCD camera, erroneous operation can be eliminated. Can be.

In other words, accurate communication and simple operability by inputting information through the line of sight improve communication between a disabled person, a nurse, a caregiver, and the like, and have been conventionally accumulated on both sides. It also has the effect of reducing discomfort and stress (irritation of not being transmitted) and helping to improve the efficiency of nursing and nursing work.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an input control device based on a line of sight of the present invention.

FIG. 2 is an explanatory diagram of a screen of a communication device operated by the input control device based on the line of sight of the present invention.

FIG. 3 is an explanatory diagram showing arbitrary points on a screen used when creating a dictionary of the input control device based on the line of sight of the present invention.

Claims (3)

[Claims] An apparatus for a severely handicapped person or the like to easily and accurately control a communication device by using a programmed computer, in which a user previously views arbitrary points on a monitor of the communication device, Means for reading and recognizing a user's image (information on the face position, direction, line of sight, etc.) at any point by a CCD camera and converting the image into a dictionary (dictionary); Means for reading the image of the monitor by the CCD camera, and comparing the image when the user looks at the key of the monitor with the image data in the dictionary which has been previously converted to a similar one from the dictionary. An input control by gaze, comprising: means for selecting image data; and means for executing a speech and operating procedure recorded in a conversation corresponding to the selected image data. Control device. 2. A device for a severely disabled person or the like to easily and accurately control a communication device by using a programmed computer, wherein a user sequentially views arbitrary points on a monitor of the communication device in advance. In each case, a process of reading and recognizing a user's image (information on the face position, orientation, line-of-sight direction, etc.) looking at an arbitrary point with a CCD camera and converting the data into a dictionary (dictionary); Reading the image with a CCD camera, and comparing the image when the user stares at the key of the monitor with the image data in the dictionary that has been previously converted to a similar one from the dictionary. An input control method based on a line of sight including a step of selecting image data and a step of uttering a conversation recorded in a location corresponding to the selected image data and executing an operation procedure. Law. 3. A recording medium storing a control program for a person with a serious disability or the like to easily and accurately control a communication device by a computer, wherein the control program is provided to a user in advance for monitoring the communication device monitor. Each point is sequentially viewed, and each time the user sees the arbitrary point (information on the face position, orientation, line-of-sight direction, etc.) is read and recognized by a CCD camera and is converted into data (dictionary). . At the time of operation, the image data of the user is read by the CCD camera, and the image when the user looks at the key of the monitor when the user actually operates is compared with the image data in the dictionary which has been converted into data in advance. A control program that allows the user to input instructions by line of sight, by selecting similar image data from the dictionary and executing the utterance and operation procedure of the conversation recorded at the location corresponding to the selected image data Recording medium.