JP2003187232A - Communication system using blink - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To develop a communication system using blinks with a small burden on such a user as a patient of serious amyotrophic lateral sclerosis (ALS), by which burdens on a care giver and nurse, etc., can also be relieved, automatic detection and automatic tracking of both eyes are enabled. <P>SOLUTION: In the communication system using the blinks, the burden to be imposed on the user is small since it is a complete non-contact type using a CCD camera for enabling the automatic detection and the automatic tracking of both eyes and the burdens on the care giver and the nurse are also small since eye positions of the user are automatically recognized and adjusted on the system's side simply by turning a power on by directing the CCD camera to a face of the user by the care giver and the nurse. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention can control a personal computer using eye blinks for patients with severe amyotrophic lateral sclerosis (ALS) and the like who have lost the vocal function and the motor function of the extremities. And contactless communication system.

[0002]

2. Description of the Related Art Conventional systems are roughly classified into contact type and non-contact type. In the contact type, a sensor is attached directly to a part of the patient's body to convey the patient's intention. As an example, by attaching an optical sensor to a device such as glasses that is worn on the face and aiming the sensor at the eye, the reflected light when the eye is open and when it is closed is sensed as an electrical signal. Input on a computer.
This system is uncomfortable because the red LED light for blink detection is always in the field of view. In addition, there is a disadvantage that the user is always required to wear glasses, which imposes a physiological burden on the user.

In the non-contact type, a sensor, a CCD camera, or the like is installed at a position apart from the patient to detect the eye movement of the patient (Keiji Taniguchi: Basics of Image Processing Engineering, Kyoritsu Publishing Co., Ltd., 1996; Nagao). Makoto: Digital Image Processing, Modern Science Company, 1978). For example, one that senses the opening and closing of eyes from a position away from the patient, and a CCD with a telephoto function
2. Description of the Related Art There is a method in which image processing is performed using a camera to determine eye movements and blinks of a patient. Many non-contact type communication systems use line-of-sight input. In order to calculate the mobility of the eye due to the line-of-sight input, there is one that tries to capture the movement of the eye by irradiating red LED light to the eye from a distant position. If you move it a little, you will not be able to gaze. On the other hand, there is also a method in which a CCD camera with a high magnification is used to capture the eye movement by image processing.
Since the CD camera is used, if the direction of the face is slightly moved from the state where the line of sight can be input, the line of sight cannot be captured.

[0004]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention A communication system using blinks capable of automatic detection and tracking of both eyes, which can reduce the burden on users such as ALS patients and reduce the burden on caregivers and nurses. Development is a challenge.

[0005]

A communication system using blinks according to the present invention utilizes the features of both contact type and non-contact type in the prior art, and improves the shortcomings of the two types as follows. It has features. 1) In the contact type, a non-contact type was adopted in order to eliminate the burden of always wearing a sensor. 2) It is not necessary to use a red LED light sensor or the like that senses the brightness of light, because the eye open / closed state and the patient's movement are determined only by image processing. 3) Fixed cameras used in conventional non-contact systems may render the system unusable when the patient's head moves. To improve this,
By using a movable camera that can control the orientation and magnification of the camera from a personal computer, the position of the patient's eyes can be tracked even if the head is moved slightly, and the system can be used even when it is moving. A CCD camera, a video camera, or the like can be used as the camera used in this system. In the case of a CCD camera, it is preferable to use a CCD camera whose control code is VISCA compatible. Further, when using a video camera, it is necessary to perform processing for digitizing an image.

In order to easily perform the operation at the time of starting the system, the contour of the face and the position of the eyes are automatically recognized. For this purpose, an image including a face is taken in from a CCD camera, and the face is cut out and the coordinates of both eyes are obtained from this information. At this time,
In consideration of the situation where the user is not in the center of the camera, the difference between the coordinates of the middle of both eyes and the center of the CCD camera is calculated, and the CCD is automatically adjusted so that the center of the CCD camera is in the middle of both eyes. Move the camera. This allows CC
Since the center coordinates of the D camera are in the middle of both eyes, zooming the CCD camera expands the face portion to the center of the image. As a result, the eyes can be automatically enlarged simply by pointing the CCD camera toward the user, and the burden on the caregiver and the nurse for initial setting can be greatly reduced.

When the position of the user's face moves for some reason, the positions of the eyes are displaced from the CCD camera in the conventional system, and the blink cannot be detected. In this case, the caregiver or the nurse must manually readjust the CCD camera to face the user's face or return the user's face to its original position. With the system of the present invention, the position of the eyes can be constantly tracked by constantly monitoring the eyes of the user and automatically re-adjusting when the position changes. Therefore, the troublesomeness of manual readjustment by a caregiver or a nurse can be prevented. In particular, a caregiver who is not accustomed to a computer can greatly benefit from the system of the present invention.

Next, the processing flow of this system will be described in more detail. First, the user acquires an image in which the user's line of sight is directed to the display from the camera, and the position of the eyes is detected from there. For the eye position detection, commercially available image analysis software or self-made image analysis software can be used as long as the software can handle moving images. However, it is necessary to be configured from a library for detecting a face most matching a database (data such as a face image registered in advance) from a moving image or a still image. In the examples of the present invention, FaceIt, which is commercially available image analysis software, was used. Next, the computer sends a command to the camera so that the position of the found eye is in the center of the image. The camera magnifies the image when the eye position is in the center of the image. A template image including the eye region is registered from the enlarged image, and template matching is performed.

Pattern recognition can be used for the eye open / close determination using template matching. Pattern recognition is an image processing technique that determines which standard pattern the input pattern is the same as the one prepared in advance (Keiji Taniguchi: Basics of Image Processing Engineering, Kyoritsu Publishing Co., Ltd., 1996; Makoto Nagao: Digital Image Processing) , Modern Science Co., Ltd., 1978), and outputs the result through the steps of preprocessing, feature extraction, and pattern matching for the input pattern (FIG. 1).

As for the input pattern, if necessary
Pretreatment. Pre-processing includes noise removal and size control.
There are classifications and rotation corrections. Next, the characteristic parameters
Feature extraction is performed using the pattern. Characteristic parameter
Various data can be considered depending on the target. In the present invention
Uses template matching for feature extraction. Target
A simple way to extract a particular shape from an image.
One method is template matching. Specific shape
It is called a template, and the weight of the template image and the input image
The higher the degree, the more similar the pattern
You In pattern matching, the features of standard patterns
Distance between the two by comparing the characteristic amount of the input pattern and the characteristic amount of the input pattern
By calculating the standard pattern closest to the input pattern.
Determine the turn. The determination method is for all regions of the feature
The input pattern is compared with the one that has the smallest distance.
Is determined as the standard pattern. In this study, the distance D
We decided to use the sum of squared differences as the calculation method. less than
The formula is shown below. However, M₁~ M _nIs the feature of standard pattern
Quantity, I₁~ I_nIs the feature quantity of the input pattern.

[0011]

[Formula 1]

An outline of the image recognition algorithm by template matching will be described below. In the present invention, 32
Recognition is performed by template matching for a color image of 0 × 240 pixels and 256 gradations of RGB. In order to speed up the matching, first, the pyramid structure (Mikio Takagi, Yoshihisa Shimoda: Image Analysis Handbook, The University of Tokyo Press, 1978) is used for images one level higher. In the image pyramid structure, for a 2 ⁿ × 2 ⁿ image, a 2 × 2 pixel area is made to correspond to one pixel on one level, and 2 ^n-1 × 2 ^n-1
The operation of making the image is repeated. It is used in various fields of image processing for speeding up. Next, matching is performed again around the obtained candidate area to extract a final candidate area. In this research, a 3 × 3 pixel region was made to correspond to one pixel on one level, and a four-layer pyramid structure was created (FIG. 2).

[0013]

[Table 1]

A binarization method was used to determine whether the eyes were opened or closed. Now, assuming that the image f is in the density range [a, b] and the binarization threshold is T (a ≦ T ≦ b), the binarization process is generally expressed by the following equation.

[0015]

[Formula 2]

F _T is a binary image. Normally, 1 is assigned to the value of the target area (black area) and 0 is assigned to the value of the background area (white area). However, in the present study, the opposite is true because the pupil and eyelashes of the eye turn black when binarized, and the number of pixels of the eye and eyelashes are used to determine the opening and closing of the eye. Then, 1 was treated as a background region (white region) and 0 was treated as a target region (black region). In addition, the origin of coordinates is at the upper left, the first argument of the image f is the horizontal coordinate, and the second argument is the vertical coordinate. In order to determine whether the eyes are open or closed from the matched image (hereinafter referred to as the recognition image), the recognition image is binarized, and the total number of black pixels at that time determines whether the eyes are open or closed. The middle value of the number of black pixels in the open state is set as the eye opening / closing threshold value to determine the eye opening / closing. As the binarization threshold value, a value determined in advance for each user is used.
In general, matching images such as eyebrows and hair other than eyes
If there is no part that becomes black when it is binarized, the pupil part becomes binarized and becomes black when the eyes are open,
Other parts become white. With your eyes closed,
The eyelashes are only binarized and blackened.

In general, the number of black pixels is large when the eyes are open, because the pupil portion becomes black when binarized compared to when the eyes are closed. With the eyes closed, only the eyelashes turn black. However, a person with narrow eyes and long eyelashes may have more black pixels when the eyes are closed. In this case, by lowering the threshold for binarization,
It can prevent the eyelashes from turning black when the value is converted. Equation 3 shows an equation that gives the eye opening / closing threshold value.

[0018]

[Formula 3]

In order to determine the eye opening / closing threshold, the user blinks for about 5 seconds after registering the template. Here, EP _max and EP _min respectively represent the maximum value and the minimum value of the number of black pixels when the recognized image is binarized, and the intermediate value obtained by dividing them by 2 is used as the eye opening / closing threshold value. Using the relationship between the number of pixels in the binarized pixel and Equation 3, the current eye state can be represented by Equation 4. Where E
_state represents the _state of eyes, and EP represents the number of black pixels when binarized.

[0020]

[Formula 4]

The intention of the user is expressed by using the characters (screen keyboard) displayed on the display.
This is done by selecting a character, pictogram or picture. The character selection method is a direct method that selects the character by directing the line of sight to the position of the character you want to select, or an indirect method that waits until the character you want to select blinks and then blinks to select the character when the character you want to select blinks. There is a law. Compared to the indirect method in which characters can be selected only with a switch, the direct method requires accurate analysis of eye movements, which complicates the system and increases the cost of the system when completed. Therefore, in the present invention, an inexpensive indirect method is adopted. In the present invention,
When selecting a character, in order to prevent the character from being selected with an unconscious blink, it is judged that the conscious blink has been made if the eyes are closed for a certain period of time, and it blinks when the eyes are closed on the on-screen keyboard. The selected character is determined to have been selected.

On the display, the characters of the Japanese syllabary or the characters that are frequently used are copied, and the characters on the screen are flashed from top to bottom, left to right, and the user is conscious when the desired character flashes. Make a targeted blink. The computer makes a judgment as to whether the eyes are open, the eyes are closed, or a conscious blink has been performed based on the result of the template matching, and if a conscious blink is performed, then Display the character selected for in a text editor on the display. The criteria for determining the intentional blink and the unconscious blink are determined based on the result of measurement using the actual device of the present invention. Investigate the relationship between the time to switch the item of intention display and the item when selecting the item of intention display by blinking, and when changing the switching time of the item on the screen, select the item of intention display by blinking and succeed. It can be determined by measuring the rate. From the measurement results by 10 healthy persons, it was optimal to judge that a conscious blink occurred when the eyes were kept closed for 1 second (Example 3).
reference).

Since the communication system using the blink of the present invention is a completely non-contact type camera using a camera capable of automatically detecting and tracking both eyes, particularly a CCD camera,
This is a system that does not require a user such as a patient to attach a detection device, a cable, or the like, and has a small burden on the user. For the caregiver and the nurse, the system side automatically recognizes and adjusts the position of the user's eyes simply by turning on the power of the caregiver or nurse with the CCD camera facing the user's face. It is a system with a low burden. The communication on the display uses not only letters but also pictograms and pictures so that anyone can easily understand it regardless of age. Further, the information selected by the intention display may be added with information such as date and time and stored as history information. Therefore, it is possible to utilize it as a guideline for better nursing and care by analyzing the relationship between intention content and time based on these information.

Even if for some reason the user's face moves and the eye position changes, the system automatically tracks the eye position,
It is a great advantage of the present invention that it is not necessary to readjust or reset the system because the eye position can always be captured. Even in the questionnaire survey for ALS patients,
The patient's wishes are that he / she can't move and stays asleep in bed for a long time. Since it hurts, there are many requests for postural changes. In the conventional fixed type system, it is not possible to move the face while using the system, but even if the user's face moves and the eye position changes, the system automatically tracks the eye position,
It can be said that the system of the present invention capable of always capturing the eye position is a superior system.

The system of the present invention also has the functions of individual authentication and individual setting so that a plurality of users can use one computer. For personal authentication and individual setting, first, the positions of both eyes are detected by the blink of the user, then the first-stage personal authentication is performed with the registered face image, and if authenticated, the password data for both eyes or one eye blinks. By executing the second-stage personal authentication by inputting, the system is individually set for each user, which is executed by a program. As described above, the communication system using the blink of the present invention can be said to be a system that is superior to conventional systems as a patient-friendly system when comprehensively judged.

[0026]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described based on examples with reference to the drawings. In particular, an example of performance evaluation of the communication system using the blink according to the present invention will be described below.

[0027]

Embodiment 1 When using a communication system using blinks according to the present invention, when a CCD camera automatically recognizes and tracks a patient's eyes, how the patient's posture and indoor environment change are automatically recognized. The influence on the automatic tracking was investigated, and the performance of the communication system (FIG. 3) using the blink according to the present invention was evaluated.

The influence of the angle of the bed is investigated by changing the angle of the bed on which the subject is sleeping with respect to the CCD camera to bed horizontal, 15 degrees, and 30 degrees, and automatically recognizing the eye position, the time required for the automatic tracking, and the recognition. The judgment was measured by an actual machine. Furthermore, the influence investigation by the face orientation is that the face orientation of the subject at each angle of the bed is front, top, bottom,
The positions were changed to left and right, and the recognition rate and time of automatic eye position recognition and automatic tracking were measured (Table 2).

[0029]

[Table 2]

The effect of the room lighting was investigated by changing the brightness of the room lighting to 100, 700, 1600 lux in addition to the angle of the bed and the direction of the subject's face. After that, it was changed to left and right, and the recognition rate and time of automatic recognition of eye positions and automatic tracking were measured (Table 3). We selected 100 lux hospital room, 700 lux reception room / dining room, and 1600 lux emergency room as the criteria for selecting the brightness of illumination. The illumination was measured as the brightness of the entire room with the sensor part facing upward at the position of the subject's eyes. As the illuminance meter, a digital illuminance meter SLX-1000 manufactured by Shiro Sangyo Co., Ltd. was used. A fluorescent lamp and an incandescent lamp (a lamp) were used as a light source, and a dimmer installed in a room was used for dimming (illuminance adjustment) of the fluorescent lamp. For light control of the incandescent lamp, LC201 manufactured by ODELIC was used.

[0031]

[Table 3]

The performance evaluation of the communication system using blinks according to the present invention was carried out in cooperation with 10 healthy persons and 2 ALS patients (Table 4).

[0033]

[Table 4]

[0034]

[Example 2] From the test results of automatic detection and automatic tracking of eye positions, a diagram of the number of recognitions and the average recognition time for two different elements was created. The number of times of recognition is the total number of times the eye positions of 10 healthy people are recognized for the two items written in the caption of the figure, if they are the experimental results of healthy people,
In the case of the experimental result of the patient, it is the total number of times the eye positions of the two patients are recognized. The average recognition time is a value obtained by dividing the total time taken by the number of recognitions by the number of recognitions,
It is the average time required to recognize the position of the eyes.
FIG. 4 shows a summary of the effects of light source illuminance and face orientation on the number of recognitions and average recognition time in healthy subjects.
It was shown to. There was almost no effect on the illuminance of the light source. When the orientation of the face was tilted to the left or right with respect to the CCD camera, the recognition rate slightly decreased and recognition took a long time, but there was no problem in practical use.

FIG. 5 shows a summary of the effects of the bed angle and the light source position on the number of times of recognition and the average recognition time in healthy persons. The recognition rate had almost no effect on the light source position. Regarding the angle of the bed, there was a tendency that the higher the angle, the better the recognition rate, but there was no practical difference. Moreover, the recognition time was almost constant upon recognition.

FIG. 6 shows a summary of the effects of the bed angle and the light source illuminance on the number of times of recognition and the average recognition time in a healthy person. The illuminance of the light source had little effect on the recognition rate and average recognition time. The recognition rate tended to decrease as the bed angle decreased, but when recognition was performed, the recognition time was almost constant.

FIG. 7 shows a summary of the effects of the angle of the bed and the orientation of the face on the number of times of recognition and the average recognition time in a healthy person. When the face was facing forward and above, the bed angle had almost no effect. The recognition rate declined when the face was facing down, left, and right, and it tended to take longer to recognize, but it was found that this can be handled by adjusting the angle of the bed.

FIG. 8 shows a summary of the effects of the light source illuminance and the light source position on the number of recognitions and the average recognition time in a healthy person. There was almost no effect on the illuminance of the light source and the position of the light source.

FIG. 9 shows a summary of the effects of the face orientation and the light source position on the number of recognitions and the average recognition time in a healthy person. The effect on the recognition rate with respect to the position of the light source was small, but with respect to the face orientation, the recognition rate decreased especially when facing to the left and right. As for the recognition time, it was found that when the face is facing left and right, it takes more time than in the other directions. Therefore, next, the effects of the bed angle and the light source position on the number of recognitions and the average recognition time were examined.

FIG. 10 shows a summary of the effects of the bed angle and the light source position on the number of recognitions and the average recognition time in the patient. If the bed angle is low,
The recognition time is affected when the light source positions are on the left and right of the patient. When the bed angle is high, the light source position has little influence on the recognition rate.

FIG. 11 shows a summary of the effects of the bed angle and the face orientation on the number of recognitions and the average recognition time in the patient. The angle of the bed has some influence, but the recognition rate decreases depending on the face orientation. Unlike normal people, the recognition rate when looking to the right is high. Also, the left is barely recognized. The cause is considered to be that the position of the CCD camera was a little further to the right in the healthy person than in the proof experiment. When I turned my face down, it became difficult to see the screen, so all of it was unrecognizable.

FIG. 12 shows a summary of the effects of the orientation of the patient's face and the light source position on the number of recognitions and the average recognition time. As a result, it was found that the recognition rate was not affected by the position of the light source. However, the recognition rate and the recognition time are slightly affected when the face is turned to the left and right.

In the case of a patient, as a result of an experiment of automatic detection and automatic tracking of eyes, the orientation of the face with respect to the CCD camera has a great influence on the automatic recognition. In particular, the recognition rate is low when the face is facing left or right. CCD
The image of the position of the eyes captured by the camera becomes smaller when it is turned sideways, and the recognition rate decreases. Also, if the face is down (C
The recognition rate also decreased when the angle was lower than that of the CD camera or when the player's body was leaning back. This is considered to be due to the case where the nostrils are mistakenly recognized as eyes and the case where the angle is shallow and the area of the eyes is small and the recognition rate is reduced. The recognition rate also decreased when the bed angle was as low as 0 degree. However, the position of the light source and the illuminance did not significantly affect the automatic recognition. Patient Kr
In a (ventilator user's) experiment, a phenomenon that cannot be recognized from the front occurred. It is considered that the problem was that the pillow pattern (blue floral pattern) used by the patient had an effect. When I tested the pillow with a plain towel, I was able to recognize it. Also, for people with dark eyebrows, the probability of catching the eyebrows with eyes was high, and those who did not wear glasses had a higher recognition rate without glasses, but If the angle is high and faces the front, there is almost no difference (glasses recognition rate 69.8%, glasses not recognition rate 81.87%). From the above results, in order to increase the recognition rate, 1) set the face of the CCD camera so that the face is facing the front as much as possible, 2) increase the bed angle if possible, 3) set the background of the face ( It was confirmed that the recognition rate could be increased by making the pillows, sheets, etc.) plain.

[0044]

[Third Embodiment] In the communication system using blinks according to the present invention, the intention content is determined by using the blink when the intention display item is selected. At this time, if the unconscious blink of the user is recognized as a blink for decision making, it will be an obstacle to the efficient use of this communication system. Therefore, in this system, a standard for distinguishing between unconscious blinks and conscious blinks is provided.

Using an actual machine, the subject's eyes were caught by a CCD camera, and in this state, they blinked for about 5 seconds. The provisional eye opening / closing threshold value was determined by the number of black pixels obtained by binarizing the eye position image at that time. The provisional eye opening / closing threshold is
At that time, it was set to an intermediate value between the maximum value and the minimum value of the number of pixels. After that, they were asked to select the intention item, and the blinking time of the intention display and the unconscious blinking time at this time were measured by an actual machine. The intention item was considered to have been selected when the item continued to be closed for about 1 second when the item flashed. However, if the eyes were closed for one second after the eyes were closed and the item was selected, it was determined that the eyes were closed and the intention item was not selected.

For 10 healthy subjects and 2 ALS patients,
Eye open / close threshold experiment, intentional blink and unconscious blink were measured, respectively. Of the measurement results, the results of the experiment of two healthy persons, one with glasses and one without.
3) and the experimental results (FIG. 14) of 2 ALS patients are shown.

In the blink determination, the portion in which the number of eye pixels is smaller than the threshold value obtained by the eye opening / closing threshold value survey is determined to be the blink. Table 5 shows the blink times of all the subjects. The unconscious blink was 0.26 seconds at the shortest and 0.54 seconds at the longest. Regarding conscious blinking, it was decided that the first second had to be closed when selecting an item, but the actual measurement result was between 1.06 and 4.53 seconds. From this result, the judgment of unconscious blink or conscious blink can be selected as the unconscious blink if it is within 1 second, and as the conscious blink if it is more than 1 second, and the intention display content of the communication system can be selected. It was concluded that

[0048]

[Table 5]

The switching time of the intention display item is 1 second and 0.
When the time was changed to 75 seconds and 0.5 seconds, the success rate was measured whether or not the intention display item specified by blinking can be accurately selected. In addition, I also examined the optimal switching time of the intention display item. As shown in FIG. 15, in order to prevent that you choose a character to predict whether or not there at the very beginning of the item "Hello" is when to blink, and the state does not look the part of the intention display items from the subject, "Hello When "" flashes, I decided to use the blink. This was repeated 4 times with a fixed switching time, and the success rate was measured. This experiment was conducted on 10 healthy subjects. For 10 healthy people, the switching time of intention items is 1 second, 0.75
Table 6 shows the success rate of selecting the specified item by blinking when the time was changed to 0.5 seconds.

[0050]

[Table 6]

As a result of measuring the success rate of character selection with respect to the change of the intention item switching time, if the switching time is 1 second, the success rate is as high as 97.5%. This means that when the switching time is set to 1 second, even if the first item suddenly blinks, there is enough time to blink and then close the eyes to select a character. When the switching time was 0.75 seconds, the success rate was 82.5%, which was a little lower than when it was 1 second, and was less than 50% at 0.5 seconds, showing a fairly low success rate. At 0.75 seconds, even if the switching time is slightly faster than 1 second, after recognizing that the character blinked,
I have time to close my eyes and select letters, but 0.5
In seconds, even if you close your eyes and consciousness from the character flashes, it will be to select the next character "Thank you" and "Hello", which means that there is no room to select a specified character. Although Figure 16 A) shows the experimental results when the switching time is 0.75 seconds, immediately after the first three successful character selection, the word "hello", as shown in FIG. 16B) flashes, eye You can see that you have closed and selected the character.
Figure 16 C), which shows an enlarged view of when that failed at the end of Figure 16 A), it is not possible to close your eyes when the "Hello" flashes, the following items "thank you" flashes I found that I closed my eyes.

[0052]

The communication system using the blink of the present invention is capable of automatically detecting and tracking both eyes.
Since it is a non-contact type that uses a CD camera, the burden on the user is small. In addition, the caregiver or the nurse only needs to turn the power on with the CCD camera facing the user's face, and the system automatically recognizes and adjusts the position of the user's eyes. It is a system with less burden on the human body and is useful for communication with patients with severe amyotrophic lateral sclerosis (ALS) and the like.

[Brief description of drawings]

FIG. 1 is a diagram showing a process of pattern recognition.

FIG. 2 is a diagram showing a pyramid structure in an image recognition algorithm.

FIG. 3 is a diagram showing an example of the configuration of a communication system using blinks according to the present invention.

FIG. 4 is a diagram showing the number of recognitions and the average recognition time for the light source illuminance and the face orientation of a healthy person.

FIG. 5 is a diagram showing the number of recognitions and the average recognition time with respect to the angle of the bed and the light source position of a healthy person.

FIG. 6 is a diagram showing the number of times of recognition and the average recognition time with respect to the angle of the bed and the illuminance of the light source of a healthy person.

FIG. 7 is a diagram showing the number of times of recognition and the average recognition time with respect to the angle of the bed and the direction of the face of a healthy person.

FIG. 8 is a diagram showing the light source illuminance of a healthy person, the number of times of recognition for a light source position, and the average recognition time.

FIG. 9 is a diagram showing the number of recognitions and the average recognition time with respect to the orientation of the face and the light source position of a healthy person.

FIG. 10 is a diagram showing the number of recognitions and the average recognition time with respect to the angle of the bed of the patient and the light source position.

FIG. 11 is a diagram showing the number of times of recognition and the average recognition time for a patient's bed angle and face orientation.

FIG. 12 is a diagram showing the number of recognitions and the average recognition time with respect to the orientation of the patient's face and the light source position.

FIG. 13 is a diagram showing the eye opening / closing threshold value survey results of normal subjects Ba (using glasses) and Iw (without glasses), and the results of surveying intention display and unconscious blinking.

FIG. 14 is a diagram showing an eye opening / closing threshold value survey result of patients Kr and Na, an intention display, and an unconscious blinking survey result.

FIG. 15 is a diagram showing an example of a case where the intention display item is normally displayed and a case where the intention display item is hidden.

FIG. 16 is a diagram showing an experimental result (three times successful and one failure failed) when the switching time of the subject Yo is 0.75 seconds and an enlarged view thereof.

Continued front page    F-term (reference) 4C341 LL10 LL30                 5B057 AA19 BA30 DA11 DB02 DB06                       DB09 DC05 DC16 DC33 DC36                 5L096 AA02 AA06 BA02 CA04 DA05                       EA43 FA06 FA54 FA62 FA69                       GA12 GA51 HA02 HA05 HA07                       JA09 JA11

Claims (9)

[Claims] 1. A communication system using a blink capable of automatically detecting and tracking both eyes. 2. A communication system using blinks according to claim 1, wherein a function capable of automatically detecting both eyes and a camera capable of performing automatic tracking from position information obtained thereby are used. . 3. The communication system using blinks according to claim 1, which has a function of detecting both eyes and a personal authentication function. 4. The communication system using blinks according to claim 1, wherein the system can be individually set for each user. 5. The positions of both eyes are detected by blinking, and then the first-stage personal authentication is performed with the registered face image. If the authentication is successful, the second-stage by inputting password data by blinking both eyes or one eye. The individual setting of the system for each user is performed by performing the individual authentication of 1.
A communication system using the blink described in any one of 4 to 4. 6. The communication system using blinks according to claim 1, which has a mouse pointer function of a computer using the blinks of both eyes. 7. The communication system using a blink according to claim 1, which is capable of discriminating between a blink of intention and an unconscious blink. 8. The communication system using a blink according to claim 1, wherein the blink of the intention display and the unconscious blink are discriminated from each other by an eye opening / closing threshold value. 9. The position of both eyes is detected by blinking, then the first-stage personal authentication is performed with the registered face image, and if authenticated, the second stage is input by inputting password data by blinking both eyes or one eye. A recording medium that records a program that causes a computer to perform individual setting of the system for each user by performing individual authentication of.