WO2015072202A1

WO2015072202A1 - Information-processing device, method and program for detecting eye fatigue on basis of pupil diameter

Info

Publication number: WO2015072202A1
Application number: PCT/JP2014/071805
Authority: WO
Inventors: 亜由美加藤
Original assignee: ソニー株式会社
Priority date: 2013-11-18
Filing date: 2014-08-20
Publication date: 2015-05-21

Abstract

The purpose of the present invention is to provide an information-processing device, control method and program for detecting eye fatigue automatically without the user having to be conscious thereof and being capable of performing output control to reduce fatigue. An information-processing device is worn on the user's head and is provided with: an image pickup unit provided on the information-processing device for imaging the user's eyes; a measurement section for measuring pupil diameter on the basis of the image taken by the image pickup unit; a detection section for detecting eye fatigue on the basis of the pupil diameter measured by the measurement section; and a control section for performing a specified output control to reduce the eye fatigue detected by the detection section.

Description

[Name of invention determined by ISA based on Rule 37.2] Information processing device, method, and program for eye fatigue detection based on pupil diameter

The present disclosure relates to an information processing device, a control method, and a program.

In recent years, visual fatigue has been taken up as a major problem. As a cause of this, it is considered that the time for viewing an electronic screen has significantly increased due to the spread of electronic devices such as televisions, PCs (personal computers), mobile phone terminals, and smartphones.

For example, the following Patent Document 1 has been proposed as a technique related to such visual fatigue. Patent Document 1 discloses a visual fatigue level detection device that measures a fatigue level based on a temporal change in pupil diameter with respect to a predetermined light stimulus.

Further, in Patent Document 2 below, a stress determination system that detects a light reaction of a pupil when flash light is emitted to a subject from a captured image and determines a stress level of the subject based on the detected light reaction. Is disclosed.

Moreover, in the following Patent Document 3, the user's body temperature, brain wave, blinking frequency, skin electrical resistance, pupil size, heart rate, and other biological information are used to determine the user's tension state or fatigue state. And the stress care apparatus which generate | occur | produces the stimulus which gives the effect of tension relaxation or fatigue reduction based on the determination result is disclosed. As the generated stimulus, for example, a relaxing odor, color, or sound is given.

JP 2012-50759 A JP 2009-142469 A JP 2006-320621 A

However, the conventional fatigue detection technology and stress care technology required a place and time for detecting fatigue, and dedicated equipment for measuring the pupil diameter.

In addition, since the above-described fatigue detection technology and stress care technology are those in which the user consciously operates the detection device and the like, they are not used unless the user is conscious of fatigue by himself / herself. For this reason, it is difficult to make the user aware of fatigue or provide stress care in a state where the user cannot feel fatigue.

Therefore, the present disclosure proposes an information processing apparatus, a control method, and a program that can automatically detect eye fatigue and perform output control for reducing fatigue without the user being aware of it.

According to the present disclosure, an imaging unit that is provided in an information processing device mounted on a user's head and that measures the pupil diameter based on an imaging unit that images the user's eyes and an image captured by the imaging unit. A detection unit that detects eye fatigue based on the pupil diameter measured by the measurement unit, and a control unit that performs predetermined output control for reducing eye fatigue detected by the detection unit, An information processing apparatus comprising:

According to the present disclosure, provided in an information processing device mounted on a user's head, the steps of imaging the user's eyes, measuring the pupil diameter based on the captured image, and A control method is proposed, which includes a step of detecting eye fatigue based on the pupil diameter and a step of performing predetermined output control for reducing the detected eye fatigue.

According to the present disclosure, a computer is provided in an information processing device mounted on a user's head, and an image capturing unit that captures an image of the user's eyes and a pupil diameter based on a captured image captured by the image capturing unit. A measurement unit for measuring, a detection unit for detecting eye fatigue based on the pupil diameter measured by the measurement unit, and a control for performing predetermined output control for reducing eye fatigue detected by the detection unit We propose a program to function as a part.

As described above, according to the present disclosure, it is possible to automatically detect eye fatigue and perform output control for reducing fatigue without the user being aware of it.

Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is a figure for demonstrating the outline | summary of the information processing apparatus by one Embodiment of this indication. It is a block diagram which shows the basic composition of the information processing apparatus by this embodiment. It is a flowchart which shows the operation | movement process of the fatigue detection based on the pupil diameter by this embodiment. It is a figure for demonstrating the display output control for reducing fatigue. It is a figure which shows an example of the fatigue warning by this embodiment. It is a flowchart which shows the operation | movement process in the case of performing a fatigue detection based on a miosis speed. It is a flowchart which shows the operation | movement process of the fatigue detection based on the miosis / mydriatic speed according to the change of the brightness | luminance by this embodiment. It is a flowchart which shows the operation | movement process for improving the precision of the fatigue detection by this embodiment. It is a flowchart which shows the operation | movement process of the stepwise output control for reducing the fatigue by this embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. Overview of information processing apparatus according to an embodiment of the present disclosure Basic configuration Operation processing 3-1. Fatigue detection based on pupil diameter 3-2. Fatigue detection based on miosis / mydriatic velocity 3-3. Fatigue detection based on miosis / mydriatic velocity according to changes in brightness 3-4. Improvement of fatigue detection accuracy 3-5. 3. Stepwise output control to reduce fatigue Summary

<< 1. Overview of Information Processing Device According to One Embodiment of Present Disclosure >>
First, an overview of an information processing apparatus 1 according to an embodiment of the present disclosure will be described with reference to FIG. As shown in FIG. 1, the information processing apparatus 1 according to the present embodiment is a glasses-type HMD (Head Mounted Display), and is worn by a user by, for example, a pattern extending over both heads being put on both ear shells. .

In addition, the information processing apparatus 1 (glasses type HMD), in a wearing state, immediately before the user's eyes, that is, a pair of display units 2a for the left eye and right eye, in a place where a lens in normal glasses is located 2b is arranged. Hereinafter, when it is not necessary to distinguish between the

display parts

2a and 2b, they are collectively referred to as the display part 2.

The display unit 2 displays a navigation screen, a mail screen, a news screen, a calendar screen, a screen for giving a warning or notification to a user, and plays a video or a photograph. In addition, the display unit 2 may be a transmissive type, and the information processing device 1 causes the display unit 2 to be in a through state, that is, a transparent or translucent state, so that the user can make the information processing device 1 like glasses. Even if it is always worn, it will not interfere with normal life. The user browses emails and news on the display unit 2 located in the lens part of the glasses type at any time while having a normal life with the information processing device 1 (glasses type HMD) mounted on the head, Navigation can be displayed.

In addition, the information processing apparatus 1 is provided with an inner camera 4 that is arranged inward so as to image the user's eyes in a worn state.

Further, the information processing apparatus 1 is provided with an illuminance sensor 6 for detecting ambient brightness.

Also, the information processing apparatus 1 is provided with a pair of

earphone speakers

3a and 3b that can be inserted into the user's right ear hole and left ear hole when worn. In addition,

microphones

5 a and 5 b that collect external sound are disposed on the right side of the display unit 2 for the right eye and on the left side of the display unit 2 for the left eye.

Note that the appearance of the information processing apparatus 1 shown in FIG. 1 is an example, and various structures for the user to wear the information processing apparatus 1 can be considered. The information processing apparatus 1 may be formed of a mounting unit that is generally of a glasses type or a head mounted type, and it is sufficient that at least the movement of the user's head can be detected.

In addition to a pair of display units 2 corresponding to both eyes, one display unit 2 may be provided corresponding to one eye. Further, in the example shown in FIG. 1, the inner camera 4 is disposed inward on the right eye side, but may be disposed on the left eye side or may be disposed on both sides. Further, in the example shown in FIG. 1, the illuminance sensor 6 is disposed inward on the left eye side, but may be disposed on the right eye side or may be disposed on both sides.

Further, the

earphone speakers

3a and 3b may not be left and right stereo speakers, but only one

earphone speaker

3a and 3b may be provided to be worn only on one ear. Further, the microphone may be one of the

microphones

5a and 5b. Furthermore, a configuration without the inner camera 4, the

microphones

5a and 5b, or the

earphone speakers

3a and 3b is also conceivable.

(background)
Here, in the conventional technique, as described above, a place and time for detecting fatigue are required, and a dedicated device for measuring the pupil diameter is required. In addition, conventional fatigue detection technology and stress care technology are not used unless the user is aware of fatigue by himself / herself.

Therefore, in this embodiment, the information processing apparatus 1 that can automatically detect eye fatigue (eye fatigue / eye strain) and perform output control to reduce fatigue without the user being aware of it. I will provide a. Specifically, by using a transmissive head wearable device (information processing apparatus 1) that a user wears on the head on a daily basis, it is possible to detect eye fatigue without worrying about location or time. Become. Further, since the information processing apparatus 1 actively detects eye fatigue for the user, even if the user is not aware of fatigue, the user is made aware of fatigue or reduces (cancels) fatigue. Output control can be performed.

The overview of the information processing apparatus 1 according to an embodiment of the present disclosure has been described above. Subsequently, a basic configuration of the information processing apparatus 1 according to the present disclosure will be described.

<< 2. Basic configuration >>
FIG. 2 is a block diagram illustrating a basic configuration of the information processing apparatus 1 according to the present embodiment. As illustrated in FIG. 2, the information processing apparatus 1 includes a main control unit 10, a display unit 2, an audio output unit 3, an inner camera 4, an audio input unit 5, an illuminance sensor 6, a communication unit 12, and a storage unit 14. .

(Inside camera)
The inner camera 4 includes a lens system including an imaging lens, an aperture, a zoom lens, a focus lens, and the like, a drive system that causes the lens system to perform a focusing operation and a zoom operation, and imaging light obtained by the lens system. An imaging unit having a solid-state imaging element array or the like that generates an imaging signal through photoelectric conversion. The solid-state imaging device array may be realized by, for example, a CCD (Charge Coupled Device) sensor array or a CMOS (Complementary Metal Oxide Semiconductor) sensor array.

As shown in FIG. 1, the inner camera 4 is arranged facing inward with the information processing apparatus 1 attached to the user. For this reason, the inner camera 4 can always image the user's eyes while the information processing apparatus 1 is worn by the user. A captured image of the user's eye captured by the inner camera 4 is output to the main control unit 10.

(Illuminance sensor)
The illuminance sensor 6 has a function of detecting ambient brightness, and outputs a detection result to the main control unit 10. A specific configuration of the illuminance sensor 6 is not particularly limited. In the present embodiment, the pupil diameter of the user's eye is measured by the measurement unit 110 described later, and fatigue is detected based on the pupil diameter by the detection unit 120. However, the pupil can contract due to the influence of external light. For example, the pupil usually shrinks when it is bright and expands in the dark. Therefore, in this embodiment, the illuminance sensor 6 is provided in order to distinguish between shrinkage due to the influence of external light and shrinkage due to fatigue.

(Main control unit)
The main control unit 10 includes, for example, a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory, and an interface unit. To control.

Further, the main control unit 10 according to the present embodiment functions as a measurement unit 110, a detection unit 120, and an output control unit 130 as shown in FIG. The measurement unit 110 measures the pupil diameter based on the captured image captured by the inner camera 4.

The detection unit 120 detects eye fatigue based on the pupil diameter measured by the measurement unit 110. Usually, since the human pupil area is reduced (miosis) during fatigue, the detection unit 120 detects a miosis that is considered fatigue based on whether the measured pupil diameter is less than a predetermined value. Can do. Here, the predetermined value at the time of determining fatigue may be set to a general value (specifically, for example, 2 mm) based on medical knowledge, or set to a value that is personalized by performing calibration in advance. May be.

Also, the detection unit 120 according to the present embodiment can detect eye fatigue based on the change of the pupil diameter over time. Specifically, the detection unit 120 detects eye fatigue by comparing with data on the change speed of the pupil diameter when the external light collected in advance is normal (before fatigue). Usually, the area of the pupil of the human eye changes according to the brightness of external light (referred to as “extrapupillary light reaction”), and the external pupillary light response is lower than normal when fatigued. Therefore, the detection unit 120 can detect eye fatigue when the extra-pupil light reaction is lower than normal. For example, the maximum pupil contraction rate, contraction speed, and re-expansion speed are used as the measurement index of the extra-pupil light reaction.

Also, the detection unit 120 can detect the progress of fatigue based on the change in pupil diameter that is considered fatigue and the change rate.

The output control unit 130 performs predetermined output control so as to reduce (eliminate) eye fatigue detected by the detection unit 120. Further, the output control unit 130 may perform predetermined output control for gradually reducing eye fatigue according to the course of eye fatigue detected by the detection unit 120. Specific examples of output control for reducing (eliminating) eye fatigue will be described in detail with reference to FIGS. 3 to 9. For example, the size, thickness, and screen of characters displayed on the display unit 2 are described. For example, controlling the contrast. Moreover, as a specific example of output control, notifying the user that the user is tired or prompting the user to take a break can be cited.

(Communication Department)
The communication unit 12 has a function of transmitting / receiving data by connecting to an external device by wireless / wired. The communication unit 12 according to the present embodiment is connected to a server on the network, for example, and transmits and receives electronic mail. Moreover, the communication part 12 may connect with the smart phone, tablet terminal, etc. which a user possesses, and may receive an image | video, a photograph, music, etc.

(Voice input part)
The audio input unit 5 includes the

microphones

5a and 5b shown in FIG. 1, and a microphone amplifier unit and an A / D converter for amplifying the audio signals obtained by the

microphones

5a and 5b. To the unit 10. The main control unit 10 enables voice input by the user by performing voice recognition on the voice data obtained by the voice input unit 5 after performing processing such as noise removal and sound source separation.

(Display section)
The display unit 2 displays image data according to control by the main control unit 10. Further, as described above, the display unit 2 may be a transmissive display device.

(Audio output part)
The audio output unit 3 includes a pair of

earphone speakers

3a and 3b shown in FIG. 1 and an amplifier circuit for the

earphone speakers

3a and 3b. Moreover, the audio | voice output part 3 may be comprised as what is called a bone conduction speaker. The audio output unit 3 outputs (reproduces) audio signal data according to control by the main control unit 10.

(Memory part)
The storage unit 14 stores programs for the main control unit 10 to execute various processes.

The configuration of the information processing apparatus 1 according to the present embodiment has been described in detail above. With the configuration described above, the information processing apparatus 1 is realized by a wearable device that is worn on the head on a daily basis, so that changes in the pupil diameter can be continuously observed regardless of the user's consciousness. When this is detected, output control for automatically reducing fatigue can be performed.

Subsequently, the operation processing of the information processing apparatus 1 according to the present embodiment will be specifically described with reference to FIGS.

<< 3. Action processing >>
<3-1. Fatigue detection based on pupil diameter>
FIG. 3 is a flowchart showing an operation process of fatigue detection based on the pupil diameter according to the present embodiment. As shown in FIG. 3, first, in step S <b> 103, the main control unit 10 of the information processing apparatus 1 activates the inner camera 4. The activation of the inner camera 4 is performed as one of initial operations when the user wears the information processing apparatus 1 that is a glasses-type HMD (an example of a head wearable device) on the head and turns on the power. Is assumed.

Next, in step S106, the measurement unit 110 performs normal pupil diameter measurement and performs an operation of registering the normal pupil diameter of the user. Specifically, the measurement unit 110 registers the pupil diameter at the normal time in a plurality of time zones and environments having different brightness as an initial operation, and performs calibration. By performing such an operation, it is possible to perform a robust fatigue detection process that is more personalized.

Next, in step S109, the illuminance sensor 6 detects the brightness of external light.

Subsequently, in step S <b> 112, the measurement unit 110 measures the pupil diameter based on the captured image of the user's eye captured by the inner camera 4.

Next, in step S115, based on the measured pupil diameter, the detection unit 120 determines whether or not the miosis is considered to be fatigue, and performs fatigue detection. Specifically, the detection unit 120 considers the brightness of the external light detected by the illuminance sensor 6 and has a miosis that falls below a predetermined value despite no change in the brightness of the external light. It is determined that the miosis is regarded as fatigue, and fatigue is detected.

The processes of S109 to S115 are repeated until it is determined that the miosis is regarded as fatigue (S115 / No). Note that it is desirable that such miosis determination is checked by always taking an image with the inner camera 4, but from the viewpoint of power consumption, it should be performed periodically / irregularly or at a predetermined set time. It may be. Alternatively, the frequency of determination of miosis may be increased in a time zone in which desk work is performed for a long time. Whether or not the user is performing desk work may be determined by combining, for example, the user's schedule information and sensor values provided by an acceleration sensor, a gyro sensor, or the like provided in the information processing apparatus 1.

Next, when it is determined that the miosis is regarded as fatigue and fatigue is detected (S115 / Yes), in step S118, the output control unit 130 performs predetermined output control to reduce fatigue. Here, a specific example of the predetermined output control for reducing fatigue will be described with reference to FIGS.

FIG. 4 is a diagram for explaining display output control for reducing fatigue. FIG. 4 shows an example of an e-mail display screen 20 displayed on the display unit 2 when the eye state is normal. Here, when eye fatigue is detected by the detection unit 120, the output control unit 130 increases the size of the character, makes it thicker, or increases the contrast, as shown in the display screen 21 in FIG. Or reducing the brightness, the fatigue of the user's eyes is reduced. This is because, for example, reading a large character is less burdensome on the eyes than reading a small character.

In addition, when the output control unit 130 performs output control for reducing fatigue in this way, an icon for notifying the user that the mode has shifted to the fatigue mode may be displayed. Specifically, for example, by displaying an icon image 30 shown on the display screen 21 in FIG. 4, the user is notified of the transition to the fatigue mode. Thereby, the user can be aware of fatigue of his / her eyes and take active measures such as taking a rest.

Further, the output control unit 130 may perform control to forcibly stop the display output because of a serious matter related to health when the degree of fatigue is larger than a specified value.

Further, the output control unit 130 may only issue a warning when the degree of fatigue is smaller than a specified value. Here, an example of the fatigue warning is shown in FIG. As shown in FIG. 5, the output control unit 130 displays an icon image 32 that warns about eye fatigue above the e-mail display screen 22, thereby prompting the user to rest.

In addition to the display output control described above, the output control unit 130 may perform output control of, for example, a fatigue healing application function. As the fatigue healing application function, for example, a massage around the eyes by a vibration unit (not shown) built in the information processing apparatus 1 or a heating / cooling element (not shown) such as a Peltier element built in the information processing apparatus 1 is used. A hot and cold massage of the eyes can be mentioned.

The specific examples of various output controls for reducing fatigue by the output control unit 130 have been described above. It should be noted that the above-described output control for reducing fatigue may be preset by the user when detecting fatigue.

Subsequently, in step S121 of FIG. 3, the main control unit 10 determines whether or not an instruction to cancel output control for reducing fatigue has been input. In the present embodiment, when eye fatigue is detected regardless of the user's consciousness, output control for automatically reducing fatigue is performed, so it may be assumed that some users want to return to a normal state. Therefore, the convenience of the information processing apparatus 1 can be further improved by making it possible to cancel the output control for reducing fatigue. There are various ways of inputting the release instruction. For example, the setting menu may be called to change the output control mode from the fatigue mode to the normal mode. Further, the information processing apparatus 1 may be provided with a physical button, and the fatigue mode / normal mode may be switched when the button is pressed for a long time.

Next, when a release instruction is received (S121 / Yes), in step S124, the output control unit 130 returns to normal output control.

The fatigue detection operation process based on the pupil diameter according to the present embodiment has been specifically described above.

<3-2. Fatigue detection based on miosis / mydriatic velocity>
In the operation processing described above, fatigue is detected based on whether or not the pupil diameter is less than a predetermined value. However, this embodiment is not limited to this. For example, fatigue is detected based on the miosis / mydriatic velocity. May be. Hereinafter, a specific description will be given with reference to FIG.

FIG. 6 is a flowchart showing an operation process when fatigue detection is performed based on the miosis speed. In addition, in each step shown in FIG. 6, when the same process as the step shown in FIG. 3 is performed, the same step number is attached | subjected.

As shown in FIG. 6, first, in step S <b> 103, the main control unit 10 of the information processing apparatus 1 activates the inner camera 4.

Next, in step S104, the measurement unit 110 continuously measures the pupil diameter at the normal time and registers the pupil miosis / mydriatic velocity at the normal time of the user. Specifically, as an initial operation, the measurement unit 110 registers the miosis / mydriatic velocity of the pupil according to changes in external light during normal times in a plurality of time zones and environments with different brightnesses, and performs calibration. .

Next, in step S110, the main control unit 10 determines whether the brightness of the external light has changed based on the detection result by the illuminance sensor 6. Further, S109 and S110 are repeatedly performed.

Subsequently, when the brightness of the external light changes (S110 / Yes), in step S113, the measurement unit 110 calculates the pupil diameter based on the captured images of the user's eyes continuously captured by the inner camera 4. Measure continuously. Note that the measurement of the pupil diameter by the measurement unit 110 is not limited to the measurement after the change of the external light, and the pupil diameter before and after the change of the external light may be continuously measured.

Next, in step S116, the detection unit 120 determines whether the miosis / mydriatic velocity is a miosis / mydriatic velocity that is considered fatigue based on the continuously measured pupil diameter. And fatigue detection. Specifically, the detection unit 120 is a miosis / mydriasis that is regarded as eye fatigue when the speed of miosis / mydriasis corresponding to the brightness of external light is lower than normal. And fatigue is detected.

Next, in step S118, as in the same step shown in FIG. 3, the output control unit 130 performs predetermined output control for reducing fatigue, and a release instruction is input by the user in steps S121 to S124. If so, return to normal output control.

<3-3. Fatigue detection based on miosis / mydriatic velocity in response to changes in brightness>
In the operation processing described above with reference to FIG. 6, eye fatigue is detected based on the speed of miosis / mydriasis according to changes in external light, but this embodiment is not limited to this, For example, eye fatigue may be detected based on the miosis / mydriatic velocity corresponding to the change in the luminance of the display unit 2. Hereinafter, a specific description will be given with reference to FIG.

FIG. 7 is a flowchart showing an operation process of fatigue detection based on the miosis / mydriatic velocity according to the change in luminance according to the present embodiment.

As shown in FIG. 7, first, in step S103, the main control unit 10 of the information processing apparatus 1 activates the inner camera 4.

Next, in step S104 ', the measurement unit 110 continuously measures the pupil diameter at the normal time and registers the pupil miosis / mydriatic velocity at the normal time of the user. Specifically, the measurement unit 110 registers the miosis / mydriatic velocity of the pupil according to a change in the luminance of the display unit 2 in a normal state in a plurality of time zones and environments with different brightness as an initial operation, and performs calibration. Perform

Next, in step S107, the main control unit 10 checks the luminance of the display unit 2.

Next, in step S108, the main control unit 10 determines whether or not the luminance of the display unit 2 has changed. In the present embodiment, the change in luminance of the display unit 2 may be a change that inevitably occurs in accordance with a change in display content or screen transition. In this case, fatigue can be detected without making the user aware of it. However, there is a concern that fatigue detection opportunities are affected by user behavior. Therefore, the main control unit 10 may detect the fatigue by changing the luminance of the display unit 2 when the luminance change due to switching of the display content does not occur for a certain period of time. Also, S107 and S108 are repeated.

Subsequently, when the luminance changes (S108 / Yes), in step S113, the measurement unit 110 continuously measures the pupil diameter based on the captured image of the user's eye continuously captured by the inner camera 4. To do.

Next, in step S116 ′, the detection unit 120 determines whether or not the speed of miosis / mydriasis is the speed of miosis / mydriasis that is considered fatigue based on the continuously measured pupil diameter. Judgment and fatigue detection. Specifically, the detection unit 120 determines that the miosis / mydriasis is considered to be eye fatigue when the speed of miosis / mydriasis corresponding to a change in luminance is lower than normal. And detect fatigue.

Next, in step S118, as in the same step shown in FIG. 3, the output control unit 130 performs predetermined output control for reducing fatigue, and in steps S121 to S124, a release instruction is input by the user. If it does, return to normal output control.

As a result, even when the display unit 2 is controlled to be in a non-transparent state and a change in external light does not affect the user's eyes, the miosis / mydriasis of the user's eyes according to the change in the luminance of the display unit 2 Eye fatigue can be detected based on speed. Moreover, since the change in the brightness of the display unit 2 is a change that inevitably occurs in accordance with a change in display content or screen transition, fatigue detection can be performed without making the user aware of it.

<3-4. Improved accuracy in fatigue detection>
In each operation process described above, the normal pupil diameter and pupil pupil miosis / mydriatic speed are measured and calibrated when the inner camera 4 is activated, but by performing more calibration, The accuracy of fatigue detection according to the present embodiment can be improved. Hereinafter, a specific description will be given with reference to FIG.

FIG. 8 is a flowchart showing an operation process for improving the accuracy of fatigue detection according to the present embodiment. Here, as an example, a case where the accuracy of the fatigue detection process based on the pupil diameter described with reference to FIG. 3 is improved will be described.

As shown in FIG. 8, first, when the main control unit 10 activates the inner camera 4 in step S103, next, in step S106, the measurement unit 110 measures the pupil diameter at the normal time to start up. Perform calibration.

Next, in steps S109 to S115, as in the same step shown in FIG. 3, the detection unit 120 detects fatigue according to the pupil diameter based on the captured image of the user's eye captured by the inner camera 4. In the present embodiment, when such fatigue detection processing is repeatedly performed, calibration is also repeatedly performed.

Specifically, when it is determined in step S115 that the miosis is not considered to be fatigue (S115 / No), in step S117, the measurement unit 110 stores the measured pupil diameter as a normal pupil diameter. Store in the unit 14. In this way, not only at startup, but also when repeatedly detecting eye fatigue, normal memory in various environments can be obtained by continuously storing the normal pupil diameter and increasing the number of calibrations. Values can be collected and the accuracy of fatigue detection can be improved.

If it is determined that the miosis is regarded as fatigue (S115 / Yes), the same processing as in FIG. 3 is performed in steps S118 to S124. Specifically, in step S118, the output control unit 130 performs predetermined output control for reducing fatigue, and returns to normal output control when a release instruction is input by the user in steps S121 to S124. .

<3-5. Stepwise output control to reduce fatigue>
In each of the operation processes described above, when the detection unit 120 detects fatigue, predetermined output control is performed to reduce fatigue. However, the output control unit 130 according to the present embodiment is adapted to the progress of fatigue. It may be done step by step. Hereinafter, a specific description will be given with reference to FIG.

FIG. 9 is a flowchart showing an operation process of stepwise output control for reducing fatigue according to the present embodiment. Here, as an example, a case will be described in which output control in the fatigue detection process based on the pupil miosis / mydriatic velocity described with reference to FIG. 6 is performed in stages.

As shown in FIG. 9, in steps S103 to S116, fatigue detection based on the miosis / mydriatic velocity of the pupil diameter is performed as in the same step shown in FIG.

Next, when the miosis / mydriatic velocity regarded as fatigue is measured (S116 / Yes), in step S119, the output control unit 130 performs predetermined output control for reducing fatigue according to the progress of fatigue. Step by step.

Specifically, for example, the output control unit 130 performs fatigue warning control when the miosis / mydriatic velocity regarded as fatigue is within the predetermined value range of the first stage, and displays when the range is the predetermined value range of the second stage. Display control is performed so as not to put a burden on the eyes by enlarging the characters displayed on the unit 2 or increasing the contrast. Further, the output control unit 130 performs control to forcibly stop the display output of the display unit 2 when the miosis / mydriatic velocity regarded as fatigue is within a predetermined range of the third stage. Thus, in this embodiment, output control for reducing fatigue can be performed in a plurality of stages according to the progress of eye fatigue.

Subsequently, in steps S121 to S124, as in the same step shown in FIG. 6, the output control unit 130 returns to normal output control when a release instruction is input by the user.

<< 4. Summary >>
As described above, the information processing apparatus 1 according to the embodiment of the present disclosure is realized by a transmissive head wearable device that a user wears on the head on a daily basis, and automatically causes eye fatigue without the user being conscious. It is possible to detect (eye fatigue / eye strain) and perform output control for reducing fatigue.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present technology is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

For example, it is possible to create a computer program for causing hardware such as a CPU, ROM, and RAM incorporated in the information processing apparatus 1 described above to exhibit the functions of the information processing apparatus 1. A computer-readable storage medium storing the computer program is also provided.

In addition, the information processing apparatus 1 according to the above-described embodiment automatically detects eye fatigue (eye fatigue / eye strain) even if the user is not conscious, but there is an active operation by the user. Eye fatigue may be detected in response to a user operation.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

In addition, this technique can also take the following structures.
(1)
An imaging unit that is provided in an information processing device mounted on a user's head and that images the user's eyes;
A measurement unit that measures a pupil diameter based on a captured image captured by the imaging unit;
A detection unit for detecting eye fatigue based on the pupil diameter measured by the measurement unit;
A control unit that performs predetermined output control for reducing eye fatigue detected by the detection unit;
An information processing apparatus comprising:
(2)
The information processing apparatus according to (1), wherein the detection unit detects eye fatigue based on a change in the pupil diameter.
(3)
The detection unit detects eye fatigue by comparing the change of the pupil diameter with data of the change speed of the pupil diameter when the external light collected in advance is normal. (2) The information processing apparatus described in 1.
(4)
The information processing apparatus according to any one of (1) to (3), wherein the measurement unit measures a change speed of the pupil diameter when an external light condition changes.
(5)
The information processing apparatus according to any one of (1) to (3), wherein the measurement unit measures a change speed of the pupil diameter when the luminance of the display unit changes.
(6)
The information processing apparatus according to (4) or (5), wherein the detection unit detects an eye fatigue process according to a change rate of the pupil diameter.
(7)
The information processing apparatus according to (6), wherein the control unit performs predetermined output control for reducing the eye fatigue step by step according to the progress of the eye fatigue.
(8)
When the eye fatigue is detected, the control unit performs at least one of control of making the currently displayed character larger, thicker, higher in contrast, or lower in luminance. The information processing apparatus according to any one of (1) to (7).
(9)
The information processing apparatus according to any one of (1) to (7), wherein the control unit performs control so as to stop display output when eye fatigue is detected.
(10)
The information processing apparatus according to any one of (1) to (7), wherein when the eye fatigue is detected, the control unit controls to drive an eye strain treatment application.
(11)
The information processing apparatus according to any one of (1) to (8), wherein the information processing apparatus is a glasses-type wearable device.
(12)
Provided in an information processing device mounted on a user's head, imaging the user's eyes;
Measuring the pupil diameter based on the captured image,
Detecting eye fatigue based on the measured pupil diameter;
Performing predetermined output control for reducing detected eye fatigue;
Including a control method.
(13)
Computer
An imaging unit that is provided in an information processing device mounted on a user's head and that images the user's eyes;
A measurement unit that measures a pupil diameter based on a captured image captured by the imaging unit;
A detection unit for detecting eye fatigue based on the pupil diameter measured by the measurement unit;
A control unit that performs predetermined output control for reducing eye fatigue detected by the detection unit;
Program to function as

DESCRIPTION OF SYMBOLS 1,

Information processing apparatus

2, 2a, 2b Display part 3 Audio |

voice output part

3a, 3b Earphone speaker 4 Inner camera 5 Audio |

voice input part

5a, 5b Microphone 6 Illuminance sensor 10 Main control part 110 Measurement part 120 Detection part 130 Output control part 12 Communication unit 14 Storage unit

Claims

An imaging unit that is provided in an information processing device mounted on a user's head and that images the user's eyes;
A measurement unit that measures a pupil diameter based on a captured image captured by the imaging unit;
A detection unit for detecting eye fatigue based on the pupil diameter measured by the measurement unit;
A control unit that performs predetermined output control for reducing eye fatigue detected by the detection unit;
An information processing apparatus comprising:
The information processing apparatus according to claim 1, wherein the detection unit detects eye fatigue based on a change in the pupil diameter.
The detection unit detects eye fatigue by comparing the change of the pupil diameter with data of the change speed of the pupil diameter when the external light collected in advance is normal. The information processing apparatus described.
The information processing apparatus according to claim 1, wherein the measurement unit measures a change speed of the pupil diameter when an external light condition is changed.
The information processing apparatus according to claim 1, wherein the measurement unit measures a change speed of the pupil diameter when the luminance of the display unit is changed.
The information processing apparatus according to claim 4, wherein the detection unit detects an eye fatigue process according to a change rate of the pupil diameter.
The information processing apparatus according to claim 6, wherein the control unit performs predetermined output control for reducing the eye fatigue step by step according to the progress of the eye fatigue.
2. The control unit performs at least one of control of making a currently displayed character larger, thicker, higher in contrast, or lower in luminance when the eye fatigue is detected. The information processing apparatus described in 1.
The information processing apparatus according to claim 1, wherein the control unit controls to stop the display output when the eye fatigue is detected.
The information processing apparatus according to claim 1, wherein the control unit controls to drive an eye strain treatment application when the eye fatigue is detected.
The information processing apparatus according to claim 1, wherein the information processing apparatus is a glasses-type wearable device.
Provided in an information processing device mounted on a user's head, imaging the user's eyes;
Measuring the pupil diameter based on the captured image,
Detecting eye fatigue based on the measured pupil diameter;
Performing predetermined output control for reducing detected eye fatigue;
Including a control method.
Computer
An imaging unit that is provided in an information processing device mounted on a user's head and that images the user's eyes;
A measurement unit that measures a pupil diameter based on a captured image captured by the imaging unit;
A detection unit for detecting eye fatigue based on the pupil diameter measured by the measurement unit;
A control unit that performs predetermined output control for reducing eye fatigue detected by the detection unit;
Program to function as