JP2022083328A - Body condition estimation device, body condition estimation method, program, and recording medium - Google Patents

Abstract

To provide a system capable of easily estimating a body condition such as a physical function and a disease.SOLUTION: A body condition estimation device includes a storage part 10, a display part 11, an eyeball image acquisition part 12, an eyeball movement data acquisition part 13, and a body condition estimation part 14. The storage part includes at least one piece of examination pattern data for visual stimulation, and the examination pattern for visual stimulation acquires two or more items of eyeball movement data. The display part displays the examination pattern for visual stimulation based on pattern data. The eyeball image acquisition part continuously images the eyeball of a subject looking at the examination pattern for visual stimulation displayed in the display part, and acquires an eyeball image. The eyeball movement data acquisition part analyzes the eyeball image and acquires combined eyeball movement data on two or more items of eyeball movements. The estimation part estimates a body condition of the subject by body condition estimation information on the combined eyeball movement data on the subject. The body condition estimation information indicates relationships between the examination pattern data for visual stimulation and the combined eyeball movement data, and the body condition.SELECTED DRAWING: Figure 1

Description

The present invention relates to a physical condition estimation device, a physical condition estimation method, a program, and a recording medium.

In recent years, driving accidents of the elderly have become a social problem, and the return of licenses according to driving ability is regarded as important. However, drivers who are not aware of their physical or cognitive decline are often reluctant to return their licenses. Further, although devices such as drive simulations have been commercialized, it is difficult to objectively judge the influence of dementia, for example, even if the driving technique can be confirmed. For this reason, it is necessary to objectively understand the necessity of license return for drivers who are not aware of the deterioration of cognitive function, etc., or drivers who are worried but do not feel the need to return the license. Not reached. In addition, it is not realistic for non-medical workers to use advanced measuring devices such as brain waves at a driver's license examination site, etc., which affects driving ability, such as deterioration of physical function and diseases such as dementia. It is also difficult to judge the possibility.

Therefore, it is considered very useful to construct a system that can objectively and easily estimate the driving ability affected by the deterioration of physical function and cognitive function even if the person is not a medical professional. In addition, regardless of driving ability, it is also useful for daily health management at home if various diseases such as dementia and brain disorders can be easily guessed.

Therefore, an object of the present invention is to provide a new system that can easily estimate a physical condition such as a physical function or a disease.

In order to achieve the above object, the physical condition estimation device of the present invention includes a storage unit, a display unit, an eyeball image acquisition unit that captures an image of the subject's eyeball, an eye movement data acquisition unit, and a subject's eye movement data acquisition unit. Including the estimation part of the physical condition
The storage unit has at least one visual stimulation test pattern data, and the visual stimulation test pattern is a pattern for acquiring two or more eye movement data of a subject.
The display unit displays a visual stimulus test pattern based on the visual stimulus test pattern data, and displays the visual stimulus test pattern.
The eyeball image acquisition unit continuously photographs the eyeball of the subject who is viewing the visual stimulation test pattern displayed on the display unit, and acquires an eyeball image.
The eye movement data acquisition unit analyzes the eye image and acquires combined eye movement data of two or more items of eye movement.
The estimation unit estimates the physical condition of the subject from the acquired combined eye movement data of the subject based on the physical condition estimation information.
The physical state estimation information is characterized by being information showing the relationship between the eye movement data and the physical state in combination with the visual stimulation test pattern data.

The method for estimating the physical condition of the present invention uses a storage unit, a display step, an eyeball image acquisition step for taking an image of the subject's eyeball, an eye movement data acquisition step, and an estimation of the physical condition of the subject. Including the process
The storage unit has at least one visual stimulation test pattern data, and the visual stimulation test pattern is a pattern for acquiring two or more eye movement data of a subject.
In the display step, a visual stimulus test pattern based on the visual stimulus test pattern data is displayed on the display unit.
In the eyeball image acquisition step, the eyeball of the subject who is looking at the visual stimulation test pattern displayed on the display unit is continuously photographed, and the eyeball image is acquired.
In the eye movement data acquisition step, the eye movement image is analyzed to acquire a combination eye movement data of two or more items of eye movement, and the eye movement data is acquired.
In the estimation step, the physical condition of the subject is estimated from the acquired combined eye movement data of the subject based on the physical condition estimation information.
The physical state estimation information is characterized by being information showing the relationship between the eye movement data and the physical state in combination with the visual stimulation test pattern data.

The program of the present invention is characterized in that a computer executes the method of estimating the physical condition of the present invention.

The recording medium of the present invention is computer readable in which the program of the present invention is recorded.

According to the present invention, a test pattern for visual stimulation is shown to a subject without using an advanced device such as a brain wave measuring device, and eyeball images of the subject are continuously photographed. Combination data of eye movements of a plurality of items can be acquired from the eyeball image, and the physical condition of the subject can be estimated from the combination data based on the physical condition estimation information. Therefore, for example, the physical condition of the subject can be easily estimated regardless of the medical staff. According to the present invention, for example, the presence or absence or degree of driving ability, the possibility of diseases such as brain disorders and dementia, and the like can be easily estimated, which can be useful for returning a driver's license and improving life.

FIG. 1 is a block diagram showing an example of the physical condition estimation device of the first embodiment. FIG. 2 is a block diagram showing another example of the hardware configuration of the physical condition estimation device of the first embodiment. FIG. 3 is a schematic diagram showing an example of linking information in the first embodiment. FIG. 4 is a block diagram showing an example of the physical condition estimation system of the second embodiment. FIG. 5 is a block diagram showing an example of the physical condition estimation system of the second embodiment.

In the body state estimation device of the present invention, for example, the estimation unit includes a body state estimation model, and the body state estimation model is a body state for estimation purposes obtained by displaying the visual stimulation test pattern. It is a model generated by learning from the eye movement data of the learning sample group and the eye movement data of the learning sample group that is not the physical state for estimation, and the physical state of the subject is obtained by the physical state estimation model. To estimate.

In the body state estimation device of the present invention, for example, the estimation unit includes a body state estimation model, and the body state estimation model is obtained by displaying the visual stimulation test pattern, and is a normal body state for estimation purposes. It is a model generated by learning from the eye movement data of the learning sample group, and the physical condition of the subject is estimated by the physical condition estimation model.

In the physical state estimation device of the present invention, for example, the items of the eye movement include pupil position, pupil diameter, pupil dilation / reduction speed, convergence and divergence movement, follow-up, sacked, fixation, and laterality of eye movement. Includes at least one selected from the group consisting of, eye movements, and irises.

In the physical state estimation device of the present invention, for example, the physical state to be estimated is the presence or absence or degree of driving ability, brain function, neural function, or cognitive function.

In the physical condition estimation device of the present invention, for example, the physical condition to be estimated is a disease.

In the physical condition estimation device of the present invention, for example, the disease is a brain disorder, dementia, or a psychiatric disorder.

The physical state estimation device of the present invention further includes, for example, a light / dark adjustment unit that adjusts the lightness / darkness of the space between the visual field of the subject and the display unit.

The physical state estimation device of the present invention has, for example, a housing, and the display unit, the eyeball image acquisition unit, and the light / dark adjustment unit are arranged inside the housing, and the light / dark adjustment unit is arranged on one surface of the housing. The housing has an observation window for viewing the display surface of the display unit inside the housing.

The physical condition estimation device of the present invention further includes, for example, a fixed portion of the head for keeping the distance between the subject's eyeball and the display portion constant.

The physical condition estimation device of the present invention includes, for example, a terminal having a display unit, a fixed unit, a light condition measurement unit, a correction unit, a storage unit, and an eyeball image acquisition unit that captures an image of a subject's eyeball. The terminal and the device are connectable, including a device including an eye movement data acquisition section and a device including an estimation section of the physical condition of the subject, and in the device, the fixing section refers to the terminal. Fixed, the optical condition measuring unit measures the optical condition of the display unit of the terminal, and the correction unit calculates the correction condition for setting the measured optical condition of the terminal to an arbitrary optical condition, and the above-mentioned. The terminal adjusts the light condition based on the calculated correction condition.

In the method for estimating the physical condition of the present invention, for example, the estimation step uses a physical condition estimation model, and the physical condition estimation model is a physical condition for estimation purposes obtained by displaying the visual stimulation test pattern. It is a model generated by learning from the combination eye movement data of a certain learning sample group and the combination eye movement data of a learning sample group that is not the physical state for estimation, and the subject is said to be based on the physical state estimation model. Estimate the physical condition of.

In the method for estimating the physical condition of the present invention, for example, the estimation step uses a physical condition estimation model, and the physical condition estimation model is obtained by displaying the visual stimulation test pattern, and is a normal body for estimation purposes. It is a model generated by learning from the eye movement data of the learning sample group which is a state, and the physical state of the subject is estimated by the physical state estimation model.

In the method for estimating the physical condition of the present invention, for example, the items of the eye movement are the pupil position, the pupil diameter, the dilation / reduction speed of the pupil, the convergence / divergence movement, the follow-up, the sacked, the fixation, and the laterality of the eye movement. Includes at least one selected from the group consisting of, eye movements, and irises.

In the method for estimating the physical condition of the present invention, for example, the physical condition to be estimated is the presence or absence or degree of driving ability, brain function, neural function, or cognitive function.

In the method for estimating the physical condition of the present invention, for example, the physical condition to be estimated is a disease.

In the method for estimating the physical condition of the present invention, for example, the disease is a brain disorder, dementia, or a psychiatric disorder.

In the method for estimating the physical condition of the present invention, for example, the display and imaging on the display unit are performed while adjusting the brightness of the space between the field of view of the subject and the display unit.

In the present invention, the "physical state" to be estimated is not particularly limited, and any item can be set. Specific examples include, for example, functions and abilities, diseases and the like. The functions and abilities are, for example, the presence or absence or degree of driving ability, brain function, neural function, cognitive function and the like. The diseases include, for example, brain disorders, dementia, mental illnesses, diabetes and the like. The brain disorders include, for example, cerebrovascular disorders such as cerebral infarction and cerebral hemorrhage, epilepsy, etc., dementia, for example, Alzheimer, etc. Dementia etc. can be mentioned. Other disorders include, for example, inner ear disorders, vestibular disorders (eg Meniere's disease, sudden deafness, vestibular neuritis, etc.), drinking or drug addiction, syphilis, schizophrenia, cerebral disorders (spinal cerebral degeneration). ) Etc. can also be given. It should be noted that the present invention is not intended to obtain the eye movement items themselves such as the change in pupil diameter and the follow-up function as test results, but uses a combination of eye movement data of these plurality of items. , Estimates the desired physical condition. Further, in the present invention, the "physical condition" to be estimated excludes, for example, the visual function directly determined from the above items.

An embodiment of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiments. In each of the following figures, the same parts are designated by the same reference numerals. Unless otherwise specified, the explanations of the respective embodiments can be referred to each other's explanations. The configurations of each embodiment can be combined unless otherwise specified.

[Embodiment 1]
An example of the physical condition estimation device and the physical condition estimation method of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of an example of the physical condition estimation device 1 of the present embodiment. The physical state estimation device 1 includes a storage unit 10, a display unit 11, an eyeball image acquisition unit 12, an eye movement data acquisition unit 13, and a physical condition estimation unit 14. The physical condition estimation device 1 may further include, for example, an output unit 15. The physical condition estimation device 1 is also referred to as, for example, a physical condition estimation system.

Next, FIG. 2 illustrates a block diagram of the hardware configuration of the physical condition estimation device 1. The physical state estimation device 1 includes, for example, a CPU (central processing unit) 101, a memory 102, a bus 103, an input device 104, a display unit 11, a photographing unit 105, a communication device 106, a storage device 107, and the like. Each part of the physical condition estimation device 1 is connected to each other by an interface (I / F) via a bus 103.

The CPU 101 is a processor that controls the entire body state estimation device 1, and is not limited to the CPU, and may be another processor. In the physical state estimation device 1, for example, the program of the present invention and other programs are executed by the CPU 101, and various information is read and written.

The physical condition estimation device 1 can be connected to a communication network by, for example, a communication device 106 connected to the bus 103, and can also be connected to an external device via the communication network. The external device is not particularly limited, and examples thereof include a terminal, a PC (personal computer), and a server. Examples of the terminal include smartphones, mobile phones, tablets and the like. The connection method between the physical condition estimation device 1 and the external device is not particularly limited, and may be, for example, a wired connection or a wireless connection. The wired connection may be, for example, a cord connection or a cable connection for using a communication network. The wireless connection may be, for example, a connection using a communication network or a connection using wireless communication. The communication line network is not particularly limited, and for example, a known communication line network can be used, and examples thereof include an Internet line, a telephone line, a LAN (Local Area Network), WiFi (Wireless Fidelity), and the like.

The memory 102 includes, for example, a main memory, and the main memory is also referred to as a main storage device. When the CPU 101 performs processing, for example, the memory 102 reads various operation programs 110 such as the program of the present invention stored in the auxiliary storage device described later, and the CPU 101 receives data from the memory 102. , Program 110 is executed. The main memory is, for example, a RAM (random access memory). The memory 102 further includes, for example, a ROM (read-only memory).

The storage device 107 is also referred to as a so-called auxiliary storage device with respect to the main memory (main storage device), for example. The storage device 107 includes, for example, a storage medium and a drive for reading and writing to the storage medium. The storage medium is not particularly limited, and may be an internal type or an external type, and examples thereof include HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, and memory card. The drive is not particularly limited. As the storage device 107, for example, a hard disk drive (HDD) in which a storage medium and a drive are integrated can be exemplified. For example, as described above, the program 110 is stored in the storage device 107, and as described above, when the CPU 101 is executed, the memory 102 reads the operation program 110 from the storage device 106. Further, the storage device 107 includes, for example, a storage unit 10. The storage device 107 may further include, for example, an estimation model 110 of the physical condition described later.

The physical condition estimation device 1 may further include, for example, an input device 104. The input device 104 is, for example, a scanner, a touch panel, a keyboard, or the like.

As described above, the physical condition estimation device 1 has a display unit 11 and an eyeball image acquisition unit 12. The display unit 11 is, for example, a display, and specific examples thereof include an LED display and a liquid crystal display. The eyeball image acquisition unit 12 may be, for example, an imaging unit 105, and may photograph the eyeball of the subject who is viewing the visual stimulation test pattern displayed on the display unit 11. The type of the photographing unit 105 is not particularly limited, and is, for example, a camera such as an IR camera or a CCD camera.

The physical condition estimation device 1 will be described in more detail by taking as an example one device including each of the above parts.

The storage unit 10 includes test pattern data 16 for visual stimulation. The visual stimulation test pattern data 16 is the data of the pattern for displaying the visual stimulation test pattern on the display unit 11, and the visual stimulation test pattern is the eye movement data of two or more items of the subject. It is a pattern to acquire. The physical condition measuring device 1 of the present invention displays a visual stimulation test pattern on the display unit 11, continuously photographs the eyeballs of a subject who is viewing the visual stimulation test pattern, and the photographed eyeballs. Two or more items of eye movements are analyzed from the image, and combined eye movement data of two or more items are acquired. Therefore, the visual stimulation test pattern data 16 may be, for example, data of a pattern that can cause the eye movement necessary for analyzing the target item of the eye movement, and can be appropriately designed according to the target item. The visual stimulation test pattern data 16 may be, for example, data of a pattern that targets a plurality of target items at the same time from the start to the end of the pattern, or the target target items over time from the start to the end of the pattern. It may be the data of the pattern that changes.

The visual stimulation test pattern is, for example, an image that changes with time, that is, a moving image. As an example of the visual stimulation test pattern, a moving image composed of four consecutive parts (for example, the first part is a moving image for a pupillary light reflex test, the second part is a moving image for a follow-up examination, and the third part is a moving image. The video for the follow-up motion test and the video for the fixative test in the 4th part) can be mentioned. If the eyeball image of the subject who is viewing this visual stimulation test pattern is acquired, the data of the following items can be acquired from the eyeball image corresponding to each part. That is, data of dilation reduction speed based on pupil miosis and mydriasis can be obtained from the first part, follow-up data can be obtained from the second part, saccade data can be obtained from the third part, and fixation data can be obtained from the fourth part. In the present invention, as described above, the purpose is not to acquire the data itself of each item, but to acquire a combination of the data of these a plurality of items as a combination eye movement data and use it for estimating the physical condition described later. ..

Since the storage unit 10 also changes the combination eye movement data obtained depending on the content of the visual stimulation test pattern, for example, the visual stimulation test pattern data 16 may be plural (data 1, 2, 3, .... .. n is a positive integer.).

The storage unit 10 may further include physical condition information 108. The physical condition information 108 includes, for example, the type of the target physical condition that can be estimated by the physical condition estimation device of the present invention, and further, the visual stimulation test pattern data to be displayed on the display unit 11 for each type of the physical condition. 16 types may be associated.

That is, the visual stimulation test pattern data 16 may be created for each type of physical condition for estimation purposes, or a plurality of physical states can be estimated by the same visual stimulation test pattern data 16. be. Therefore, for example, the storage unit 10 may store each type of visual stimulation test pattern data 16 in association with an estimable type of physical condition, or each type of physical condition to be estimated. However, it may be stored in association with the visual stimulation test pattern data 16 that can be used for estimation. As a specific example, as shown in the schematic diagram of FIG. 3, in the former case, for example, the visual stimulation test pattern data 1 and the corresponding physical condition type A are associated with the stimulation test pattern data 2. , The corresponding physical condition type B is associated, and the stimulation test pattern data 3 and the corresponding physical condition types A and B are associated with each other. Further, the same applies to the latter case, and as shown in FIG. 3, for example, the physical condition type A and the stimulation test pattern data 1 and 3 are associated with each other, and the physical condition type B and the stimulation test are linked. An example is given that the pattern data 2 and 3 are associated with each other.

The storage unit 10 may further include, for example, subject information 109. The subject information 109 may be input by, for example, an input device 104 or the like and stored in the storage unit 10. The subject information 109 is associated with identification information for each subject, for example. The identification information is not particularly limited as long as it can identify each of a plurality of subjects, and is, for example, an ID such as an identification number, a name, an address, a telephone number, an e-mail address, or the like. The subject information 109 may be further associated with supplementary information for each subject, for example. Examples of the supplementary information include driving information such as medical history, interview information, driving history, and accident history.

The display unit 11 displays the visual stimulation inspection pattern based on the visual stimulation inspection pattern data 16. When a plurality of visual stimulation test pattern data 16 are stored in the storage unit 10, the display unit 11 is linked to the physical condition as described above, for example, according to the type of the physical condition of the target estimation target. The visual stimulation test pattern based on the attached visual stimulation test pattern data 16 can be displayed.

As a specific example, the display unit 11 first displays, for example, the type of physical information information 108 stored in the storage unit 10. Next, when the subject or an assistant of the subject (including, for example, a caregiver, a medical worker, etc.) selects the type of physical condition that needs to be estimated for the subject, the selection is made. The result is input to the physical condition estimation device 1 by the input device 104. Then, the visual stimulation test pattern data 16 associated with the input type of physical information is selected, and the visual stimulation test pattern based on the data is displayed on the display unit 11. Further, when there are a plurality of visual stimulation test pattern data 16 linked to the type of input physical information, for example, by further displaying them on the display unit 11, the subjects can select the data. The selection result is input, and the visual stimulation test pattern is displayed on the display unit 11.

For example, it is preferable to arrange the display unit 11 slightly above the position of the eyeball of the subject who sees the display unit 11 with an inclination so as to tilt toward the subject. This allows, for example, the subject's eyes to face up and the eyelids to open. Therefore, if an image is taken by the eyeball image acquisition unit 12 from below the eyeball of the subject, for example, the influence of the eyelids and eyelashes on the eyeball image can be suppressed. Further, with such a configuration, the eyeball image acquisition unit 12 does not enter the field of view of the subject, and the entire field of view can be covered by the display unit 11.

The eyeball image acquisition unit 12 continuously photographs the eyeball of the subject who is viewing the visual stimulation test pattern displayed on the display unit 11. At this time, for example, it is preferable that the eyeball is located at a certain distance from the display unit from the start to the end of the display of the visual stimulation test pattern. As a result, for example, it is possible to prevent the analysis result from being changed in the acquisition of the eye movement data, which will be described later, due to the change in the distance between the display unit 11 and the eyeball of the subject. Further, by displaying the same visual stimulation test pattern, for example, when acquiring combined eye movement data from a plurality of subjects or acquiring combined eye movement data from a sample group as described later, the same conditions are used. Since the acquired combined eye movement data can be used, the visual stimulation is performed in a state where the eyeball is located at a certain distance from the display unit for each subject and each subject of the sample group. It is preferable to display the inspection pattern. It is preferable to acquire the eyeball image as data in which the time series is aligned for each of the left and right, for example.

The distance between the eyeball and the display surface of the display unit 11 is preferably, for example, a distance in which the display surface on the display unit 11 on which the pattern is displayed is larger than the visual field of the subject. The upper limit of the distance is, for example, 40 cm or less, and the lower limit is, for example, a distance that does not allow eyepieces. The distance can be appropriately set, for example, according to the size of the display surface of the display unit 11. The size of the display surface is not particularly limited, the lower limit of the vertical length is, for example, 15 cm or more, the upper limit is not particularly limited, for example, 70 cm or less, and the lower limit of the horizontal length is. For example, it is 20 cm or more, and the upper limit is not particularly limited, and is, for example, 1 m or less. As a specific example, for example, when the vertical length of the display surface is 15 cm or more and the horizontal length is 20 cm, the distance is, for example, 15 to 40 cm.

Further, the physical condition estimation device 1 is preferably provided with a fixed portion of the head, for example, because the position of the subject's eyeball with respect to the display surface can be fixed. With the fixing portion, for example, it is easy to keep the distance between the subject's eyeball and the display portion constant. The fixed portion of the head is not particularly limited, and examples thereof include a jaw fixing portion for fixing the head by placing the jaw of the subject.

Further, as with the distance between the eyeball and the display surface of the display unit 11, for example, the same visual stimulus is applied to the positional relationship between the midpoint between the two eyeballs and the central axis on the display surface of the display unit 11. It is preferable that the positional relationship is the same with respect to the inspection pattern. For example, it is preferable that the midpoint between the two eyeballs and the central axis of the display surface of the display unit 11 coincide with each other.

If the same visual stimulation test pattern is displayed and the distance from the display unit to the eyeball differs depending on the individual subject and the individual subject, for example, correction may be performed. The correction will be described later.

The eye movement data acquisition unit 13 analyzes the eye movement image and acquires combination eye movement data of two or more items of eye movement. The items of eye movements obtained by the analysis of the eye movements are not particularly limited, and are, for example, pupil position, pupil diameter, pupil dilation / reduction rate, convergence and divergence movements, follow-up, sacked, fixation, and eyeballs. Left-right differences in movement, eye movements, iris, etc. can be mentioned. Since the individual items listed in the example are known as eye movement items, the pattern (for example, images and moving images) displayed to the subject in order to obtain the data of each item is based on, for example, a known technique. Can be set. Therefore, for example, the visual stimulation test pattern can be created as one moving image by stitching together images and / or moving images required for a plurality of target items, for example.

It should be noted that the present invention aims to confirm the eye movement of the subject for each item and diagnose the eye of the subject based on the data for each item, for example, as in the conventional ophthalmologic examination. is not. In the present invention, data on two or more items of eye movements are acquired from the eyeball images of the subject from the start to the end of the display of the pattern, and a combination of a plurality of data of these plurality of items, that is, a plurality of data. The physical condition described later is estimated using the overall behavior. That is, the present invention does not independently inspect each item whose meaning has already been clarified, and evaluate each result, as in the case of an ophthalmic examination, and the body has already been examined. It is not possible to estimate the physical condition from a single eye movement data whose direct relationship with the condition is known. The present invention uses data on the overall behavior of a plurality of items (combined eye movement data of two or more items) instead of a single item. Therefore, for example, two or more items are preferable, and it is more preferable that the items include an item obtained by pupillary light reflex and an item obtained by nystagmus detection as basic items.

The physical state estimation unit 14 estimates the physical state of the subject from the physical state estimation information with respect to the combination eye movement data of the subject acquired for the displayed visual stimulation test pattern. .. The physical state estimation information is information showing the relationship between the eye movement data and the physical state in combination with the visual stimulation test pattern data.

The physical condition estimation unit 14 preferably has, for example, a physical condition estimation model 111. The estimation unit 14 has the physical condition estimation model 14, for example, as long as the physical condition estimation model 14 can be used in the estimation process of the estimation unit 14, and in the hardware configuration shown in FIG. 2, the storage device 107 has the physical condition estimation model 14. It may be stored or incorporated into the CPU 101. According to this embodiment, for example, the physical condition of the subject can be estimated from the combined eye movement data of the subject by the physical condition estimation model 111. Therefore, in this embodiment, for example, it can be said that the physical state estimation model 111 has the physical state estimation information.

The physical state estimation model 111 is, for example, a model generated by learning, and is a combination of a specific visual stimulus test pattern data and the eye movement image data of a subject who is viewing the visual stimulus test pattern. This is a model that represents the relationship (algorithm) between the eye movement data and the target physical condition. The physical state estimation model 111 is, for example, a combination of eye movement data of a learning sample group which is a physical state for estimation purpose and a learning sample group which is not a physical state for estimation purpose, which is obtained by displaying the test pattern for visual stimulation. Combined eye movement data is used as learning data and is generated by learning. The learning may be so-called machine learning, and deep learning may be used therein.

The learning sample group that is the physical state for the estimation purpose and the learning group that is not the physical state for the estimation purpose can be appropriately set according to the physical state of the target. When the physical condition for estimation is dementia, for example, the same stimulus is used for a learning sample group for patients with dementia and a learning sample group for healthy people without dementia. By displaying the test pattern, the eyeball image is acquired and the combined eye movement data is acquired, and the combined eye movement data of the learning sample group for dementia and the combined eye movement data of the learning sample group not dementia are combined with the learning data. can do. Further, for the purpose of estimating the classification of the type of dementia in more detail, for example, a sample group for learning Alzheimer's dementia and a sample group for learning non-Alzheimer's dementia can be set. .. Further, the physical condition is not limited to the estimation purpose, and the degree (stage) thereof is, for example, a low-level learning sample group, a medium-level learning sample group, and a high-level learning sample group. Combination of learning sample groups for each stage Eye movement data can be used as learning data.

Further, for the generation of the physical state estimation model 111, for example, only the learning sample group of the subject who is in a normal physical state may be used as the learning data. In this case, it is possible to generate a model at a lower cost than training both a normal training sample and an abnormal training sample, and the generated model also enables detection of an unknown anomaly. ..

Further, when estimating the driving ability as the physical condition, it is preferable that the learning data includes, for example, the past accident history as the information of the subject.

The body state estimation device 1 of the present invention may include, for example, a body state estimation model 111 already generated by learning, and further includes a learning unit (not shown) that generates a body state estimation model 111. May be good. For machine learning using the learning data, for example, a commercially available system may be used, and the physical condition estimation device 1 may include the system as the learning unit. The physical condition estimation model 111 generated by the physical condition estimation device 1 is stored in, for example, a storage device.

The physical condition estimation device 1 may have an output unit 15 as described above. For example, the physical state estimation device 1 may display the physical information estimated by the estimation unit 14 on the display unit 11 in the device via the output unit 15, or may display the physical information estimated by the estimation unit 14 on the display unit 11 in the device, or the external body information via the output unit 15. It may be output to a device or the like.

As described above, the method for estimating the physical condition of the present invention includes, for example, a step of acquiring an eyeball image, a step of acquiring eye movement data, and a step of estimating the physical condition of the subject using a storage unit. The physical condition estimation method of the present invention can be executed, for example, by using the physical condition estimation device of the present invention. As the physical condition estimation method of the present invention, for example, unless otherwise specified, the description of the physical condition estimation device of the present invention can be incorporated.

The storage unit has at least one test pattern data for visual stimulation. The storage unit is, for example, the storage unit 10 of the physical condition estimation device 1.

In the display step, a visual stimulus inspection pattern based on the visual stimulus inspection pattern data is displayed on the display unit. This step can be executed, for example, by the display unit 11 of the physical condition estimation device 1.

The eyeball image acquisition step is a step of continuously photographing the eyeball of a subject who is viewing the visual stimulation test pattern displayed on the display unit and acquiring an eyeball image, for example, a physical condition. This can be executed by the eyeball image acquisition unit 12 of the estimation device 1.

The eye movement data acquisition step is a step of analyzing the eye movement image and acquiring a combination eye movement data of two or more items, for example, an eye movement data acquisition unit 13 of the physical condition estimation device 1. Can be executed by.

The estimation step is a step of estimating the physical condition of the subject from the acquired combined eye movement data of the subject based on the physical condition estimation information. This step can be performed, for example, by the estimation unit 14 of the physical condition estimation device 1.

The physical condition estimation method of the present embodiment may further include, for example, an output step. The output step is a step of outputting the estimated physical condition of the subject, and can be executed by, for example, the output unit 15 of the physical condition estimation device 1.

According to the present embodiment, for example, by displaying the same visual stimulation test pattern, estimating the physical condition of the same subject multiple times, and accumulating the estimation results over time, the body of the estimation item Changes in the subject can be analyzed for the condition. In addition, the same visual stimulation test pattern is displayed, combined eye movement data of the same subject is acquired multiple times, these are accumulated over time, and the accumulated eye movement data over multiple times is desired. By estimating the physical condition of the subject, it is possible to analyze the change of the subject by changing the type of the target physical condition each time. In addition, the same visual stimulation test pattern is displayed, combined eye movement data is acquired for multiple subjects under the same conditions, and the same type of physical condition is estimated, and the body among the subjects is estimated. You can also compare states.

(Modification 1)
The physical condition estimation device 1 further adjusts the brightness of the space between the field of view of the subject and the display unit 11 (for example, the display surface of the subject side (display surface side)). It may have a portion (not shown).

Depending on the item of eye movement data, for example, for the pattern, an eyeball image under dark conditions, an eyeball image under bright conditions, and an eyeball image under switching conditions between dark and bright conditions may be acquired. Is also preferable. Therefore, according to the light / dark adjustment unit, for example, in order to acquire a target eyeball image, the space is switched between a dark condition and a light condition at an arbitrary timing according to the type of the visual stimulation test pattern. Can be done.

The type of the light / dark adjustment unit is not particularly limited. The physical condition estimation device has, for example, a housing, and a display unit 11, an eyeball image acquisition unit 12, and a light / dark adjustment unit are arranged inside the housing, and the housing is provided on one surface of the housing. There is a form having an observation window for viewing the display surface of the display unit 11 inside the display unit 11. The shape of the observation window is not particularly limited, and is, for example, a goggle type. According to this embodiment, the inside of the housing can be set to a dark condition, for example, when the light / dark adjustment unit is not used. Further, the type of the light / dark adjustment unit is not particularly limited. Since a display capable of displaying the pattern is generally capable of adjusting the brightness (adjusting the amount of light), for example, the light amount adjusting unit included in the display unit 11 can be used as the light / dark adjusting unit. The amount of light can be changed from a dark condition to a bright condition or the brightness under the bright condition can be adjusted at a timing necessary for displaying the visual stimulation test pattern. The light / dark adjustment unit may be, for example, the light / dark adjustment unit of the display unit 11.

The physical condition estimation device 1 may be provided with an optical information measuring unit in the vicinity of the observation window, and may further have a correction unit for correcting the influence of optical information. According to the optical information measuring unit, information on light conditions (luminous intensity of the screen, illuminance, etc.) at the time of use can be acquired, and the influence of the eyeball image or the influence of the eyeball motion data received by these light conditions is corrected by the correction unit. can do.

(Modification 2)
The physical condition estimation device 1 may further include, for example, a correction unit (not shown). As described above, when the display of the same visual stimulation test pattern and the distance from the display unit to the eyeball differs depending on the individual subject and the individual subject for the learning data, for example, the correction. It is preferable to make corrections according to the unit. The correction may be performed on the acquired eyeball image, for example, and the eye movement data may be acquired from the corrected eyeball image. Further, in the correction, for example, the eye movement data is acquired from an uncorrected eye image, the acquired eye movement data is corrected, and the physical condition is estimated based on the combination of the corrected eye movement data. May be good.

The method of correction is not particularly limited, and for example, the eyeball image may be corrected based on the difference in the eyeball image caused by the difference in the distance from the display unit to the eyeball, or the eyeball from the display unit. The eye movement data may be corrected based on the difference in the eye movement data that occurs according to the difference in the distance to.

Regarding the positional relationship between the midpoint between the two eyeballs and the central axis on the display surface of the display unit 11, if the positional relationship differs each time the eyeball image is acquired, the same method as for the distance may be applied. Can be corrected.

(Modification 3)
The physical state estimation device 1 may be, for example, a device having a housing structure or a terminal as exemplified in the modification 1.

In the device having the housing structure, for example, a display unit 11, an eyeball image acquisition unit 12, and a light / dark adjustment unit are arranged inside the housing, and a display inside the housing is displayed on one surface of the housing. An example is a form having an observation window for viewing the display surface of the unit 11. Further, in the housing, it is preferable to further have a fixing portion of the head for fixing the eyeball position of the subject to be constant on the outside of the surface having the observation window.

On the other hand, examples of the terminal include smartphones, mobile phones, tablets, wearable terminals and the like. Examples of the wearable terminal include goggles-type terminals and the like. As the terminal, for example, a smartphone and a goggle type terminal are preferable. When the terminal is a smartphone, for example, it is preferable to use it in combination with a goggle-type fixing tool because the distance and the positional relationship between the subject's eyeball and the display unit 11 can be fixed. The goggle-type fixing tool has, for example, a fixing portion to which a smartphone can be attached and detached. For example, the subject fixes the smartphone to the fixing portion of the fixing tool, attaches it to the head, and sees the inspection pattern for visual stimulation displayed on the display of the smartphone. At the same time, the smartphone can take a picture of the subject's eyeball with the smartphone's camera, acquire combined eye movement data from the eyeball image, and estimate the physical condition. Further, in the case of a goggle-type wearable terminal, for example, by arranging a camera inside the goggles, it is possible to estimate the physical condition in the same manner.

When the physical condition estimation device 1 is a terminal, for example, the subject can easily estimate the physical condition at home or on the go on a daily basis.

(Modification example 4)
The visual stimulation test pattern may include, for example, a wind jet in addition to the above-mentioned moving image. In this case, in the physical condition estimation device 1, for example, the storage unit 10 has wind injection data as the visual stimulation inspection pattern data, and further has a wind injection unit. The eye movement data acquisition unit 13 can acquire, for example, the reaction time that caused the blink due to the wind from the acquired eye image.

Further, the visual stimulation test pattern may include, for example, the movement of an index object in addition to the above-mentioned moving image. In this case, in the physical state estimation device 1, for example, the storage unit 10 has the movement data of the index object as the inspection pattern data for visual stimulation, and further has a moving object and a movement adjusting unit for moving the moving object. The eye movement data acquisition unit 13 can measure, for example, the reaction time that caused the near-sight reflex when the index substance approaches the eyeball side from the acquired eyeball image.

(Modification 5)
As described above, the physical condition estimation device 1 may be connectable to the external terminal via, for example, a communication network. The estimation result obtained by the physical condition estimation device 1 may be transmitted to the terminal of the subject himself or the terminal of a medical worker for information sharing, for example.

[Embodiment 2]
The physical state estimation device of the present invention is, for example, a device (hereinafter, physical state) to which each part can be connected via a communication network in addition to one device including each part as shown in the first embodiment. It may be an estimation system). The communication network is, for example, the same as described above. As the present embodiment, a system (physical state estimation system of the present invention) in which a terminal having a display unit and a device on which other units are mounted can be connected via a communication network will be described by way of an example. ..

Further, the device on which each part other than the display part used in the physical state estimation system is mounted is the second physical state estimation device of the present invention. Unless otherwise specified, the second physical condition estimation device can incorporate the description of the first embodiment. The second physical condition estimation device is, for example, a fixing unit for fixing a terminal, a storage unit, an eyeball image acquisition unit for capturing an image of a subject's eyeball, an eye movement data acquisition unit, and the subject. Includes an estimate of the examiner's physical condition. The second physical condition estimation device may further include, for example, a light condition measuring unit for measuring the light condition of the terminal and a correction unit. The device can be connected to the terminal, and the connection method is not particularly limited, and the connection via the communication network as described above can be mentioned. In the apparatus, the fixing unit fixes the terminal, the light condition measuring unit measures the light condition of the display unit of the terminal, and the correction unit measures the measured light condition of the terminal with arbitrary light. The correction condition for making the condition is calculated, and the terminal adjusts the optical condition based on the calculated correction condition.

FIG. 4 is a block diagram showing a configuration of an example of the physical condition estimation system of the present embodiment. The terminal 2 such as a smartphone has a display as the display unit and can be connected to the device 3 via the communication network 4. The device 3 is an example of the second physical condition device of the present invention, and has a storage unit 10, an eyeball image acquisition unit 12, an eye movement data acquisition unit 13, and a physical condition estimation unit 14, and further, an estimation result. Has an output unit 15 for outputting the terminal 2 to the terminal 2 via the communication network 4. The terminal 2 and the server 3 may be connected by wire.

The device 3 has, for example, a fixed portion of the terminal 2 and an observation window. In the device 3, for example, when the terminal 2 is fixed to the fixed portion, the observation window is arranged so that the subject can see the display (display portion) of the terminal 2. According to the system, first, the visual stimulation test pattern is displayed on the display of the fixed terminal 2. While the subject is looking at the display on the display through the observation window, the eyeball of the subject is photographed by the eyeball image acquisition unit (camera) 12 of the device 3. Then, the device 3 acquires the combination eye movement data from the acquired eye movement image and estimates the physical condition, and outputs the estimation result to the terminal 2 via the communication network 4. The terminal 2 that has received the estimation result can display the estimation result on the display and, if necessary, store the estimation result in the storage unit of the terminal 2.

(Modification 1)
The system of the present invention can be used by individual users using individual terminals 2 (for example, smartphones), for example, as shown in FIG. In this case, even when a plurality of users display the same visual stimulation test pattern on the terminal, the display conditions do not always match depending on the terminal 2 used. Therefore, the device 3 may further have, for example, a correction unit (not shown) that corrects the individual usage conditions, that is, the display conditions of the terminal 2.

The individual differences of smartphones include, for example, the size of the display, the number of pixels, the DPI, the brightness of the display itself depending on the type of display (for example, liquid crystal or organic EL, backlight performance, etc.), the calculation performance of the smartphone itself, etc. Can be given. Regarding the brightness, for example, it is preferable to quantitatively measure the brightness of the display with an illuminance meter and make it uniform among smartphones. Therefore, it is preferable that the device 3 includes, for example, the optical condition measuring unit and the correction unit that measure display conditions such as brightness. According to this device 3, for example, the light condition measuring unit can measure the brightness condition of the display of the smartphone, and the correction unit can calculate the correction condition for setting the desired brightness condition. Then, by outputting the correction condition to the smartphone (terminal 2) from the device 3, the smartphone that has received the correction condition can display the display under the desired brightness condition.

[Embodiment 3]
The program according to the third embodiment of the present invention is a program capable of executing the physical information estimation method of the present invention on a computer. Alternatively, the program of this embodiment may be recorded on a computer-readable recording medium, for example. The recording medium is not particularly limited, and examples thereof include a storage medium as described above.

Although the invention of the present application has been described above with reference to the embodiments, the invention of the present application is not limited to the above-described embodiment. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the configuration and details of the present invention.

According to the present invention, a test pattern for visual stimulation is shown to a subject without using an advanced device such as a brain wave measuring device, and eyeball images of the subject are continuously photographed. Combination data of eye movements of a plurality of items can be acquired from the eyeball image, and the physical condition of the subject can be estimated from the combination data based on the physical condition estimation information. Therefore, for example, the physical condition of the subject can be easily estimated regardless of the medical staff. According to the present invention, for example, the presence or absence or degree of driving ability, the possibility of brain damage and dementia, and the like can be easily estimated, which can be useful for returning a driver's license and improving life.

1 Physical condition estimation device 2 Terminal 3 Device (second physical condition estimation device)
4 Communication network 10 Storage unit 11 Display unit 12 Eyeball image acquisition unit 13 Eye movement data acquisition unit 14 Physical condition estimation unit 15 Output unit 16 Visual stimulation test pattern data 101 CPU
102 Memory 103 Bus 104 Input device 105 Imaging unit 106 Communication device 107 Storage device 108 Physical condition information 109 Subject information 110 Program 111 Physical condition estimation model

Claims (21)

Includes a storage unit, a display unit, an eyeball image acquisition unit that captures an image of the subject's eyeball, an eye movement data acquisition unit, and an estimation unit of the subject's physical condition.
The storage unit is
It has at least one test pattern data data for visual stimulation,
The visual stimulation test pattern is a pattern for acquiring eye movement data of two or more items of a subject.
The display unit is
The inspection pattern for visual stimulation based on the inspection pattern data for visual stimulation is displayed, and the inspection pattern for visual stimulation is displayed.
The eyeball image acquisition unit is
The eyeball of the subject who is looking at the visual stimulation test pattern displayed on the display unit is continuously photographed, and an eyeball image is acquired.
The eye movement data acquisition unit is
By analyzing the eyeball image, eye movement data which is a combination eye movement data of two or more items of eye movement is acquired.
The estimation unit is
With respect to the acquired eye movement data of the subject, the physical condition of the subject is estimated from the physical condition estimation information.
The physical state estimation information is a physical state estimation device characterized in that it is information indicating a relationship between visual stimulation test pattern data, eye movement data, and physical state. The estimation unit includes a physical condition estimation model.
The physical state estimation model is obtained by displaying the visual stimulation test pattern, and is obtained by eye movement data of a learning sample group that is a physical state for estimation purpose and eye movement data of a learning sample group that is not a physical state for estimation purpose. From, it is a model generated by learning,
The physical condition estimation device according to claim 1, wherein the physical condition of the subject is estimated by the physical condition estimation model. The estimation unit includes a physical condition estimation model.
The physical state estimation model is a model generated by learning from eye movement data of a learning sample group which is a normal physical state for estimation purpose, which is obtained by displaying the visual stimulation test pattern.
The physical condition estimation device according to claim 1, wherein the physical condition of the subject is estimated by the physical condition estimation model. The eye movement item is selected from the group consisting of pupil position, pupil diameter, pupil dilation / reduction rate, vergence divergence movement, follow-up, sacked, fixation, laterality of eye movement, eye shake, and iris. The physical condition estimation device according to any one of claims 1 to 3, further comprising at least one. The physical condition estimation device according to any one of claims 1 to 4, wherein the physical condition to be estimated is the presence or absence or degree of driving ability, brain function, nerve function, or cognitive function. The device for estimating a physical condition according to any one of claims 1 to 5, wherein the physical condition to be estimated is a disease. The physical condition estimation device according to claim 6, wherein the disease is a brain disorder, dementia, or a psychiatric disorder. The physical condition estimation device according to any one of claims 1 to 7, further comprising a light / dark adjusting unit for adjusting the light / darkness of the space between the field of view of the subject and the subject side of the display unit. .. Has a housing,
The display unit, the eyeball image acquisition unit, and the light / dark adjustment unit are arranged inside the housing.
The physical condition estimation device according to claim 8, further comprising an observation window for viewing the display surface of the display unit inside the housing on one surface of the housing. The physical condition estimation device according to any one of claims 1 to 9, further comprising a fixed portion of the head for keeping the distance between the subject's eyeball and the display portion constant. A terminal with a display unit and
Fixed unit, light condition measurement unit, correction unit, storage unit, eyeball image acquisition unit for capturing an image of the subject's eyeball, eye movement data acquisition unit, and physical condition estimation unit for the subject. Including equipment including
The terminal and the device are connectable and can be connected.
In the above device
The fixing portion fixes the terminal and
The optical condition measuring unit measures the optical condition of the display unit of the terminal, and the optical condition measuring unit measures the light condition.
The correction unit calculates correction conditions for setting the measured light conditions of the terminal to arbitrary light conditions.
The physical condition estimation device according to any one of claims 1 to 10, wherein the terminal adjusts light conditions based on the calculated correction conditions. Use the storage
Includes a display step, an eyeball image acquisition step of taking an image of the subject's eyeball, an eye movement data acquisition step, and an estimation step of the subject's physical condition.
The storage unit is
It has at least one test pattern data for visual stimulation and
The visual stimulation test pattern is a pattern for acquiring eye movement data of two or more items of a subject.
The display step is
A visual stimulation test pattern based on the visual stimulation test pattern data is displayed on the display unit, and the visual stimulation test pattern is displayed.
The eyeball image acquisition step is
The eyeball of the subject who is looking at the visual stimulation test pattern displayed on the display unit is continuously photographed, and an eyeball image is acquired.
The step of acquiring the eye movement data is as follows.
By analyzing the eyeball image, a combination eye movement data of two or more items of eye movement is acquired, and the eye movement data is acquired.
The estimation process is
With respect to the acquired combined eye movement data of the subject, the physical condition of the subject is estimated from the physical condition estimation information.
The physical state estimation information is a method for estimating a physical state, which is information showing a relationship between eye movement data and a physical state in combination with a visual stimulation test pattern data. The estimation process uses a physical condition estimation model and
The physical state estimation model is a combination eye movement of a learning sample group that is a physical state for estimation purpose and a learning sample group that is not a physical state for estimation purpose, which is obtained by displaying the test pattern for visual stimulation. It is a model generated by learning from motion data,
The method for estimating a physical condition according to claim 12, wherein the physical condition of the subject is estimated by the physical condition estimation model. The estimation process uses a physical condition estimation model and
The physical state estimation model is a model generated by learning from eye movement data of a learning sample group which is a normal physical state for estimation purpose, which is obtained by displaying the visual stimulation test pattern.
The method for estimating a physical condition according to claim 12, wherein the physical condition of the subject is estimated by the physical condition estimation model. The eye movement item is selected from the group consisting of pupil position, pupil diameter, pupil dilation / reduction rate, vergence divergence movement, follow-up, sacked, fixation, laterality of eye movement, eye shake, and iris. The method for estimating a physical condition according to any one of claims 12 to 14, further comprising at least one. The method for estimating a physical condition according to any one of claims 12 to 15, wherein the physical condition to be estimated is the presence or absence or degree of driving ability, brain function, nerve function, or cognitive function. The method for estimating a physical condition according to any one of claims 12 to 16, wherein the physical condition to be estimated is a disease. The method for estimating a physical condition according to claim 17, wherein the disease is a brain disorder, dementia, or a psychiatric disorder. The method for estimating a physical condition according to any one of claims 12 to 18, wherein the display and imaging are performed on the display unit while adjusting the brightness of the space between the field of view of the subject and the display unit. A program comprising causing a computer to execute the method for estimating a physical condition according to any one of claims 12 to 19. A computer-readable recording medium comprising recording the program according to claim 20.

Images