JP3356340B2 - Pupil area measuring device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pupil area measuring device for measuring the area of a pupil of a subject, and more particularly to an apparatus for accurately measuring a change in pupil area due to a mental factor by removing the influence of the brightness of a presentation screen. The present invention relates to a pupil area measuring device capable of measuring the pupil area at a high speed.

[0002]

2. Description of the Related Art The size of a pupil changes according to the brightness of an object to be observed, but also changes due to mental factors.
When observing an object with a high degree of interest or observing the object with mental concentration, the size of the pupil increases. In this way, the pupil size is sensitive to psychological changes such as mental tension, interest, and attention.

[0003] By using this phenomenon to measure the change in pupil area, it has been investigated what kind of object a person is interested in. In addition, it is possible to objectively know the interests and psychological state of the person at that time. By using this measurement method, for example, it is possible to know a change in interest or a psychological change of a student during a lecture at a university, and to give appropriate guidance to the student. It can also be used to measure the mental state of an athlete in image training. Further, such a pupil reaction measurement has an important medical significance in order to know the physiology of the retinal cerebral cortex and the like, and has been studied with interest in terms of exploring a visual information processing function.

Conventionally, a change in the pupil area has been measured using the apparatus shown in FIG. As shown in the figure, a conventional pupil area measuring device includes a screen presenting unit 300 for presenting a pattern of a landscape or a person, a pupil photographing unit 310 for photographing an image of a subject's eye, and image processing of the photographed image. And an image processing unit 320 for calculating the pupil area. For example, a slide projector, a video monitor, or the like is used as the screen presenting unit 300, and a pupil photographing unit 310, for example, having a shape of goggles or a hat that can be worn by a subject is used. With this measuring device, it is possible to objectively evaluate the mental change of the subject with respect to the screen presented on the screen presenting unit 300 as a change in the pupil area.

Next, a general configuration of the pupil photographing section 310 is shown in FIG. The visible light from the screen presentation unit 300 passes through the filter 311 and enters the subject's eye. The subject can observe the screen with the incident light. The light reflected from the eye returns along the same path, is reflected by the filter 311, enters the camera 312, and captures an image of the eye.
The filter 311 is a filter for separating visible light of the screen presented on the screen presentation unit 300 and light reflected from the eyes. As the camera 312, for example, a CCD camera is used. The captured image is provided to the image processing unit 320, and image processing such as binarization is performed based on a luminance difference due to a difference in reflectance between the iris and the pupil, and the pupil is extracted. Then, the area is obtained from the extracted pupil image.

As described above, by using the conventional pupil area measuring device, it is possible to objectively evaluate mental changes such as a degree of interest and attention of the user on the presentation screen and a mental state.

[0007]

However, since the size of the pupil also changes depending on the brightness of the presentation screen, the change in the size of the pupil is caused by a change in brightness or by a mental change. It is necessary to distinguish between them. For this reason, in the conventional measuring device, only the mental change is extracted by previously adjusting all the presentation screens to have the same brightness. Specifically, the illumination light of the slide projector is changed for each presentation screen so that the brightness of the entire presentation pattern is constant, or the illumination light or the brightness of the camera is adjusted when shooting with a video camera. . Therefore, it took a lot of time and effort to create screens with the same brightness. In addition, a screen whose luminance differs greatly between the center and the end changes the brightness of the screen depending on which part of the screen the subject looks at, so such a screen is not suitable for a presentation screen. Therefore, the content of the presentation is limited, and the display is limited to a still image of a part of a landscape image or a human image.

[0008] In addition, since a television program aims to obtain a high audience rating, it is important to know the viewer's interest or interest in television images in real time. However, since the brightness of television images is changing moment by moment, it has been difficult to separate pupil changes due to light reflection from pupil changes due to mental factors such as interest or interest. For this reason, although it is important to measure the viewer's interest or interest in television images, the conventional pupil area measuring device cannot measure it, which is a problem.

Further, since the presentation screen is affected by changes in the surrounding brightness, it is necessary to perform measurement in a room or a dark room where the brightness is kept constant, and the surrounding brightness such as outdoors changes. There is a problem that the measurement environment is limited, for example, measurement cannot be performed in an environment.

An object of the present invention is to solve such a problem and to provide a pupil area measuring device capable of accurately measuring a change in pupil area due to a mental factor.

[0011]

In order to solve the above problems, a pupil area measuring apparatus according to the present invention is provided with (a) a screen presenting section for presenting a predetermined screen, and (b) a screen presenting section. An imaging unit that captures an image of the eye of the subject who has observed the screen,
(C) an image processing unit that analyzes an image captured by the imaging unit to extract a pupil contour, and (d) a pupil area calculation unit that calculates a pupil area from the pupil contour extracted by the image processing unit.
(E) a light detection unit that detects a change in the brightness of the screen presented on the screen presentation unit; and (f) an area of the pupil of the subject who observes the screen based on the brightness change of the screen presented on the screen presentation unit. And (g) reading out from the correction table the correction data corresponding to the luminance change of the screen detected by the light detection unit.
A pupil area correction unit that corrects the pupil area calculated by the pupil area calculation unit using the correction data. here,
The correction table may store correction data of the individual subject measured in advance.

Further, in the pupil area measuring apparatus according to the present invention, the time until the information of the luminance change of the screen detected by the light detecting section reaches the pupil area correcting section is delayed in the above configuration.
A delay processing unit is provided for matching the timing at which the correction data is read from the correction table with the timing at which the pupil area information calculated by the pupil area calculation unit reaches the pupil area correction unit. Here, the delay processing in the delay processing unit may be performed with a different delay time for each subject.

[0013]

When the subject observes the screen presented on the screen presenting section, the pupil area of the subject changes due to a change in luminance of the screen and a mental factor obtained from the content of the screen. According to the present invention, of the change in the pupil area of the subject, a change due to a change in screen brightness is removed, and only a change due to a mental factor is extracted. Therefore, the mental state of the subject such as interest or interest can be grasped from the change in the extracted pupil area.

The pupil area of the subject is obtained by extracting the contour of the pupil from the image of the eye of the subject photographed by the photographing section by the image processing section, and further calculating the pupil area from the contour of the pupil by the pupil area calculating section. Obtained by calculation. The pupil area obtained in this manner is corrected to remove a change due to a change in luminance of the presentation screen. In the correction, first, a change in the brightness of the screen presented on the screen presentation unit is detected by the light detection unit, and is provided to the pupil area correction unit. In the pupil area correction unit, correction data corresponding to the detected luminance change of the screen is read from the correction table, and the pupil area is corrected using the correction data.

In this case, if the correction data stored in the correction table is data measured in advance for each subject, more accurate correction of the pupil area can be performed.

Since a human reaction to an external stimulus requires a certain period of time, the pupil area of the subject does not change immediately even if the brightness of the screen presented on the screen presentation unit changes. In the case where the delay in the change of the pupil area affects the measurement of the pupil area, the present invention deals with the problem by providing a delay processing unit. In other words, the delay processing unit delays the time until the information on the luminance change of the presentation screen reaches the pupil area correction unit, so that even when the pupil area change is delayed, the timing of reading the correction data from the correction table The timing at which the information on the pupil area calculated by the pupil area calculation unit reaches the pupil area correction unit is matched.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, first to fourth embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing a configuration of the pupil area measuring device according to the first embodiment.
As shown in the figure, the pupil area measuring apparatus of the present embodiment includes a screen presenting unit 10 for presenting a predetermined screen, a half mirror 20 for transmitting and reflecting light from the presented screen, and a light reflected by the half mirror 20. Detector 30 for detecting the intensity of light
And a camera 40 that captures an image of the eye by inputting light reflected by the subject's eye when light transmitted through the half mirror 20 is provided. Further, an image processing unit 50 that analyzes an image captured by the camera 40 to extract a pupil contour, a pupil area calculation unit 60 that calculates a pupil area from the extracted pupil contour, and stores the calculated pupil area data. A pupil area storage unit 70; Furthermore, a correction table creation unit 80 that creates a correction table from the light intensity data detected by the photodetector 30 and the pupil area data calculated by the pupil area calculation unit 60
And a correction table storage unit 90 for storing the generated correction table. Furthermore, the correction table storage unit 9
An arithmetic unit 100 for correcting the pupil area data stored in the pupil area storage unit 70 with the correction data of the correction table stored in 0, and a display unit 110 for displaying the corrected pupil area data. Here, a slide projector, a video monitor, or the like is used for the screen presentation unit 10. In addition, a photodiode or the like is used for the light detection unit 30.

Next, the operation of this embodiment will be described.
The operation of this embodiment mainly includes two processes. The first is to present a white screen of uniform brightness to the screen presenting unit 10 and measure the brightness of this screen and the pupil area of the subject who has observed this screen, respectively. This is a process for creating a curve. The second presents various screens whose luminance changes on the screen presenting unit 10, measures the luminance of this screen and the pupil area of the subject who has observed this screen, and uses the correction table described above. This is a process for correcting pupil area data so as to remove the influence of the luminance of the presentation screen. By correcting the pupil area data in this way, it is possible to measure a change in the pupil area due to a mental factor such as interest or interest.

First, a correction table or a correction curve creation process as the first process will be described. Screen presentation unit 1
In the case of 0, a white screen in which the entire screen has uniform brightness is presented with various brightnesses, and the intensity of light reflected by the half mirror 20 is detected by the photodetector 30. The light transmitted through the half mirror 20 is incident on the eye of the subject, and the reflected light from the eye is reflected by the half mirror 20 and is incident on the camera 40.
A pupil image of the subject is captured by this incidence. If the transmission ratio of the half mirror 20 is known, the amount of light entering the eye and the amount of light entering the photodetector 30 are obtained by proportional distribution. The image captured by the camera 40 is provided to the image processing unit 50, where the image processing unit 50 performs processing such as binarization, and extracts the contour of the pupil. The contour of the pupil extracted by the image processing unit 50 is provided to the pupil area calculation unit 60, and the pupil area is calculated. From the pupil area calculated in this way and the screen brightness detected by the photodetector 30, a correction table or a correction curve for the brightness is created by the correction table creating unit 80. The correction table of the pupil area with respect to the luminance is stored in the correction table storage unit 90. Here, an example of a correction curve of the pupil area with respect to the luminance is shown in the graph of FIG. The vertical axis of this graph indicates the pupil diameter, and the horizontal axis indicates the luminance of the presentation screen.

Next, a description will be given of a second process of measuring pupil area change due to mental factors. Various screens whose brightness and color change with time are presented on the screen presenting unit 10, and the intensity of light reflected by the half mirror 20 of the incident light is detected by the photodetector 30. The light transmitted through the half mirror 20 is incident on the eye of the subject, and the subject observes the presentation screen. The reflected light from the subject's eyes is reflected by the half mirror 20 and is incident on the camera 40. A pupil image of the subject is captured by this incidence. The pupil image captured by the camera 40 is stored in an image processing unit 50.
Then, image processing such as binarization is performed, and the pupil area calculation unit 60 calculates the pupil area. The pupil area data thus calculated is sent to the pupil area storage unit 70. Since the pupil area obtained here includes both causes of the luminance of the presentation screen and mental factors, the factor due to the luminance of the presentation screen is deleted from the correction data of the correction table stored in the correction table storage unit 90. There is a need to. The light intensity data detected by the light detection unit 30 is input to the correction table storage unit 90, and correction data corresponding to the light intensity data is read from the correction table. This correction data and the pupil area storage unit 7
The pupil area data stored in 0 is sent to the arithmetic unit 100, and the pupil area from which the luminance change of the presentation screen is removed is obtained by subtracting the correction data from the pupil area data. The pupil area data thus obtained is displayed on the display unit 110.

As described above, by using this embodiment, the component of the pupil area change due to the change in the brightness of the presentation screen can be removed, so that the brightness of the presentation screen, which is conventionally required, must be kept constant in advance. For example, problems such as inability to measure mental changes and mental state can be solved. In addition, there is no limitation that only a presentation screen having uniform luminance over the entire screen can be used as in the related art. Further, since a video such as a television program can be used, a change in a pupil due to a mental state can be measured moment by moment, and an objective evaluation of a viewer can be obtained in real time.

Next, a second embodiment will be described with reference to FIG. The configuration of the second embodiment is different from that of the first embodiment shown in FIG. 1 in that a filter 120 is provided instead of the half mirror 20, and a half mirror 130 is provided between the filter 120 and the camera 40. The point is that an auxiliary illumination light source 140 is provided. The filter 120 functions similarly to the half mirror 20, but has a characteristic of reflecting near-infrared light and transmitting visible light. The auxiliary illumination light source 140 is a light source that emits near-infrared light. The screen presentation unit 10 includes:
Over time, screens of various brightnesses from bright to dark are presented. A good image can be captured by the camera 40 during a bright screen, but the camera 4
Insufficient light quantity to 0 makes it difficult to capture good images. Even in such a case, the irradiation light from the auxiliary illumination light source 140 is given to the eye of the subject.
Regardless of the brightness of the screen presented by the screen presentation unit 10,
Good images can be taken at any time. In this embodiment, near-infrared light that is insensitive to the eyes is used so that the pupil area does not change due to light from the auxiliary illumination light source 140.
The near-infrared light emitted from the auxiliary illumination light source 140 is reflected by the half mirror 130 and the filter 120, respectively, and enters the eye of the subject. The near-infrared light reflected from the eye is reflected by the filter 120, passes through the half mirror 130,
The light is incident on the camera 40. With this incidence, an image of the eye is captured by the camera 40. Subsequent processing is the same as in the first embodiment.

Next, the relationship between the time change of the pupil area and the light stimulus will be described with reference to the graph of FIG. From the figure,
When a light stimulus due to a change in luminance is given, a change in the pupil shape starts after a lapse of (t1 + t2) time since the stimulus is given. Similar delays are also observed in changes in the speed and acceleration of the pupil shape. In order to more accurately remove the change due to the luminance change of the presentation screen from the change in the pupil area, it is necessary to consider such a pupil reaction delay.

Next, FIG. 5 shows a configuration of a pupil area measuring apparatus according to a third embodiment in which a pupil reaction delay is considered. The third embodiment is different from the second embodiment shown in FIG. 3 in that the third embodiment includes a delay time calculation unit 150, a delay time storage unit 160, and a delay circuit 170. In this embodiment, the delay time is calculated at the same timing as when the correction table is created by the correction table creation unit 80. That is, incident light from a white screen with uniform brightness presented to the screen presentation unit 10 is detected by the photodetector 30, and this optical signal is provided to the delay time calculation unit 150. At the same time, the screen of the eye photographed by the camera 40 is processed by the image processing unit 50 and the pupil area calculation unit 60, and the pupil area data obtained by the pupil area calculation unit 60 is given to the delay time calculation unit 150.
The delay time calculation unit 150 calculates a delay time from the time when the optical signal is input to the time when the pupil area data is input, and the delay time obtained by this calculation is used as a delay time used by the delay circuit 170, Delay time storage unit 1
60.

Next, in the measurement processing of the pupil area change due to mental factors, the optical signal from the photodetector 30 is
0, the transmission of the optical signal to the correction table storage unit 90 is delayed by the delay time stored in the delay time storage unit 160. During this time, the pupil of the subject changes, the pupil area after the change is calculated by the pupil area calculation unit 60, and stored in the pupil area storage unit 70. Therefore, the timing at which the correction data and the pupil area data are provided to the calculation unit 100 coincides with each other. For this reason, the correction processing of the pupil area in the calculation unit 100 is correctly performed, and the pupil area excluding the influence of the luminance change of the presentation screen can be extracted with high accuracy.

By the way, the human observes the presentation screen while moving his / her gaze to various parts of the presentation screen and watching the intriguing parts. The brightness of the screen differs depending on the screen content at the position of the line of sight, and the pupil changes due to the brightness. FIG. 6 shows the configuration of a pupil area measuring apparatus according to the fourth embodiment, which can measure a spatial pupil area in consideration of such a movement of the line of sight. This fourth embodiment is shown in FIG.
The difference from the second embodiment shown in FIG.
80, 190, pinhole plate 200, pupil center of gravity calculation unit 2
10, center-of-gravity shift calculation unit 220, and galvano controller 23
0 is provided.

When the subject's eyes face straight ahead,
A pinhole plate 200 is placed in front of the photodetector 30 so that light at the center of the screen presentation unit 10 (approximately 3 degrees as viewed) enters the photodetector 30. Limit the incident area. When the eyeball moves and the line of sight moves, the pupil of the eye captured by the camera 40 also moves. The captured image is processed by the image processing unit 50 to extract the contour of the pupil. The extracted contour of the pupil is provided to the pupil center-of-gravity calculation unit 210, and the center of gravity of the pupil is calculated. Information on the center of gravity of the pupil calculated by the pupil center-of-gravity calculator 210 is provided to the center-of-gravity shift calculator 220 in real time, and the amount of movement of the center of gravity of the pupil is calculated. The amount of movement of the center of gravity of the pupil calculated in this way is provided to the galvano controller 230, and the galvano controller 230 controls the angles of the galvanometer mirrors 180 and 190 according to the amount of movement of the center of gravity. The movement of the galvanometer mirrors 180 and 190 allows the photodetector 30 to always measure the amount of light from the intersection between the screen presentation unit 10 and the line of sight. Subsequent processing is the same as in the first embodiment. Then, the pupil area for the true brightness in the gaze direction is obtained, and the pupil area due to mental action in the gaze direction can be measured.

[0028]

According to the pupil area measuring apparatus of the present invention, it is possible to measure the pupil area of the subject from which the factors of the luminance change have been removed by using a presentation screen whose luminance changes with time. That is, the pupil area measured by this device is
It changes only by the interest in the presentation screen and the mental effect of the interest. Therefore, an objective evaluation of a video such as a television program can be obtained in real time.

Conventionally, since the presentation screen changes depending on the surrounding brightness, it is necessary to perform measurement in a room or a dark room where the brightness is kept constant. However, according to the present invention, measurement can be performed even in an environment where ambient brightness changes, such as outdoors.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a pupil area measuring device according to a first embodiment.

FIG. 2 is a diagram illustrating an example of a correction curve of a pupil area with respect to luminance.

FIG. 3 is a block diagram illustrating a configuration of a pupil area measuring device according to a second embodiment.

FIG. 4 is a diagram showing a relationship between temporal changes in pupil area with respect to light stimulation.

FIG. 5 is a block diagram illustrating a configuration of a pupil area measuring device according to a third embodiment.

FIG. 6 is a block diagram showing a configuration of a pupil area measuring device according to a fourth embodiment.

FIG. 7 is a perspective view showing a measuring method using a conventional pupil area measuring device.

FIG. 8 is a block diagram showing a configuration of a conventional pupil area measuring device.

[Explanation of symbols]

10: Screen presenter, 20, 130: Half mirror, 30
... photodetector, 40 ... camera, 50 ... image processing unit, 60 ...
Pupil area calculation unit, 70: pupil area storage unit, 80: correction table creation unit, 90: correction table storage unit, 100: calculation unit, 110: display unit, 120: filter, 140: auxiliary illumination light source, 150: delay time Calculation unit, 160: delay time storage unit, 170: delay circuit, 180, 190: galvanometer mirror, 200: pinhole plate, 210: pupil center of gravity calculation unit, 220: center of gravity movement calculation unit, 230: galvano controller.

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-5-127 (JP, A) JP-A 5-285113 (JP, A) JP-A-2-65836 (JP, A) JP-A-2- 116348 (JP, A) JP-A-5-145830 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G01B 11/28 A61B 3/14

Claims (4)

(57) [Claims] A screen presenting unit that presents a predetermined screen; a photographing unit that photographs an image of an eye of a subject who has observed the screen presented by the screen presenting unit; and an image photographed by the photographing unit. An image processing unit that analyzes the pupil outline by analyzing the pupil outline; a pupil area calculation unit that calculates the pupil area from the pupil outline extracted by the image processing unit; and a luminance change of a screen presented to the screen presentation unit. And a correction table storing correction data for removing an influence of a change in the area of the pupil of the subject who has observed the screen due to a change in luminance of the screen presented on the screen presentation section, A pupil area correction unit that reads correction data corresponding to the luminance change of the screen detected by the light detection unit from the correction table, and corrects the pupil area calculated by the pupil area calculation unit using the correction data. Be prepared And a pupil area measuring device. 2. The pupil area measuring device according to claim 1, wherein the correction table stores correction data of the individual subject measured in advance. 3. A timing for reading correction data from the correction table by delaying a time until information of a luminance change of a screen detected by the light detection unit reaches the pupil area correction unit, and calculating the pupil area. The pupil area measuring device according to claim 1, further comprising a delay processing unit configured to match a timing at which information on the pupil area calculated by the unit reaches the pupil area correction unit. 4. The apparatus according to claim 3, wherein the delay processing section delays a time required for information on a change in screen luminance to reach the pupil area correction section with a delay time which differs for each subject. The pupil area measuring device according to the above.