JP2011224213A - Head mounting type pupil detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a head mounting type pupil detecting device, small-sized, used even in a mobile environment, and securing an external visual field.SOLUTION: This head mounting type pupil detecting device includes: a light receiving part (12) disposed in a position away from the front of the visual field and configured to receive the reflected light from an eyeball and generate an eyeball image; an optical system (11) for guiding the eyeball image onto the light receiving part; and a pupil detecting means (13, 14) for detecting a pupil from the eyeball image, wherein the visual field of the light receiving part to the eyeball is limited to be smaller than a movable range of the pupil depending on the movement of the eyeball at least in one direction.

Description

  The present invention relates to a head-mounted pupil detection device that can be mounted on a head and detects a pupil of a user.

  2. Description of the Related Art Conventionally, an image observation apparatus that is incorporated in a head-mounted display device or the like and detects an observer's line of sight to adjust an image observation state is known (see, for example, Patent Document 1). There has also been proposed a small gaze detection device considering use in a mobile environment (see, for example, Patent Document 2).

Japanese Patent No. 3683934 JP 2002-143094 A

  However, since the image observation apparatus described in Patent Document 1 is large, there is a problem that it is not suitable for a mobile environment, cannot secure an external field of view, and cannot be used. In addition, the gaze detection device described in Patent Document 2 is a hindrance in order to shoot an eyeball in a wide range, so that the camera must be arranged at a certain distance in front of the eyes. The field of view was obstructed. For this reason, it is used for measurement experiments and the like, but there is a problem that it is not suitable for general users to wear every day.

  In order to solve the above problems, an object of the present invention is to provide a head-mounted pupil detection device that is small in size and can be used in a mobile environment, and has an external field of view.

  In order to solve the above-described problems, the head-mounted pupil detection device according to the present invention arranges a light receiving unit at a position avoiding the field of view in front of the eyeball and guides reflected light from the eyeball by the optical system. At that time, the optical system is downsized by limiting the light receiving field range of the light receiving unit. Furthermore, the width of the reflection part included in a part of the optical system and arranged in front of the field of view is made smaller than the movable range of the pupil due to the movement of the eyeball.

  That is, the head-mounted pupil detection device according to the present invention is a head-mounted pupil detection device that can be mounted on the head and detects the user's pupil, and is disposed at a position avoiding the front of the field of view. A light receiving unit that receives reflected light from the eyeball to generate an eyeball image; an optical system that guides the reflected light from the eyeball onto the light receiving unit; and a pupil detection unit that detects a pupil from the eyeball image; The light receiving field for the eyeball of the light receiving unit is limited to be smaller than the movable range of the pupil due to the movement of the eyeball in at least one direction.

  According to the head-mounted pupil detection device, the light receiving unit is disposed at a position avoiding the front of the field of view, so that the visual field in front of the user's field of view is not blocked. In addition, since the optical system can be reduced in size and thickness by limiting the field of view of the light receiving unit to the eyeball, the pupil can be detected without blocking the user's field of view as much as possible.

  Further, in the head mounted pupil detection device according to the present invention, the optical system includes at least one reflection unit that is disposed in a visual field in front of the eyeball and reflects reflected light from the eyeball toward the light receiving unit. And the width of at least one of the reflecting portions is smaller than 4 mm, which is an average pupil diameter of a person.

  According to the head-mounted pupil detection device, the optical system arranged in front of the eyes is miniaturized so that the appearance of the device is not noticeable, and the reflection unit arranged in front of the eyes By setting the diameter to be equal to or smaller than the pupil diameter, the outside world viewed from the wearer can be visually recognized as see-through, so that the wearer can also use the device without worrying about the presence of the device.

  The head-mounted pupil detection device according to the present invention further includes a display unit that displays an image and a display control unit that controls an image to be displayed on the display unit, and the display unit transmits the image light to the display unit. An optical system guides the eyeball to form a virtual image display screen in which the video is enlarged.

  According to this head-mounted pupil detection device, by mounting the display means integrally, it is possible to observe a virtual image image using the same optical system, and to detect the state of the pupil during image observation. Can do.

  In the head mounted pupil detection device according to the present invention, the light receiving field of the light receiving unit is smaller than the movable range of the pupil due to the movement of the eyeball when observing the display screen in at least one direction. And

  According to the head-mounted pupil detection device, the optical system can be miniaturized by limiting the light receiving field so that it is smaller than the movable range of the pupil when the user observes the display screen.

  In the head mounted pupil detection device according to the present invention, the light receiving field of the light receiving unit is larger than a movable range of the pupil center due to the movement of the eyeball when observing the display screen.

  According to the head-mounted pupil detection device, it is possible to reduce the size while securing a necessary minimum visual field range.

  In the head-mounted pupil detection apparatus according to the present invention, the pupil detection means detects at least one of a pupil position, a pupil center position, or a pupil diameter from the eyeball image.

  According to the head mounted pupil detection device, the control of the display unit can be switched according to the state of the pupil.

  Further, in the head-mounted pupil detection device according to the present invention, the display control unit identifies a line-of-sight position on the display screen corresponding to the detected pupil position or pupil center position, and based on the identification result The display unit is controlled.

  According to the head mounted pupil detection device, it is possible to select a menu item displayed on the display screen only by moving the line of sight (pupil) and display the selected menu contents.

  Further, in the head mounted pupil detection device according to the present invention, the display control unit identifies the presence or absence of a line of sight on the display screen from the detected pupil position or pupil center position, and based on the identification result, The display unit is controlled.

  According to this head-mounted pupil detection device, since it is an optical system that does not block the field of view, it is possible to detect the presence of a line-of-sight position in the outside world by detecting the pupil position.

  Further, in the head mounted pupil detection device according to the present invention, the pupil detection means determines whether or not the detected pupil center position is located within a light receiving field of the light receiving unit, and the display control unit includes: When the pupil center position is located within the light receiving field of the light receiving unit, an image is displayed on the display unit, and when the pupil center position is not located within the light receiving field of the light receiving unit, the display is performed. The video is not displayed on the screen.

  According to the head-mounted pupil detection device, the pupil position can be detected by simple processing, and the display control can be switched by determining the presence or absence of the line of sight in the display screen.

  Further, in the head-mounted pupil detection device according to the present invention, it is determined whether or not the area of the pupil in the light receiving field of the light receiving unit is equal to or greater than a predetermined value, and the display control unit includes the light receiving unit When the area of the pupil in the light receiving field is equal to or greater than a predetermined value, an image is displayed on the display unit, and when the area of the pupil in the light receiving field of the light receiving unit is smaller than the predetermined value The video image is not displayed on the display unit.

  According to the head-mounted pupil detection device, the pupil position can be detected by simple processing, and the display control can be switched by determining the presence or absence of the line of sight in the display screen.

  Further, in the head mounted pupil detection device according to the present invention, the pupil detection means determines whether or not the light reception intensity of the light receiving unit is equal to or lower than a predetermined value, and the display control unit is configured to receive the light receiving unit. When the received light intensity is less than or equal to a predetermined value, an image is displayed on the display unit, and when the received light intensity of the light receiving unit is greater than a predetermined value, no image is displayed on the display unit. And

  According to the head-mounted pupil detection device, the pupil position can be detected by simple processing, and the display control can be switched by determining the presence or absence of the line of sight in the display screen.

  Further, in the head-mounted pupil detection device according to the present invention, when the pupil detection unit lacks a part of the pupil of the eyeball image, the part of the pupil lacking the part is not missing. The curvature is calculated from the contour of the lens, and at least one of the pupil position, the pupil center position, or the pupil diameter is detected.

  According to the head-mounted pupil detection device, the pupil position, the pupil center position, and the pupil diameter can be detected even when the received eyeball image lacks a part of the pupil.

  Further, in the head mounted pupil detection device according to the present invention, the optical system is configured so that the center of the pupil does not enter the light receiving field of the light receiving unit when looking in front of the field of view. It is characterized by.

  According to the head-mounted pupil detection device, the outside view can be secured without blocking the front view, and the center of the pupil is not detected when looking at the outside world. In this case, processing other than pupil detection is stopped. Power consumption can be reduced by reducing the driving frequency.

  As described above, according to the present invention, it is possible to provide a head-mounted pupil detection device that is small in size and can be used in a mobile environment and secures an external field of view.

1 is an external view of a head-mounted pupil detection device according to a first embodiment of the present invention. It is a functional block diagram of the head mounted pupil detection device of a 1st embodiment by the present invention. It is a figure which shows the optical system of the head mounted pupil detection apparatus of 1st Embodiment by this invention. It is a figure which shows the light-receiving visual field of the light-receiving part of the head mounted pupil detection apparatus of 1st Embodiment by this invention. It is a figure which shows the see-through effect of the head mounted pupil detection apparatus of 1st Embodiment by this invention. It is an external view of the head mounted pupil detection apparatus of 2nd Embodiment by this invention. It is a figure which shows the optical system of the head mounted pupil detection apparatus of 2nd Embodiment by this invention. It is a figure which shows the light-receiving visual field of the light-receiving part of the head mounted pupil detection apparatus of 2nd Embodiment by this invention. It is a functional block diagram of the head mounted pupil detection device of a 3rd embodiment by the present invention. It is a figure which shows the optical system of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a figure which shows the light reception visual field of the light-receiving part of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a figure which shows the visual field position detection on the display screen of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a flowchart which shows the operation | movement of menu operation of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a flowchart which shows the operation | movement of the eyes | visual_axis presence determination by the pupil center detection of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a figure explaining the operation | movement of the eyes | visual_axis presence determination by the pupil center detection of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a flowchart which shows the operation | movement of the eyes | visual_axis presence determination by the pupil area of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a figure explaining the operation | movement of the eyes | visual_axis presence determination by the pupil area of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a flowchart which shows the operation | movement of the eyes | visual_axis presence determination by the light reception intensity | strength of the head mounted pupil detection apparatus of 3rd Embodiment by this invention. It is a figure which shows the optical system of the head mounted pupil detection apparatus of 4th Embodiment by this invention. It is a figure which shows the light reception visual field of the light-receiving part of the head mounted pupil detection apparatus of 4th Embodiment by this invention. It is a figure explaining arrangement | positioning of the head mounted pupil detection apparatus of 4th Embodiment by this invention.

  Embodiments of a head-mounted pupil detection device (hereinafter simply referred to as “pupil detection device”) according to the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is an external view of a pupil detection device according to the first embodiment. FIG. 2 is a functional block diagram of the pupil detection device of the first embodiment. The pupil detection device 1 includes a housing 15 containing an optical system 11, a light receiving unit 12, an image processing unit 13, and a pupil detection unit 14, an illumination unit 16, and a fixing unit 17.

  The illumination unit 16 is held by the housing 15 and illuminates the eyeball with invisible light (for example, infrared light). The illumination unit 16 is not an essential component, but is suitable for illuminating the eyeball and increasing the reflected light from the eyeball in order to improve the pupil detection rate.

  The optical system 11 takes in reflected light from the user's eyeball from the optical system 11 and guides it onto the light receiving unit 12.

  The light receiving unit 12 is disposed at a position avoiding the front of the user's field of view, receives reflected light from the eyeball guided by the optical system 11, generates an eyeball image, and outputs the eyeball image to the image processing unit 13. The light receiving unit 12 is, for example, a light receiving element or an imaging element such as a CCD or a CMOS.

  The image processing unit 13 performs image processing for extracting a pupil from the eyeball image acquired from the light receiving unit 12 and outputs the result to the pupil detection unit 14.

  The pupil detection unit 14 detects the pupil position and the pupil center position from the eyeball image acquired from the image processing unit 13. Then, it is determined where the pupil (pupil center) is located in the light receiving field of the light receiving unit 12 or in the light receiving field, and a pupil detection signal is output to the outside. When a part of the pupil of the eyeball image acquired from the image processing unit 13 is missing, for example, the curvature is calculated from the outline of the pupil of the missing part, and the position of the curvature center is set as the pupil center position.

  The fixing part 17 is a fixing member for enabling the housing 15 to be mounted on the user's head. For example, as shown in FIG. 1, the fixing unit 17 may be fixed to a spectacle frame and attachable to the head via the spectacle frame, or may be linearly extended to the ear. It may be fixed. That is, what is necessary is just to be able to fix the housing 15 to the head. When glasses are used, the optical system 11 is disposed outside the glasses lens.

  3 is a diagram showing an optical system of the pupil detection device 1, FIG. 3 (a) shows a front view of the pupil detection device 1, and FIG. 3 (b) shows a top view of the pupil detection device 1 and reflection from the eyeball. Shows the optical path of light. The optical system 11 includes an eyepiece lens 111, a reflection unit 112, and an imaging lens 113. Reflected light from the eyeball is incident on the reflecting portion 112 through the eyepiece 111 in front of the optical system 11. The reflection unit 112 reflects incident light and is received by the light receiving unit 12 through the imaging lens 113.

  FIG. 4 is a view showing a light receiving field of the light receiving unit, FIG. 4 (a) shows a light receiving field of the light receiving unit in the conventional pupil detecting device, and FIG. 4 (b) is a light receiving in the pupil detecting device 1 of the present embodiment. The light reception visual field of the part 12 is shown. In the conventional pupil detection device, in order to detect the position of the pupil, an area covering the entire movable range of the pupil due to the movement of the eyeball can be photographed, that is, the light receiving field B of the light receiving unit includes the movable range A of the pupil. The light reception viewing angle is set as follows. However, for that purpose, it is necessary to arrange a large eyepiece optical system in front of the eyes, and it is necessary to arrange the camera at a position away from the face (position that blocks the field of view) in order to secure a viewing angle. It was not suitable for daily mobile environments. On the other hand, in the pupil detection device 1 of the present embodiment, the vertical widths of the eyepiece lens 111 and the reflection unit 112 are narrow, so that the vertical direction of the light reception field B of the light reception unit 12 is shown in FIG. The visual field in the direction is narrower than the pupil movable range A.

  FIG. 5 is a diagram for explaining the see-through effect of the pupil detection device 1. FIG. 5A shows a state of shielding the external environment when the conventional pupil detection device 5 in which the width of the eyepiece optical system is larger than that of the pupil is used. In this case, the external light in the direction covered by the eyepiece optical system cannot enter the eyeball, resulting in a blind spot without superimposing the background on the image.

  FIG. 5B shows a state of shielding the outside when the pupil detection device 1 is used. When viewed from the front direction of the eyeball, the pupil detection device 1 is such that the vertical widths of the eyepiece lens 111 and the reflection unit 112 are smaller than the average pupil diameter of a person. Therefore, since external light in the front direction can reach the retina through the pupil, the scenery behind the optical system can also be superimposed and seen (see-through). The vertical width of the eyepiece 111 and the reflector 112 of the pupil detection device 1 is preferably about 4 mm or less in order to secure an external field of view because the average pupil diameter of a person is about 4 mm.

  Therefore, according to the pupil detection device 1 of the present embodiment, it is possible to make the optical system small by limiting the visual field range in the vertical direction, and it is possible to place the optical system close to the eyeball. It is not troublesome to wear, and the appearance can be made inconspicuous. Further, a see-through effect can be obtained in the vertical direction.

(Second Embodiment)
Next, a pupil detection device according to a second embodiment of the present invention will be described with reference to the drawings. In addition, the same reference number is attached | subjected to the same component as 1st Embodiment, and description is abbreviate | omitted.

  FIG. 6 is an external view of the pupil detection device 2 of the second embodiment. The pupil detection device 2 includes an optical system 11 (excluding the reflection unit 18), a light receiving unit 12, an image processing unit 13, and a housing 15 including a pupil detection unit 14, an illumination unit 16, a fixing unit 17, and a reflection unit. 18. The functional block diagram of the pupil detection device 2 is the same as the functional block diagram (FIG. 2) of the pupil detection device 1 of the first embodiment.

  The reflector 18 is provided in the spectacle lens. The reflection unit 18 is, for example, a plane mirror, and reflects the reflected light from the eyeball illuminated by the illumination unit 16 so as to enter the housing 15.

  The casing 15 is attached to the temple part of the glasses via the fixing part 17.

  FIG. 7 is a diagram illustrating an optical system of the pupil detection device 2, and FIG. 7A is a front view of the pupil detection device 2. However, when the pupil detection device 2 is mounted, the casing 15 is disposed at a position that does not block the user's front view, so that only the reflecting portion 18 is located in the front view. FIG. 7B shows a top view of the pupil detection device 2 and an optical path of light reflected from the eyeball. The optical system 11 includes a reflection unit 18 and an imaging lens 113. The reflected light from the eyeball is reflected by the reflecting unit 18 and is received by the light receiving unit 12 through the imaging lens 113.

  FIG. 8 is a diagram illustrating a light receiving field of the light receiving unit 12 of the pupil detection device 2. As shown in FIG. 7, since the horizontal width of the reflecting portion 18 is narrow, the horizontal visual field of the light receiving field B of the light receiving unit 12 is narrower than the pupil movable range A.

  Therefore, according to the pupil detection device 2 of the present embodiment, by limiting the horizontal visual field range, the optical system can be made compact, and it is not troublesome to wear, and the appearance is inconspicuous. be able to. Further, a see-through effect can be obtained in the horizontal direction.

(Third embodiment)
Next, a pupil detection device according to a third embodiment of the present invention will be described with reference to the drawings. In addition, the same reference number is attached | subjected to the same component as 1st Embodiment, and description is abbreviate | omitted.

  The external view of the pupil detection device 3 of the third embodiment is the same as the external view (FIG. 1) of the pupil detection device 1 of the first embodiment. FIG. 9 is a functional block diagram of the pupil detection device 3. The pupil detection device 1 includes an optical system 11, a light receiving unit 12, an image processing unit 13, a pupil detection unit 14, a display unit 19, a display control unit 20, a housing 15, an illumination unit 16, and a fixing unit 17. Prepare. Compared to the pupil detection device 1 of the first embodiment, the display device 19 and the display control unit 20 are further provided.

  The display control unit 20 has a drive circuit for the display unit 19 and outputs a video signal to the display unit 19. Further, the display control unit 20 controls the display unit based on the pupil detection signal acquired from the pupil detection unit 14.

  The display unit 19 displays a video based on the video signal input from the display control unit 20, guides the video light to the eyeball by the optical system 11, and forms a virtual image display screen in which the video is enlarged.

  The pupil detection unit 14 detects the pupil position and the pupil center position from the eyeball image acquired from the image processing unit 13. Then, it is determined where the pupil (pupil center) is located in the light receiving field of the light receiving unit 12 or whether it is located in the light receiving field, and a pupil detection signal is output to the display control unit 20. When a part of the pupil of the eyeball image acquired from the image processing unit 13 is missing, for example, the curvature is calculated from the outline of the pupil of the missing part, and the position of the curvature center is set as the pupil center position.

  The pupil detection unit 14 detects the pupil diameter from the eyeball image acquired from the image processing unit 13. When a part of the pupil of the eyeball image acquired from the image processing unit 13 is missing, for example, the curvature is calculated from the outline of the pupil that is not missing, and the curvature radius is set as the radius of the pupil.

  The pupil diameter can be used when determining the pupil position or the pupil center position. Further, the display unit 19 can be controlled by the display control unit 20 according to the pupil diameter. The pupil changes depending on the surrounding brightness environment. The pupil diameter is small in a bright environment, and the pupil diameter is large in a dark environment. For example, when the pupil diameter is small, the display unit 19 displays The brightness is controlled to be increased, and when the pupil diameter is large, the display brightness of the display unit 19 is controlled to be decreased. In particular, in the see-through pupil detection device 3 in which external light also passes through the pupil, when the surroundings are bright, the display brightness is brightened so that it can be observed without being defeated by ambient ambient light, and the surroundings are dark. In this case, it is possible to observe without feeling dazzling by reducing the display brightness.

  FIG. 10 is a diagram illustrating an optical system of the pupil detection device 3. The optical system 11 includes an eyepiece lens 111, a reflection unit 112, an imaging lens 113, and a beam splitter 114.

  As shown in FIG. 10A, the image light emitted from the display unit 19 passes through the beam splitter 114 and is guided to the eyeball by the optical system 11. On the other hand, as shown in FIG. 10B, the reflected light from the eyeball is guided by the optical system 11 and reflected by the beam splitter 114 to form an eyeball image on the light receiving unit 12. This makes it possible to detect the pupil while observing the video. The beam splitter 114 may be a half mirror, or may reflect only infrared light used as invisible light for eyeball illumination, or may reflect only specific polarized light.

  FIG. 11A is a diagram showing the movable range of the pupil due to the movement of the eyeball when the virtual image display screen is observed. Point a in the figure indicates the pupil center when viewing the upper left corner of the display screen, point b indicates the pupil center when viewing the upper right corner of the display screen, and point c looks at the lower right corner of the display screen. The point d indicates the pupil center when viewing the lower left corner of the display screen, and the point e indicates the pupil center when viewing the center of the display screen. Therefore, the pupil center movable range D when observing the display screen is indicated by a region connecting the points a to e. When the angle of view of the display image is about 10 degrees in the horizontal direction, the pupil position moves by about ± 1 mm, so that the points a and b, points b and c, points c and d, points d and a are Each distance is about 2 mm. Region C shows the movable range of the pupil during display screen observation.

  FIG. 11B is a diagram illustrating a light receiving field of the light receiving unit 12 of the pupil detection device 3. The light receiving field E includes the pupil center movable range D when observing the display screen, and is set so that the vertical width of the light receiving field E is smaller than the vertical width of the pupil movable range C when observing the display screen. Is done. By making the vertical width of the light receiving field E equal to or smaller than the pupil diameter, the field of view necessary for detecting the pupil can be covered and the field of view can be minimized. For this reason, the vertical dimension of the optical system 11 can be reduced as much as possible to the pupil diameter or less, and an optical system having a see-through can be obtained.

  FIG. 12 is a diagram illustrating the detection of the line-of-sight position on the display screen. As shown in FIG. 12A, by detecting the pupil center located within the pupil center movable range D during display screen observation, the line-of-sight position in the display screen corresponding to the position can be detected. FIG. 12B is a diagram showing an example of a menu screen, and the circles in the figure indicate menu items. When the display screen is a menu screen as shown in FIG. 12B, the menu corresponding to the pupil center position is detected by detecting the pupil center and identifying the line-of-sight position on the display screen corresponding to the detected pupil center position. The contents of the item can be displayed. Therefore, the menu item displayed on the display screen can be selected only by moving the line of sight (pupil), and the selected menu contents can be displayed.

  FIG. 13 is a flowchart showing the menu operation of the head-mounted pupil detection device 3. The head-mounted pupil detection device 3 guides the image light of the menu screen generated by the display unit 19 and the display control unit 20 to the eyeball by the optical system 11 (step S101). Next, the reflected light from the eyeball irradiated by the illumination unit 16 is guided to the light receiving unit 12 by the optical system 11 to obtain an eyeball image (step S102). Then, the pupil is extracted by the image processing unit 13 (step S103), and the pupil center position is detected by the pupil detection unit 14 (step S104). Further, the display control unit 20 determines which menu position on the menu screen the pupil center position corresponds to, and displays the corresponding menu item on the display unit 19 (step S105).

  Next, determination of the presence or absence of the line of sight on the display screen by the pupil detection device 3 will be described below with reference to the drawings.

  Assuming that the pupil detection device 3 is always worn in a mobile environment, when the user is not looking at the display screen displayed by the display unit 19 (when facing the front), the pupil center is located at the light receiving unit 12. It is preferable to arrange the optical system 11 at a position where it is not detected. Thereby, an external field view is ensured and can be used even in a mobile environment. Furthermore, by disposing the optical system 11 at such a position, the pupil detection device 3 can determine whether the user is looking at the display screen or not, so that the pupil center is not detected. When (when facing the front), it is possible to perform power saving processing such as stopping the processing of the display unit 19 and the display control unit 20 or lowering the drive frequency.

  FIG. 14 is a first example of the gaze presence determination on the display screen, and is a flowchart showing the gaze presence determination operation based on pupil center detection. The head-mounted pupil detection device 3 acquires an eyeball image by the illumination unit 16, the optical system 11, and the light receiving unit 12 (step S201), and extracts a pupil by the image processing unit 13 (step S202). Then, the pupil detection unit 14 detects the pupil center position (step S203), and determines whether or not the pupil center is located within the light receiving field E of the light receiving unit 12 (step S204). If there is a pupil center in the light receiving field E, the display control unit 20 determines that the display screen is being viewed (step S205), and if there is no pupil center in the light receiving field E, the display screen is viewed. It is determined that it is not present (step S206). The display control unit 20 switches the control of the display unit 19 based on the determination result of whether or not the display screen is being viewed. For example, the display image is displayed when viewing the display screen, and the display image display is stopped when the display image is not viewed.

  That is, as illustrated in FIG. 15A, the display control unit 20 determines that the display screen is viewed when the center of the pupil is located within the light receiving field E, and is illustrated in FIG. Thus, when the center of the pupil is not located within the light receiving field E, it is determined that the display screen is not being viewed. Since the pupil detection device 3 has a limited field of view, when the pupil center is in the light reception field E, it can be regarded as viewing the display screen. Thereby, the presence or absence of the line of sight on the display screen can be determined by a simple processing means.

  FIG. 16 is a flow chart showing a second example of the gaze presence / absence determination on the display screen, and the operation of gaze presence / absence judgment by pupil area detection. The head-mounted pupil detection device 3 acquires an eyeball image using the illumination unit 16, the optical system 11, and the light receiving unit 12 (step S301), and extracts a pupil using the image processing unit 13 (step S302). Then, the pupil detection unit 14 calculates the area of the pupil in the light receiving field E (step S303), and determines whether or not the calculated pupil area is equal to or larger than the threshold (step S304). If it is equal to or greater than the threshold, the display control unit 20 determines that the display screen is being viewed (step S305), and if it is smaller than the threshold, it is determined that the display screen is not being viewed (step S306). Based on the determination result, the control of the display unit 19 is switched.

  That is, as shown in FIG. 17A, when the area of the pupil in the light receiving field E is equal to or larger than the threshold, it is determined that the display screen is being viewed, and as shown in FIG. When the area of the inner pupil is smaller than the threshold value, it is determined that the display screen is not viewed. Since the pupil detection device 3 has a limited field of view, it can be regarded as viewing the display screen when a certain pupil or more is included in the light receiving field. Thereby, the presence or absence of the line of sight on the display screen can be determined by a simple processing means.

  FIG. 18 is a flowchart showing a third example of the determination of the presence / absence of the line of sight on the display screen, and shows the operation of determining the presence / absence of the line of sight based on the received light intensity. The head-mounted pupil detection device 3 acquires an eyeball image by the illumination unit 16, the optical system 11, and the light receiving unit 12 (step 401), and the reflected light from the eyeball received by the light receiving unit 12 by the pupil detection unit 14 is obtained. It is determined whether the received light intensity is greater than or equal to a threshold value (step S402). If the received light intensity of the reflected light from the eyeball is greater than or equal to the threshold value, the display control unit 20 determines that the display screen is being viewed (step S403), and the received light intensity of the reflected light from the eyeball is smaller than the threshold value. In this case, it is determined that the display screen is not viewed (step S404), and the control of the display unit 19 is switched based on the determination result.

  Since the pupil detection device 3 has a limited field of view, it can be considered that the intensity of the reflected light decreases when the pupil enters the light receiving field more than a certain level, and the display screen is viewed with the degree of decrease. Thereby, the presence or absence of the line of sight on the display screen can be determined by a simple processing means. According to the third example of the gaze presence / absence determination, there is no need to extract an image of the pupil, so there is no need to provide the image processing unit 13, and the pupil detection device 3 can be further downsized.

  According to the pupil detection device 3 of the present embodiment, a virtual image can be observed using the same optical system by mounting the display unit integrally. Further, by detecting the pupil at the time of observing the display screen, it is possible to perform menu operations and power saving processing when the display screen is not viewed.

(Fourth embodiment)
Next, a pupil detection device according to a fourth embodiment of the present invention will be described with reference to the drawings. In addition, the same reference number is attached | subjected to the same component as 2nd Embodiment, and description is abbreviate | omitted.

  The external view of the pupil detection device 4 of the fourth embodiment is the same as the external view (FIG. 6) of the pupil detection device 2 of the second embodiment, and the functional block diagram of the pupil detection device 4 is the pupil detection device. 3 is the same as the functional block diagram (FIG. 9). Compared to the pupil detection device 2 of the second embodiment, the display device 19 and the display control unit 20 are further provided.

  FIG. 19 is a diagram illustrating an optical system of the pupil detection device 4 according to the fourth embodiment. The optical system 11 includes a reflection unit 18, an imaging lens 113, and a beam splitter 114. As shown in FIG. 19A, the image light emitted from the display unit 19 passes through the beam splitter 114 and is guided to the eyeball by the optical system 11. On the other hand, as shown in FIG. 19B, the reflected light from the eyeball is guided by the optical system 11 and reflected by the beam splitter 114 to form an eyeball image on the light receiving unit 12.

  FIG. 20 is a diagram illustrating a light receiving field of the light receiving unit 12 of the pupil detection device 4. The light receiving field E includes the pupil center movable range D when observing the display screen, and the horizontal width of the light receiving field E of the light receiving unit is smaller than the horizontal width of the pupil movable range C when observing the display screen. Is set as follows. By setting the horizontal width of the light receiving field E of the light receiving unit to a field having an average pupil diameter or less, it is possible to suppress the field of view to a minimum while covering the field range necessary for detecting the pupil. For this reason, the horizontal width of the optical system 11 can be reduced as much as possible to the pupil diameter or less, and an optical system having a see-through can be obtained. This makes it possible to detect the pupil while observing the video.

  Assuming that the pupil detection device 4 is always worn in a mobile environment, when the user is not looking at the display screen displayed by the display unit 19 (when facing the front), the pupil center is located at the light receiving unit 12. It is preferable to arrange the optical system 11 at a position where it is not detected. For example, as shown in FIG. 21A, the reflector 18 is arranged at a position slightly shifted from the front of the pupil so that the optical system does not block the front of the field of view when the user is facing the front. When arranged in this manner, when facing the front, the center of the pupil is positioned away from the light receiving field E as shown in FIG.

  According to the pupil detection device 4 of the present embodiment, a virtual image can be observed using the same optical system by mounting the display unit integrally. Further, by detecting the pupil at the time of observing the display screen, it is possible to perform menu operations and power saving processing when the display screen is not viewed.

  Each of the above embodiments has been described as a representative example, but it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims.

  As described above, according to the present invention, a small-sized pupil detection device having a see-through effect can be provided, which is useful for any application for detecting a pupil.

1, 2, 3, 4 Pupil detection device 11 Optical system 12 Light receiving unit 13 Image processing unit 14 Pupil detection unit 15 Housing 16 Illumination unit 17 Fixing unit 19 Display unit 20 Display control unit 111 Eyepiece lens 18, 112 Reflecting unit 113 Connection Image lens 114 Beam splitter

Claims (13)

A head-mounted pupil detection device that can be mounted on the head and detects a user's pupil,
A light receiving unit that is arranged at a position that avoids the front of the field of view and receives reflected light from the eyeball to generate an eyeball image;
An optical system for guiding the reflected light from the eyeball onto the light receiving unit;
Pupil detection means for detecting a pupil from the eyeball image,
The head-mounted pupil detection device characterized in that a light receiving field of the light receiving unit with respect to the eyeball is limited to be smaller than a movable range of the pupil due to movement of the eyeball in at least one direction. The optical system includes at least one reflection unit that is disposed in a visual field in front of the eyeball and reflects reflected light from the eyeball toward the light receiving unit;
The head-mounted pupil detection device according to claim 1, wherein the width of at least one of the reflection parts is smaller than 4 mm, which is an average pupil diameter of a person. A display unit for displaying images;
A display control unit for controlling an image to be displayed on the display unit,
3. The head mounted pupil detection according to claim 1, wherein the display unit guides image light to an eyeball by the optical system to form a display screen of a virtual image obtained by enlarging the image. apparatus.   4. The head mounted pupil detection according to claim 3, wherein a light receiving field of the light receiving unit is smaller than a movable range of the pupil due to movement of an eyeball when observing the display screen in at least one direction. apparatus.   5. The head mounted pupil detection device according to claim 4, wherein a light receiving field of the light receiving unit is larger than a movable range of a pupil center due to movement of an eyeball when observing the display screen.   The head mounted according to any one of claims 1 to 5, wherein the pupil detecting means detects at least one of a pupil position, a pupil center position, or a pupil diameter from the eyeball image. Type pupil detection device.   The display control unit identifies a line-of-sight position on the display screen corresponding to the detected pupil position or pupil center position, and controls the display unit based on the identification result. The head-mounted pupil detection device described in 1.   The display control unit identifies presence or absence of a line of sight on the display screen from the detected pupil position or pupil center position, and controls the display unit based on the identification result. The head-mounted pupil detection device as described. The pupil detection means determines whether or not the detected pupil center position is located within a light receiving field of the light receiving unit;
The display control unit displays an image on the display unit when the pupil center position is within the light receiving field of the light receiving unit, and the pupil center position is not positioned within the light receiving field of the light receiving unit. 9. The head-mounted pupil detection apparatus according to claim 8, wherein no image is displayed on the display unit. The pupil detection means determines whether or not the area of the pupil in the light receiving field of the light receiving unit is a predetermined value or more,
The display control unit displays an image on the display unit when the area of the pupil in the light receiving field of the light receiving unit is equal to or larger than a predetermined value, and the area of the pupil in the light receiving field of the light receiving unit 9. The head mounted pupil detection device according to claim 8, wherein when the value is smaller than a predetermined value, no image is displayed on the display unit. The pupil detection means determines whether or not the light receiving intensity of the light receiving unit is a predetermined value or less,
The display control unit displays an image on the display unit when the light receiving intensity of the light receiving unit is equal to or lower than a predetermined value, and displays the image when the light receiving intensity of the light receiving unit is higher than a predetermined value. The head-mounted pupil detection device according to claim 8, wherein no image is displayed on the screen.   When a part of the pupil of the eyeball image is missing, the pupil detection means calculates a curvature from the outline of the pupil lacking the part, and the pupil position, the pupil center position, or the pupil diameter The head-mounted pupil detection device according to claim 6, wherein at least one of the following is detected.   The optical system is configured such that the center of the pupil does not enter the light receiving field of the light receiving unit when looking in front of the field of view. The head-mounted pupil detection device described in 1.

Images