JP6241107B2 - Gaze direction detection device and gaze direction detection method - Google Patents

Description

  The present invention relates to an apparatus and method for detecting the direction of a line of sight, and more particularly to an apparatus and method for specifying a position (region) on a screen to be watched from the direction of the detected line of sight.

  There is known a digital camera equipped with a line-of-sight input AF function that automatically detects the direction of a user's line of sight and automatically sets an AF (autofocus) area within an angle of view. The direction of the line of sight can be obtained from the shape of the pupil and its position, but in addition, the feature amount indicating the face direction included in the entire face image and the direction indicating the eye direction included in the image around the eye A method of estimating the direction of the line of sight by using a quantity is also proposed (see Patent Document 1).

JP 2012-038106 A

  However, if the face is away from the camera when capturing an image for detecting the line of sight, it is difficult to estimate the direction of the line of sight from image information around the pupil and the eyes. Further, when the gaze direction is estimated from the face direction when the face distance is sufficiently close, the estimation accuracy decreases.

  An object of the present invention is to perform eye gaze detection by an optimum method in accordance with the size of a captured face image.

  The gaze direction detection device of the present invention includes a monitor that displays an image, a gaze detection imaging unit that captures the direction in which the monitor is directed, and a face that detects a face area from the first image captured by the gaze detection imaging unit. Area detection means; face area size calculation means for calculating the size of the face area; face feature point detection means for detecting facial feature quantities including eyes, nose and mouth from the first image; A first line-of-sight estimation unit that estimates a line-of-sight direction from position information of one or a plurality of feature quantities; a second line-of-sight estimation unit that detects a pupil from the first image and estimates the direction of the line of sight; According to this, the first or second line-of-sight estimation means is selected, and monitor area specifying means for specifying an area on the monitor on which the estimated line of sight gazes is provided.

  The monitor area specifying means selects the first line-of-sight estimation means when the size of the face area is smaller than the second threshold, and selects the second line-of-sight estimation means when larger than the second threshold. Further, the monitor area specifying means does not estimate the direction of the line of sight when the size of the face area is smaller than the first threshold value smaller than the second threshold value. Further, it is preferable that the monitor region specifying unit selects the second line-of-sight estimation unit and specifies the region on the monitor when both eyes are detected from the first image even when the face region is not detected.

  It is preferable that a plurality of illumination light sources be arranged around the photographing lens of the line-of-sight detection photographing means. Moreover, it is preferable to provide specific area recognition means for letting the user know the area specified by the monitor area specifying means. The specific area notifying means surrounds the specified area with a specific color frame, displays a specific mark in the area, or colors the entire area with a specific color component, or a combination thereof. Indicates that the region is specified.

  It is preferable to further include input means for instructing enlargement of the image displayed on the monitor. When the enlargement of the image is instructed, the image of the area specified by the monitor area specifying means is enlarged and displayed on the monitor, and the line of sight is detected. It is preferable that the second image is captured using the image capturing unit, and the region on the monitor is further specified by the monitor region specifying unit based on the second image. The input means includes an operation unit for fixing the area specified by the monitor area specifying means, and after fixing the area, an image of a normal size that is not enlarged is displayed on the monitor. Further, it is preferable that the line-of-sight detection photographing means is displaced integrally with the monitor with respect to the apparatus main body.

  An imaging apparatus according to the present invention is an imaging apparatus including any one of the above-described gaze direction detection means and an image photographing means different from the gaze detection imaging means, and a monitor image photographed by the image photographing means The area displayed on the monitor and specified by the monitor area specifying means is used as the AF area.

  The gaze direction detection method of the present invention includes a gaze detection image capturing step for capturing the gaze direction of the photographer, and a face area detection step for detecting a face area from the first image captured in the gaze detection image capturing step, A face area size calculating step for calculating the size of the face area, a face feature point detecting step for detecting a feature quantity of a face including eyes, nose, and mouth from the first image; In accordance with the size of the face area, a first line-of-sight estimation step that estimates the direction of the line of sight from the position information of the feature amount, a second line-of-sight estimation step that detects the pupil from the first image and estimates the direction of the line of sight, A monitor region specifying step of selecting the first or second line-of-sight estimation step and specifying an area on the monitor on which the estimated line of sight gazes.

  According to the present invention, line-of-sight detection can be performed by an optimum method in accordance with the size of a face image to be shot.

It is a rear view of the digital camera of this embodiment. It is a block diagram which shows typically the structure of the digital camera of FIG. It is a figure which shows the various attitude | positions of the user in the camera imaging | photography using the through image of a monitor. It is a figure which illustrates a mode that the 1st step AF area was selected in the through image. It is a figure which illustrates a mode that the AF area of the 1st step was expanded and the AF area of the 2nd step was selected. It is a flowchart of AF area setting processing of the present embodiment. It is a flowchart of the first half of the gaze detection process of this embodiment. It is a flowchart of the latter half part of the gaze detection process of this embodiment. It is a figure which shows the condition of the image for gaze detection when the whole user's face is image | photographed by the distance away from the camera for gaze detection. It is a figure which shows the condition of the image for gaze detection when the whole user's face is image | photographed at a distance closer than FIG. It is a figure which shows the condition of the image for eyes | visual_axis detection when a part of face including eyes, a nose, and a mouth is image | photographed at near distance. It is a figure which shows the condition of the image for eye-gaze detection when a part of face including only eyes and a nose is image | photographed at the very near distance. It is a flowchart of the gaze direction estimation process using the image of a pupil. It is a figure explaining the principle of the method of estimating a gaze direction from a pupil shape. It is a flowchart of the gaze detection process in a modification.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a rear view of a digital camera equipped with a line-of-sight direction detection apparatus according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a configuration of the digital camera of FIG.

  The digital camera 10 is a camera equipped with a line-of-sight input AF function that automatically detects a user's line of sight and automatically selects an AF (autofocus) area. As shown in FIG. 1, an image display monitor 11 such as an LCD is provided on the back of the digital camera 10 to display a through image, a captured image, a mode setting screen, a parameter setting screen, and the like.

  In the center of the upper side of the monitor 11, a fixed focus type (pan focus) line-of-sight detection camera 12 is arranged. Illumination light sources 13 </ b> A and 13 </ b> B (line-of-sight detection light source unit 13) are arranged on both the left and right sides of the line-of-sight detection camera 12 at a predetermined distance. In this embodiment, an infrared camera is used for the line-of-sight detection camera 12, and infrared LEDs are used for the illumination light sources 13 and 14. That is, the face of the user who views the monitor 11 is irradiated with infrared rays from the light sources 13 and 14 and photographed by the line-of-sight detection camera 12.

  Further, a switch 16 is disposed at a position corresponding to a release button (release switch) 15 disposed on the top surface on the back surface of the digital camera 10. In the present embodiment, the switch 16 is a dial switch, and an input function key 17 such as a four-way key or an OK button is disposed below the switch 16. Input signals from the switch group 14 including the release button 15, dial switch, function key 17 and the like are input to the controller 18, and various input and predetermined processes are executed.

  An imaging unit 19 for taking a photograph is provided on the front side of the camera body, and a through image captured by the imaging unit 19 is temporarily held in the image memory 20 through the controller 18 and is output to the monitor 11. . On the other hand, an image taken by the camera 12 for detecting the line of sight is temporarily held in the image memory 20 through the controller 18 and used for a line-of-sight detection process described later. When the release button 15 is half-pressed, the controller 18 turns on the line-of-sight detection light source 13 and shoots with the line-of-sight detection camera 12. Further, the still image captured by the imaging unit 19 when the release button 15 is fully pressed is temporarily displayed on the monitor 11 and recorded on a removable recording medium such as the memory card 22.

  In the example of FIG. 1, the finder 21 is provided on the top surface of the digital camera 10, but the line-of-sight input AF processing of the present embodiment is premised on photographing through a through image displayed on the monitor 11. It may be a structure without a finder.

  In the line-of-sight input AF process of the present embodiment, the face of the user who views the through image displayed on the monitor 11 is photographed by the line-of-sight detection camera 12, the direction of the user's line of sight is detected based on this image, and the user gazes. The position (area) on the monitor 11 is estimated. Then, an AF process for focusing on a subject in a predetermined range including the gaze position is performed.

  In photographing while viewing the through image of the monitor 11, the position of the user's face relative to the camera 10 differs depending on the photographing situation, as shown in FIGS. 3 (a) to 3 (c). For example, as shown in FIG. 3A, when the camera 10 is held in front of the face and the through image of the monitor 11 is viewed up close, the eye-gaze detection camera 12 captures an up image of the face. On the other hand, as shown in FIGS. 3B and 3C, when the camera 10 is taken away from the face and the through image of the monitor 11 is viewed from a distance, the face is zoomed out, and the area occupied by the face in the shooting screen is as follows. Get smaller. When the face image becomes smaller, it becomes difficult to estimate the line of sight only by the shape and position of the pupil. Further, since the distance from the monitor 11 is large, the influence of the error due to the difference in the direction of the line of sight increases. Therefore, in the present embodiment, as will be described later, the gaze direction estimation method is changed in accordance with the situation of the image captured by the gaze detection camera 12, and the gaze direction is always detected by an optimal method, and appropriate AF is performed. Select an area.

  That is, the position on the monitor 11 where the user is gazing is estimated from the eye position in the image and the detected line-of-sight direction, and the area including the position is set as the AF area. For example, as shown in FIG. 4, the screen (image) M is divided into nine areas of 3 × 3 in the vertical and horizontal directions, and one of them is selected from the direction of the line of sight. In the example of FIG. 4, it is determined that the area A1 on the left side of the middle stage is an area that the user is gazing at, and is set as the first stage AF area.

  In the illustrated example, the [] symbol is displayed in the area A1, indicating that the area is set as the AF area. The method of indicating the AF area is not limited to this, and other symbols may be used, the area may be surrounded by a specific color frame, or the area may be colored with a specific color. Alternatively, a combination of these may indicate the AF area.

  Furthermore, in this embodiment, when the dial switch 16 (FIG. 1) is rotated in a predetermined direction (enlargement direction) in the state of FIG. 4, that is, in the state where the first-stage AF area is set, the image of the area A1 is enlarged. Then, as shown in FIG. 5, the whole is displayed on the screen M of the monitor 11. The image in the screen M is also divided into, for example, 9 × 3 × 3 areas. When enlarged display is performed, line-of-sight detection using the line-of-sight detection camera 11 is executed again, and the user's gaze area in the enlarged image is displayed. The identified area is set as the second stage AF area. In the example of FIG. 5, it is determined that the area A2 at the center of the lower stage is the area that the user is gazing at, and is set as the AF area of the second stage.

  That is, after selecting the first-stage AF area A1, the user operates the dial switch 16 so that a narrower area within the angle of view (an area of 1/81 of the original image in the illustrated example) is set as the AF area. The subject to be focused can be selected more finely. After setting the AF area A2 in the second stage, the dial switch 16 is operated in the reduction direction to return (reduce) to the state shown in FIG. 4 (full screen display), and the AF area reselection using the line-of-sight input is repeated. It is also possible. In the state of FIG. 5 in which the second stage AF area A2 is set, the dial switch 16 is operated in the enlargement direction to enlarge the AF area A2 on the screen M, and the third stage AF is performed using the line-of-sight input. It is good also as a structure which can set an area. For example, when a predetermined button is operated during the AF area setting process, the selected AF area is fixed.

  Next, an AF area setting process in the line-of-sight input AF process of the present embodiment will be described with reference to the flowchart of FIG. 6 and FIGS. This process is executed by the controller 18 when the release button 15 is half-pressed, for example.

  In the AF area setting process, first, a line-of-sight detection process is executed in step S100. That is, infrared rays are emitted from the light sources 13 and 14 as illumination light, and a user who is looking at a through image on the monitor 11 is photographed by the line-of-sight detection camera 12. Then, the direction of the user's line of sight is detected by a method to be described later using an image in which the user is captured, and if the gaze area on the screen of the monitor 11 is specified, the area is determined as the AF area.

  In step S102, it is determined whether or not an AF area (focus area) has been confirmed. If not, the process in step S100 is repeated. If the AF area has been determined, it is determined whether or not the dial switch 16 has been operated in step S104, and if it has been operated, it has been determined whether it has been operated in the enlargement or reduction direction.

  If the dial switch 16 is operated in the direction of enlarging the image, the identified AF area (gaze focus area) is enlarged in step S106, and the processes in and after step S100 are repeated. On the other hand, when the dial switch 16 is operated in the direction of reducing the image, the AF area (gaze focus area) is reduced in step S108, and the processes in and after step S100 are repeated. In addition, when the dial switch 16 is not operated and a certain time elapses, or when the operation is performed in the enlargement / reduction direction but further enlargement / reduction is not possible, and a certain time elapses, or a predetermined button is operated In this case, the present process is terminated, the currently set area is fixed to the AF area, and the display on the monitor 11 is returned to an unexpanded state.

  Next, the gaze detection process of FIG. 6 will be described with reference to the flowcharts of FIGS. 1, 2, 7, and 8 and FIGS. 9 to 12. 9 to 12 are schematic diagrams exemplarily showing six different states of images (input images) G captured by the line-of-sight detection camera 12 and held in the image memory 20.

  As described above, in the line-of-sight detection process, first, the light sources 13A and 13B are turned on and an image is captured by the line-of-sight detection camera 12 and acquired as an input image G. In step S202, a conventionally known facial organ detection process is performed on the input image G. That is, feature points such as eyes, nose and mouth are detected, and the face area F and eye areas E1 and E2 are detected.

  In step S204, it is determined whether or not the face area F is detected from the input image G. The face area F is an area that includes substantially the entire face as shown in FIGS. 8 and 9, and if it is determined that the face area F has been detected, the size of the detected face area F ( Size) is calculated, and the sizes (sizes) of the eye regions E1 and E2 detected in step S208 are calculated.

  Thereafter, in step S210, it is determined whether or not the size of the face area F is equal to or less than the first threshold Th1. At the time of capturing an image for line-of-sight detection, the user's face is separated from the monitor as shown in FIGS. 3B and 3C, the size of the face region F is small as shown in FIG. When it is determined that it is equal to or less than Th1, the size of the face region F is considered to be too small for line-of-sight detection. Therefore, the process returns to step S200, and a new line-of-sight detection image is taken as an input image. And the process after step S202 is repeated.

  On the other hand, when it is determined in step S210 that the area of the face region F is larger than the first threshold Th1 (for example, a line-of-sight detection image is captured in the state of FIGS. 3A to 2B, When the image of FIG. 10 is acquired), in step S212, it is determined whether or not the size of the face region F is equal to or smaller than the second threshold Th2 that is larger than the first threshold Th1. If it is determined that the size of the face area F is equal to or smaller than the second threshold (Th1 <the size of the face area F ≦ Th2), it is considered that the face area F is too small to perform gaze detection using the pupil. Therefore, in step S214, the line-of-sight direction is estimated from the positional relationship of the facial feature points P1 to P10 such as eyes, nose, and mouth in FIG. 10, and the line-of-sight detection process ends.

  On the other hand, if it is determined in step S204 that the face area F is not detected, or if it is determined in step S212 that the size of the face area F is larger than the second threshold Th2, in step S216, the input image G Whether or not both eyes (eye areas E1, E2) are detected is determined. That is, as shown in FIGS. 11 and 12, when the user's face is close to the camera at the time of shooting, only a part of the user's face is shot and the face area F is not detected, or the entire input image G is If the face area F is occupied, there is a high possibility that eye-gaze detection can be performed from the captured eye image. Therefore, it is determined whether or not both eyes are included in the input image G and eye-gaze detection by the pupil can be performed. .

  If it is determined in step S216 that both eyes are included in the input image G, the process proceeds to step S218, and pupil detection processing is performed. On the other hand, if it is determined that both eyes are not included, the process returns to step S200, a new image is taken by the line-of-sight detection camera 12, and the above-described processing is repeated.

  In step S218, a pupil is detected from the eye regions E1 and E2, and in step S220, it is determined whether or not the size (for example, area) of the pupil is equal to or greater than a third threshold value. If the size of the pupil is greater than or equal to the third threshold value, the gaze direction is estimated by the pupil in step S222, and the gaze detection process ends. If the size of the pupil is smaller than the third threshold value, in step S224, the gaze direction is estimated using the facial feature points in the same manner as in step S214, and the gaze detection process ends.

  Next, with reference to the flowchart of FIG. 13 and FIG. 14, the gaze direction estimation process using the pupil image in step S222 of FIG. 8 will be described.

  In the present embodiment, the line-of-sight direction is estimated using a change in the shape of the pupil depending on the direction of the line of sight. That is, the pupil has an elliptical shape inclined in various directions depending on the direction of the line of sight, and the line-of-sight direction is estimated from the position of the center of the eyeball, the shape of the pupil, and the inclination thereof.

In the gaze direction estimation process using the pupil shape change, in step S300, an image of the eye area (E1, E2) is acquired, and in step S302, the image is smoothed. In step S304, the image is converted to grayscale, and binarized in step S306. In step S308, a contour extraction process is performed on the binarized eye image, and an ellipse fitting is performed in step S310. In step S312, the line-of-sight direction (φ, θ) is obtained by ellipse fitting using the ellipse parameters (a, b, α) shown in FIG.
tan φ = ± √ ((a ² −b ² ) / b ² ) · sin α (1)
tan θ = ± √ ((a ² −b ² ) / b ² ) · cos α (2)
Here, a is the major axis of the ellipse, b is the minor axis, and α is the inclination angle of the ellipse on the image plane (X′Y ′ plane) and corresponds to the inclination of the line of sight. Since the elliptical shape is symmetric with respect to the origin and has the same parameters, it is necessary to select a code based on the pupil center coordinates.

  In FIG. 14, if the + direction of the Z-axis is the direction in which the camera is directed, the line-of-sight direction is the PO direction in the figure. In FIG. 14, β is the angle of the line of sight, and is obtained from the pupil flatness b / a.

The estimation of the line-of-sight direction using the pupil is not limited to the above method, and can be obtained from the position (x, y) on the image of the pupil center P with respect to the eyeball center O, for example. For example, as shown in FIG. 14, the line-of-sight direction (φ, θ) is obtained by the following equation.
tan φ = y / √ (r ² −x ² −y ² ) (3)
tan θ = x / √ (r ² −x ² −y ² ) (4)
It should be noted that any other combination of gaze direction vectors (for example, (φ, θ), (φ, ψ), (θ, ψ), from information such as x, y, a, b, α, etc. (Φ, θ, ψ, etc.) may be estimated.

  When the eye position is specified in the eye-gaze detection image and the eye-gaze direction is estimated from the feature amount of the pupil or the face, it is specified which divided area (for example, 3 × 3) of the monitor 11 is being watched. The

  As described above, according to the present embodiment, the orientation of the face is specified based on the positional relationship of some or all of the feature quantities of the eyes, nose, and mouth according to the size of the captured face image, and the pupil shape By specifying the direction of the line of sight from the position of the center of the pupil and the line of sight, the line of sight can be detected by a method optimal for the size of the face image.

  Next, a modification of the line-of-sight detection process will be described with reference to the flowchart of FIG. In the modified example, a configuration using the facial feature amount in particular for the direction of the nose is employed.

  In the line-of-sight detection processing according to the modification, in steps S400 to S406, processing similar to that in steps S200 to S206 in FIG. 7 is performed. That is, an input image for line-of-sight detection is acquired in step S400, face organ detection processing is executed on the input image G in step S402, and feature points such as eyes, nose and mouth are detected, and a facial region F, eye region E, etc. are detected. In step S404, it is determined whether or not the face area F is detected from the input image G. When the face area F is detected, the size (size) of the detected face area F is calculated in step S406. .

  In step S408, it is determined whether or not the size of the face area F is equal to or smaller than the first threshold Th1, and if it is equal to or smaller than the first threshold Th1, it is displayed on the monitor 11 that the line-of-sight direction cannot be detected in step S410. Then, the gaze detection process ends. On the other hand, if it is determined in step S408 that the size of the face area F is larger than the first threshold Th1, the size of the face area F is less than the second threshold Th2 that is larger than the first threshold Th1 in step S416. In step S418, the face direction is detected from the feature amount of the entire face, and in step S420, the user's gaze direction is estimated based on the face direction. The line-of-sight detection process ends. If the size of the face region F is larger than the second threshold Th2 in step S416, the direction of the nose is calculated from the nose feature amount in step S414, and the line-of-sight direction of the user based on the nose direction in step S420. Is estimated, and the gaze detection process ends.

  On the other hand, if it is determined in step S404 that the face area F is not detected, it is determined in step S412 whether or not a nose is detected. If the nose is detected, the direction of the nose is calculated in step S414, and if no nose is detected, it is determined in step S422 whether or not both eyes are detected. If both eyes have not been detected, the process returns to step S400 and the same processing is repeated. If both eyes have been detected, the processing from step S218 to step S224 in FIG. 8 is executed, and the line-of-sight detection processing ends. In step S224 of the modified example, the line-of-sight direction is estimated based on the direction of the nose. When the nose is detected in step S412, and the direction of the nose is calculated in step S414, the line-of-sight direction of the user is estimated based on the direction of the nose in step S420, and the line-of-sight detection process ends.

  As described above, also in the modified example, the same effect as in the present embodiment can be obtained.

  In this embodiment, an infrared image is used as the line-of-sight detection image, but a normal color image may be used. In addition, the number of illumination light sources used for capturing the line-of-sight detection image may be one or three or more, and any configuration that can illuminate the entire face as viewed on the monitor is acceptable. For example, the lenses may be arranged at substantially equal intervals around the lens (120 ° intervals for three lenses, 90 ° intervals for four lenses). By using a plurality of illumination light sources, uneven illumination of the face area can be reduced, so that each accuracy of face area detection, face feature point detection, and pupil detection can be improved.

  In the present embodiment, the line-of-sight input AF of a digital camera has been described as an example. However, the present invention is limited to AF processing and a camera as long as the gaze area on the screen is specified by detecting the line of sight of the user viewing the screen. It is not a thing. In this embodiment, since the monitor is provided on the back side of the camera, the direction of the photography camera and the line-of-sight detection camera are opposite to each other, but the line-of-sight detection camera may face the same direction as the monitor. For example, when the monitor can be rotated vertically and horizontally with respect to the camera body, the line-of-sight detection camera moves integrally with the monitor. In the present embodiment, the AF process is not particularly mentioned, but any of contrast AF, phase difference AF, and the like can be used.

DESCRIPTION OF SYMBOLS 10 Digital camera 11 Monitor 12 Eye-gaze detection camera 13A, 13B Light source 14 Switch group 15 Release button 16 Dial switch (input part)
17 Function keys (input section)
18 controller 19 imaging unit 20 image memory 21 finder A1 first stage specific area A2 second stage specific area

Claims (13)

A monitor for displaying images;
Gaze detection imaging means for imaging the direction in which the monitor is directed;
A face area detecting means for detecting a face area from the first image photographed by the eye-gaze detecting photographing means;
Face area size calculating means for calculating the size of the face area;
Facial feature point detection means for detecting facial feature quantities including eyes, nose and mouth from the first image;
First line-of-sight estimation means for estimating a line-of-sight direction from position information of any one or a plurality of feature quantities of the face;
Second line-of-sight estimation means for detecting a pupil from the first image and estimating a direction of the line of sight;
Monitor area specifying means for selecting the first or second line-of-sight estimation means according to the size of the face area, and specifying the area on the monitor where the estimated line of sight gazes. A gaze direction detecting device.   The monitor area specifying means selects the first line-of-sight estimation means when the size of the face area is smaller than a second threshold, and selects the second line-of-sight estimation means when larger than the second threshold. The gaze direction detecting device according to claim 1, wherein   3. The gaze direction according to claim 2, wherein the monitor area specifying unit does not estimate the gaze direction when the size of the face area is smaller than a first threshold value smaller than the second threshold value. Detection device.   The monitor area specifying unit selects the second line-of-sight estimation unit and specifies an area on the monitor when both eyes are detected from the first image when the face area is not detected. The gaze direction detecting device according to claim 1 or 2.   The gaze direction detection apparatus according to claim 1, wherein a plurality of illumination light sources are arranged around a photographing lens of the gaze detection photographing unit.   The gaze direction detecting device according to claim 1, further comprising: a specific area notifying unit that allows a user to recognize an area specified by the monitor area specifying unit.   The specific area recognition means is one of surrounding a specific area with a specific color frame, displaying a specific mark on the area, and coloring the entire area with a specific color component. Or it shows that the said area | region is specified using these combination, The gaze direction detection apparatus of Claim 6 characterized by the above-mentioned.   An input means for instructing enlargement of an image displayed on the monitor is provided, and when the enlargement of the image is instructed, an image of an area specified by the monitor area specifying means is enlarged and displayed on the monitor, and the line-of-sight detection 8. The second image is photographed by using a photographing means, and an area on the monitor is further specified by the monitor area specifying means based on the second image. The line-of-sight direction detection apparatus according to 1.   The line-of-sight direction detection apparatus according to claim 8, wherein the input unit includes an operation unit that fixes an area specified by the monitor area specifying unit.   The line-of-sight direction detection apparatus according to claim 9, wherein after the region is fixed, an image having a normal size that is not enlarged is displayed on the monitor. The visual axis detecting photographing means, gaze direction detecting apparatus according to any one of claims 1-10, characterized in that integrally displaced with the monitoring to the apparatus main body. An imaging apparatus and a different image capturing means and the line-of-sight direction detecting apparatus according, the shadow unit Taking for the sight line detection in any of claims 1 to 11,
A monitor image photographed by the image photographing means is displayed on the monitor, and an area specified by the monitor area specifying means is used for an AF area. A line-of-sight detection image capturing step for capturing the line-of-sight direction of the photographer;
A face area detecting step of detecting a face area from the first image captured in the line-of-sight detection image capturing step;
A face area size calculating step for calculating the size of the face area;
A facial feature point detecting step for detecting facial feature quantities including eyes, nose and mouth from the first image;
A first line-of-sight estimation step of estimating a line-of-sight direction from position information of any one or a plurality of feature quantities of the face;
A second line-of-sight estimation step of detecting a pupil from the first image and estimating a direction of the line of sight;
Characterized in that it comprises said in response to the size of the face region, the first or selects the second sight estimation step, monitor area specifying step of the sight line to identify the region on the watch to makes the chromophore at the distal end Nita estimated The gaze direction detection method.

Images