JP2917953B2 - View point position detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a viewpoint position detecting device used in a stereoscopic display device or the like which constantly detects a viewpoint position of an observer and always presents a correct stereoscopic image to the observer in accordance with the detected viewpoint position. .

[0002]

2. Description of the Related Art In recent years, demands for stereoscopic display have been increasing, and various stereoscopic display devices have been studied. In recent years, attention has been paid to a stereoscopic display device that realizes stereoscopic vision by presenting an image having binocular parallax spatially separated from the left and right eyes using a lenticular screen. This device has the advantage that no special glasses need to be worn. However, this device has a problem that the area where stereoscopic vision is possible is limited to about the distance between human eyes, and the observer must take an unnatural posture of always keeping his head fixed for observation. Was.

In order to solve such a problem, a viewpoint-following type stereoscopic display device which constantly detects the observer's viewpoint position and always presents a correct stereoscopic image to the observer in accordance with the detected viewpoint position has been proposed. Known (see, for example,
7247, JP-A-5-7123, etc.). In the conventional viewpoint-following type stereoscopic display device, it is necessary to attach a magnetic sensor, a reflecting mirror, and the like to the observer to detect the viewpoint position, which gives the observer a sense of incongruity and wears special glasses. This may reduce the advantage of the three-dimensional display device using a lenticular screen that is unnecessary. In addition, when used by a large number of unspecified observers, the use of a wearable viewpoint detection device causes a problem in hygiene.

Accordingly, for example, Japanese Patent Laid-Open No. 4-105432 discloses
Japanese Patent Publication No. 7-82539 and JP-B-7-82539 describe that only the pupil is extracted from the captured image of the observer and the pupil position of the observer is detected in a non-contact manner. In these pupil detection devices, pupil detection is performed using the reflection characteristics of the human pupil. The human pupil has a property that, when illuminated by an infrared light illuminating device having the optical axis coaxial with the imaging device, the reflectance of the pupil becomes larger than the reflectance of other portions of the face, and the optical axis of the pupil is not aligned with the optical axis. When illuminated by a coaxial infrared light illuminator, there is a property that the reflectance of the pupil becomes smaller than the reflectance of other parts of the face. Therefore, when comparing the corresponding pixels of the respective images obtained using these different illumination devices, the difference in the pupil portion is larger than the difference in the face portion,
Only the human pupil can be extracted.

FIG. 13 is a diagram for explaining a conventional pupil detection device.

As shown in FIG. 13, a conventional pupil detecting device includes an image pickup device 104, a first infrared light illuminating device 101 and a second infrared light illuminating device 102 for illuminating the face of an observer 120, respectively. And First infrared light illumination device 1
01 has its optical axis arranged coaxially with the imaging device 104, and the second infrared light illuminating device 102 has its optical axis arranged non-coaxially with the imaging device 104.

In this pupil detection device, the first infrared light detection
Image I obtained using device 101 ₁ And the second infrared light
Image I obtained using the detection device 102_Two Difference with
Wachi I₁ -I_Two Is calculated by the image difference calculation unit 111, and the
After that, the pupil extraction unit 112 performs appropriate binarization processing, and performs observation.
The pupil position of the person 120 is detected. Thus, conventional
In the detection device of the above, each infrared light illumination device 10
Comparison of the corresponding pixels of the image obtained using
The difference operation is used for the calculation.

[0008]

However, in the conventional pupil detecting device, an image obtained by using the first infrared light illuminating device and an image obtained by using the second infrared light illuminating device are different from each other. In the difference image, there is a difference in the illumination characteristics of each illumination device as a noise component. Therefore, in order to detect a pupil with high accuracy, it is necessary to perform binarization in consideration of a signal-to-noise ratio between a pupil component and a noise component. There are individual differences in the reflection characteristics of the pupil, and when the observer uses spectacles or the like, the reflected light intensity of the pupil decreases, and the signal-to-noise ratio decreases. When the signal-to-noise ratio fluctuates as described above, it is difficult to uniquely determine the binarization threshold when extracting the pupil portion by the binarization process, and it is difficult to perform stable pupil detection.

The problem of the conventional pupil detection device will be described with reference to FIGS.

FIG. 14 is a diagram showing an image obtained by the image pickup device of the pupil detection device shown in FIG. 13. FIG. 14A shows an observer obtained by an illumination device having an optical axis coaxial with the image pickup device. (B) is an image of the observer obtained by the illumination device having the optical axis non-coaxial with the imaging device. In addition, FIG.
(F) is a line segment A in FIGS. 10 (a) and (b), respectively.
5 shows a luminance distribution of an image between B and B.

[0011] As shown in FIG. 15 (a), the image I ₁ the optical axis is obtained by using the illumination device of the imaging apparatus and the coaxial pupil component 121a is a component corresponding to the pupil portion 120a
Is higher than the luminance of the component corresponding to the face portion 120b. On the other hand, as shown in FIG. 15B, in the image I ₂ obtained using the illumination device whose optical axis is non-coaxial with the imaging device,
The luminance of the pupil component 121b is lower than the luminance of the component corresponding to the face portion 120b. Therefore, these images I ₁ ,
Taking the difference I ₁ -I ₂ of I _2, the pupil component 121c as shown in FIG. 15 (c) is emphasized, it can be extracted pupil by an appropriate binarization.

In the difference image, a difference in illumination characteristics between the respective illumination devices exists as a noise component 122. In order to detect the pupil with high accuracy, it is necessary to binarize the pupil in consideration of the signal-to-noise ratio between the pupil component 121c and the noise component 122. In this case, the threshold value Th1 for binarization may be set as shown in FIG.

However, the intensity of the reflected light from the pupil decreases due to individual differences in the reflection characteristics of the pupil and the use of eyeglasses.
As shown in (d), the peak value of the pupil component 121d may decrease. At this time, the difference I ₁ −I ₂ between the image I 2 and the image I 2 obtained using the non-coaxial illumination device shown in FIG. 15E is obtained as shown in FIG. 15F. Then, the luminance of the pupil component 121f decreases. As a result, the signal-to-noise ratio between the pupil component 121f and the noise component 122 becomes small, and it becomes difficult to determine the threshold value Th2.

For example, in an image obtained using an illumination device whose optical axis is not coaxial with the imaging device, the luminance of the pupil component is 5%.
0 (arbitrary unit; the same applies hereinafter), the luminance of the component corresponding to the face portion is 40, and the luminance of the pupil component is 0 and the luminance of the face When the luminance of the component corresponding to the part is 30, the luminance of the pupil component in the difference image is 50, and the luminance of the noise component is 10 in the difference image. Although the signal-to-noise ratio at this time is 5, the luminance of the pupil component is reduced from 50 to 10
, The signal-to-noise ratio becomes 1, and pupil extraction by binarization becomes unstable. As described above, when the pupil extraction becomes unstable, it becomes difficult to discriminate the pupil from the face of the observer, and it becomes impossible to detect the pupil accurately.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a viewpoint position detecting apparatus which stably extracts an observer's pupil by suppressing a change in a signal-to-noise ratio of a pupil image signal and accurately detects a viewpoint position. .

[0016]

According to the present invention, there is provided a viewpoint position detecting apparatus, comprising: an image pickup means for picking up an image of a face of a person to be detected; and an optical axis for illuminating the face of the person to be detected. First illumination means substantially coaxial with the axis, and second illumination means wherein the optical axis is non-coaxial with the optical axis of the imaging means;
A first image, which is an image captured by the imaging unit in a state where the subject is illuminated by the first illumination unit, and a first image in which the subject is illuminated by the second illumination unit Adding means for adding, for each pixel, the luminance of a pixel corresponding to a second image which is an image captured by the image capturing means; and luminance of a pixel corresponding to the first image and the second image. Is subtracted for each pixel, and the calculation results obtained by the addition means and the subtraction means are divided by each pixel.Based on the result of the division by the division means, Extracting means for extracting a portion corresponding to the pupil of the subject.

Further, the viewpoint position detecting device of the present invention has an image pickup means for picking up an image of the face of the person to be detected, and an optical axis for illuminating the face of the person to be detected is substantially equal to the optical axis of the image pickup means A first illuminating means coaxial with the second illuminating means, the optical axis of which is non-coaxial with the optical axis of the imaging means, and the image pickup in a state where the subject is illuminated by the first illuminating means Of a pixel corresponding to a first image that is an image captured by the means and a second image that is an image captured by the imaging means in a state where the subject is illuminated by the second illumination means. Adding means for adding the luminance for each pixel;
A dividing unit that divides the luminance of a pixel corresponding to the first image and the operation result obtained by the adding unit for each pixel; and, based on a result of the division by the dividing unit, Extracting means for extracting a portion corresponding to the pupil.

In the present invention configured as described above, the first image which is an image of the face of the subject illuminated by the first illumination means which is coaxial with the imaging means, and which is non-coaxial with the imaging means. A second image, which is an image of the face of the subject illuminated by the second illumination means, is used. Then, an addition process and a subtraction process are performed on the two types of images for the brightness of the corresponding pixels for each pixel, and the ratio of the calculation results is obtained. Alternatively, an addition process is performed on two types of images, and a ratio between the result and the first image is obtained. Thus, in addition to the subtraction operation, the addition operation and the division operation are used,
By extracting the pupil of the subject from the result, the variation of the signal-to-noise ratio of the pupil image can be suppressed even when the pupil reflection luminance of the subject changes. As a result, the pupil of the subject is stably extracted, and accurate detection of the viewpoint position becomes possible.

In order to obtain the above two types of images, it is assumed that each of the illuminating means irradiates light of a different wavelength, and the imaging means is provided with a spectral means for separating these wavelengths and a light corresponding to the separated light. Using two image sensors, or illuminating each illuminating means alternately at a fixed cycle assuming that both irradiate light of the same wavelength. May be configured to capture the above image.

The above calculations and the extraction of the portion corresponding to the pupil can be performed by an analog circuit.

[0021]

Next, embodiments of the present invention will be described with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram of a first embodiment of a viewpoint position detecting device according to the present invention.

As shown in FIG. 1, the viewpoint position detecting device of the present embodiment includes a first illuminating device 1 for irradiating infrared light having a wavelength of λ1 and a second illuminating device 1 for irradiating infrared light having a wavelength of λ2. Illumination device 2, an imaging device 4 that captures the face of an observer 20 (a person to be detected) illuminated by the first illumination device 1 and the second illumination device 2, and an image captured by the imaging device 4. , Observer 20
And an image processing device 10 for performing a predetermined process for detecting the pupil position of the image. In the present embodiment, a light emitting diode having a wavelength of 850 nm is used as a light source of the first lighting device 1, and a light source of a wavelength of 940 n is used as a light source of the second lighting device 2.
m light emitting diodes were used.

First lighting device 1 and second lighting device 2
Are for illuminating the face of the observer 20.
The infrared light emitted from the first illumination device 1 illuminates the face of the observer 20 after being reflected by the half mirror 3. Here, the infrared light reflected by the half mirror 3 has its optical axis coaxial with the optical axis of the optical system of the imaging device 4, and the first lighting device 1 Illuminate 20 faces. The second illumination device 2 is arranged at a position where its optical axis is non-coaxial with the optical axis of the optical system of the imaging device 4.

The image processing device 10 stores a first image memory 1 for storing an image I ₁ obtained by using the first lighting device 1.
1, a second image memory 12 for storing the image I ₂ obtained by using the second illumination device 2, addition operation the brightness of a corresponding pixel of the two images I _1, I ₂ is added for each pixel Part 13
And a subtraction operation unit 14 for subtracting the luminance of the corresponding pixel of both images I ₁ and I ₂ for each pixel, and a division operation for dividing the calculation results of the addition operation unit 13 and the subtraction operation unit 14 for each pixel A pupil extraction unit 16 that extracts a portion corresponding to the pupil of the observer 20 based on the division result obtained by the division operation unit 15; and a pupil position detection unit 17 that obtains the center of gravity of the portion corresponding to the pupil. Consists of Here, the first
The image memory 11 and the second image memory 12 having a resolution of 512 × 512 pixels were used.

The image pickup device 4 senses infrared light. Hereinafter, the imaging device 4 will be described with reference to FIG.

The imaging device 4 converts the incident light into two wavelengths λ1,
It is composed of a dichroic prism 5, a first imaging device 6, a second imaging device 7, and a lens 8, which are spectral means for dividing into λ2, and separates and captures infrared images of two wavelengths of λ1 and λ2. Can be. A solid-state image sensor is used as each of the first image sensor 6 and the second image sensor 7, and they are driven synchronously so that images at the same time can be captured. Further, a filter 9 for blocking visible light is inserted in the image pickup device 4 so as not to be affected by an external lighting device such as an indoor light that emits visible light.

In the present embodiment, the image pickup device 4 conforms to the NTSC standard, has a horizontal frequency of 15.75 kHz,
The vertical frequency is 60 Hz. The dichroic prism 5 has a separation wavelength of 900 nm, and the filter 9 has a cutoff wavelength of 800 nm. Note that, instead of the dichroic prism 5, a dichroic mirror and a normal mirror may be used in combination.

The observer 20 has a first illuminating device 1 and a second illuminating device 2 for irradiating infrared light having different wavelengths.
The image of the observer 20 is separated and imaged by the dichroic prism 5 in the imaging device 4 into the first imaging device 6 and the second imaging device 7. The infrared light of the wavelength λ1 of the first lighting device 1 reaches the first image sensor 6, and the infrared light of the wavelength λ2 of the second lighting device 2 reaches the second image sensor 7. Since the first illumination device 1 illuminates the observer 20 coaxially with the imaging device 4, the first imaging device 6
In, the pupil of the observer 20 is imaged brightly. On the other hand, the second
Since the illumination device 2 is arranged non-coaxially with the imaging device 4, the second imaging device 7 images the pupil of the observer 20 darkly. The image captured by the first image sensor 6, that is, the image I ₁ obtained using the first lighting device 1, is stored in the first image memory 11 in the image processing device 4. The image captured by the second image sensor 7, that is, the second illumination device 2
Is obtained by using the _second image in the image processing device 4.
Is stored in the image memory 12.

Next, the sum and difference of the luminance of the corresponding pixels of the image memories 11 and 12 are obtained by the addition operation unit 13 and the subtraction operation unit 14, respectively, and the ratio of the operation results is obtained by the division operation unit 15. . After that, binarization is performed by the pupil extraction unit 16 to obtain the pupil position of the observer 20. Since the obtained pupil positions are two, the pupil position detecting unit 17 calculates the center of gravity of the image, obtains the obtained position, that is, the center position of the two pupils, and determines this position as the viewpoint position of the observer 20. Output as These various operations and binarization processing are realized by an electronic computer. The viewpoint position detection is performed for each field of the imaging device 4, that is, 60 times per second. The resolution of the viewpoint position detection is 1 mm.

Next, the principle of detection by the viewpoint position detecting device of the present embodiment will be described with reference to FIGS.

FIGS. 3A and 3B are diagrams showing images obtained by the image pickup device shown in FIG. 2, wherein FIG. 3A shows an image of the observer picked up by the first image pickup device, and FIG. It is an image of the observer captured by the imaging device. 4 (a) to 4 (f)
3A and 3B respectively show a luminance distribution of an image between line segments AB in FIGS.

As shown in FIG. 4A, in the image I ₁ obtained by using the first illuminating device 1, the luminance of the pupil component 21a, which is the component corresponding to the pupil portion 20a, is reduced.
It becomes higher than the luminance of the component corresponding to b. On the other hand, FIG.
As shown in (b), in the image I ₂ obtained using the second illumination device 2, the luminance of the pupil component 21b is lower than that of the face portion 20b.
Is lower than the luminance of the component corresponding to.

Here, the addition operation unit 13 and the subtraction operation unit 14
And using the division operation unit 15 for each pixel (x, y)
The performance based on the function I (x, y) shown in the following equation (1)
Perform the calculation. I (x, y) = (I₁(x, y) -I_Two(x, y)) / (I₁(x, y) + I_Two(x, y)) (1) In equation (1), I₁ (X, y) is the first image sensor
6 shows the luminance distribution of the image taken in_Two (X, y)
Indicates the luminance distribution of the image captured by the second image sensor 7.
You. The function I (x, y) is expressed as -1 ≦ I (x, y) ≦
A function that takes on a value between 1 and₁ (X, y) = I_Two 
(X, y) takes a value of 0 (however, I ₁ (X,
y) = I_Two (X, y) ≠ 0) and I₁ (X, y)
> I_Two In (x, y), 0 <I (x, y) ≦ 1
And I₁ (X, y) <I_Two -1 at (x, y)
≤I (x, y) <0.

When the above calculation is performed on the image luminance distributions shown in FIGS. 4A and 4B, the luminance of the component corresponding to the face portion 20b is obtained as shown in FIG. 4C. The difference becomes the noise component 22 and appears near 0 (zero). Normally, the brightness of the pupil component 21b in FIG. 4B is sufficiently small, so that the pupil component 21c in FIG. Therefore, if the threshold value Th1 is set to a value having a sufficient margin for the noise component 22 and the binarization process is performed, only the pupil component 21c can be extracted.

Here, even if the observer 20 uses glasses or the like and the luminance of the pupil component 21d decreases as shown in FIG. 4D, the pupil component 21e in FIG. If the luminance is sufficiently small, the above calculation is performed so that the luminance of the pupil component 21f takes a value close to 1 as shown in FIG. Therefore, the binarization processing for extracting the pupil component 21f may be performed at the same threshold Th1 as described above, and even if the pupil reflection intensity of the observer 20 decreases, the pupil component 21f can be stably extracted. it can.

Using the viewpoint position detecting device described above,
Face of the observer 20 at a distance of 2 m from the viewpoint position detection device
I took each image I₁ , I_Two Upper pupil component (coordinates
(X _P , Y_P )) And face components (coordinates (x_F , Y
_F )) Is I₁ (X_P , Y_P) = 50 (arbitrary unit;
Hereinafter the same), I_Two (X_P , Y_P ) = 0, I₁ (X_F , Y
_F) = 40, I_Two (X_F , Y_F ) = 30. this
At this time, the result of the luminance comparison operation of the pupil component and the face component is I
(X_P , Y_P ) = 1.00, I (x_F , Y_F ) = 0.1
4 and the signal-to-noise ratio is 7. Therefore, binary
Assuming that the threshold value Th1 of the conversion process is 0.5, only the pupil is completely
I was able to extract.

Here, when the observer 20 uses glasses, the luminance of the pupil component decreases from 50 to 10, and each luminance becomes
I ₁ (x _P , y _P ) = 10, I ₂ (x _P , y _P ) = 0,
I ₁ (x _F , y _F ) = 40, I ₂ (x _F , y _F ) = 30
It became. In this case, the result of the luminance comparison operation of each component is I
_{(X P, y P) =} 1.00, I (x F, y F) = 0.1
The signal-to-noise ratio remained at 7, which was the same as when no glasses were used. Since there is no change in the signal-to-noise ratio, the threshold value Th is also used in this case as in the case where no glasses are used
When the binarization process was performed with 1 = 0.5, only the pupil could be completely extracted.

As described above, in the viewpoint position detecting device according to the present invention, even if the luminance of the pupil component, that is, the pupil reflected light intensity fluctuates, the signal-to-noise ratio does not fluctuate, and stable pupil extraction can be performed. Thus, accurate viewpoint position detection could be performed.

In this embodiment, an example has been shown in which the optical axis of the first illumination device 1 is made coincident with the optical axis of the imaging device 4 using the half mirror 3, but the pupil of the observer 20 is imaged brightly. If possible, the two optical axes do not have to coincide exactly. Further, the second lighting device 2 may use lighting that radiates visible light, such as an indoor light, instead of the device that radiates infrared light. In this case, the filter 9 is not inserted into the imaging device 4. The dichroic prism 5 separates visible light and infrared light. Further, the imaging device used in the imaging device 4 is not limited to a solid-state imaging device, and various imaging devices such as an imaging tube can be used.

(Second Embodiment) FIG. 5 is a block diagram of a second embodiment of the viewpoint position detecting device of the present invention.

The viewpoint position detecting device of the present embodiment is different from the first embodiment in the configuration of the image processing device 30. That is, the first image memory 31 and the second image memory 32
The operation unit for the images I ₁ and I ₂ stored in the memory is composed of only the addition operation unit 34 and the division operation unit 35,
The division operation unit 35 obtains, for each pixel, the luminance ratio of the corresponding pixel between the image I ₁ stored in the first memory 31 and the operation result of the addition operation unit 34. The pupil extraction unit 36 and the pupil position detection unit 37 are the same as in the first embodiment.

Since the structure of the parts other than the image processing apparatus 30 is the same as that of the first embodiment, the same reference numerals as those in FIG. 1 are used and the description thereof is omitted.

In the present embodiment, the addition operation unit 34 and the division operation unit 35 can output the operation result shown in equation (2). I (x, y) = I ₁ (x, y) / (I ₁ (x, y) + I ₂ (x, y)) (2) In this case, the function I (x, y) is I ₁ (x, y) y) = I ₂
(X, y) takes a value of 0.5 (however, I ₁
(X, y) = I ₂ (x, y) ≠ 0) and I ₁
When (x, y)> I ₂ (x, y), 0.5 <I
Taking a value of (x, y) ≦ 1, I ₁ (x, y) <I ₂
(X, y) takes a value of 0 ≦ I (x, y) <0.5.

Equation (2) indicates that the range of values output by equation (1) in the first embodiment is -1 ≦ I (x, y) ≦
It is linearly converted from 1 to 0 ≦ I (x, y) ≦ 1, and the principle and effect of detecting the viewpoint position are the same as in the first embodiment.

(Third Embodiment) FIG. 6 is a block diagram of a third embodiment of a viewpoint position detecting device according to the present invention. Also,
FIG. 7 is a configuration diagram of an imaging device of the viewpoint position detection device shown in FIG.

The viewpoint position detecting device according to the present embodiment irradiates the first illuminating device 41 and the second illuminating device 42 with infrared light having the same wavelength. Here, the light source of each of the lighting devices 41 and 42 has a wavelength of 850.
nm light emitting diode is used. The arrangement of the first lighting device 41 and the second lighting device 42 is the same as in the first embodiment.

[0048] The imaging device 44, as shown in FIG. 7, one of the imaging element 46, a lens 48, is composed of a filter 49 for blocking visible light, image brightness signal S _ii is from the image pickup device 46 Is output. The video luminance signal _Sii output from the image sensor 46 is input to the image processing device 10. As the filter 49, a filter having a cutoff wavelength of 800 nm is used. As described above, the first lighting device 41
And the second illumination device 42 emits infrared light having the same wavelength.
Therefore, in the present embodiment, the imaging device 44
It is not necessary to provide a spectroscopic means inside. Further, the image sensor 46
Since only one is required, the configuration of the imaging device 44 is simpler than that of the first embodiment.

Since the image processing apparatus 10 is the same as that used in the first embodiment, the same reference numerals as those in FIG. 1 are used, and the detailed description is omitted.

Further, in this embodiment, a synchronization signal generator 51 and an illumination controller 52 are provided. The first illumination device 41 and the second infrared light illumination device 42 use the synchronization signal _Ssy from the synchronization signal generation device 51 to _{operate the} imaging device 4.
Light is emitted alternately in synchronization with the field period of No. 4. The synchronization signal _Ssy generated by the synchronization signal generator 51 has a horizontal frequency of 15.75 kHz and a vertical frequency of 60 Hz according to the NTSC standard. The lighting control apparatus 52 separates a vertical synchronizing signal of the synchronizing signal S _sy, and serves to emit the illumination devices 41, 42 in the light emitting pattern shown in FIG. 8 for each field. In the present embodiment, the first lighting device 41 emits light during the odd field, and the second lighting device 41 emits the light during the even field.
Lighting device 42 emits light.

In the image processing apparatus 10, the first image
The memory 11 is a video luminance signal S_iiOnly in odd fields
Sampling, and the second image memory 12 stores the image luminance signal.
No. S _iiSample only on even fields of
Is set to Therefore, as shown in FIG.
The image when illuminated by the first illumination device 41 is the first image
It is stored in the moat 11 and illuminated by the second lighting device 42
The current image is stored in the second image memory 12.

The sum and difference of the luminances of the corresponding pixels of the image memories 11 and 12 are obtained by the addition operation unit 13 and the subtraction operation unit 14, respectively, and the ratio of the operation results is obtained by using the division operation unit 15. . Then, the pupil extraction unit 1
Then, the pupil position detector 17 calculates the center of gravity of the image, thereby detecting the viewpoint position. In the present embodiment, as described above, the storage of the image in the first image memory 11 and the second image memory 12 requires the time for two fields, that is, the time for one frame. Detection is performed 30 times per second. The resolution of detection of the viewpoint position is 1 mm.

The principle of pupil extraction is the same as in the first embodiment, and the division operation unit 15 outputs a result based on equation (1).

Using the viewpoint position detecting device of this embodiment, the face of the observer at a distance of 2 m from the viewpoint position detecting device is photographed, and the pupil components (coordinates (x _P , y
_P)), and the face component (coordinates (x _F, y _F)) each brightness at that _{I 1 (x P, y P} ) = 50 ( arbitrary units; hereinafter the _{same), I 2 (x P,} y P) = 0, I ₁ (x _F , y _F ) =
40, I ₂ (X _F , Y _F ) = 30. At this time,
The result of the luminance comparison operation of the pupil component and the face component is I (x
_{P, y P) = 1.00,} I (x F, a y _F) = 0.14, the signal-to-noise ratio is 7. Therefore, the threshold value of the binarization process was set to 0.5, and only the pupil could be completely extracted.

Here, if the observer uses glasses, the pupil formation
The brightness of the minute decreases from 50 to 10 and each brightness is I₁ (X
_P , Y_P ) = 10, I_Two (X_P , Y_P ) = 0, I₁ (X
_F , Y _F ) = 40, I_Two (X_F , Y_F ) = 30
Was. In this case, the result of the luminance comparison operation of each component is I (x
_P , Y_P ) = 1.00, I (x_F , Y_F ) = 0.14
And the signal-to-noise ratio is 7
It was still. Since there is no change in the signal-to-noise ratio,
In this case, the threshold is set to 0.5 as in the case where no glasses are used.
Binarization was performed, and only the pupil was completely extracted.
Came.

(Fourth Embodiment) FIG. 9 is a configuration diagram of a viewpoint position detecting apparatus according to a fourth embodiment of the present invention.

The viewpoint position detecting device of the present embodiment is different from the third embodiment in the configuration of the image processing device 30. That is, in the present embodiment, the same image processing apparatus 30 as that used in the second embodiment is used, and the arithmetic unit for the images stored in the first image memory 31 and the second image memory 32 is used. , An addition operation unit 34 and a division operation unit 35. Therefore, in this image processing apparatus,
The addition unit 34 and the division unit 35 output the calculation result shown in Expression (2), as in the second embodiment.

Since the configuration except for the image processing apparatus is the same as that of the third embodiment, the same reference numerals as those in FIG. 6 are used and the description thereof is omitted.

The principle and effect of detection of the viewpoint position are the same as in the third embodiment, and a description thereof will be omitted.

(Fifth Embodiment) FIG. 10 is a block diagram of a fifth embodiment of the viewpoint position detecting device of the present invention.

The viewpoint position detecting device of the present embodiment is similar to the first embodiment with respect to the first illuminating device 1, the second illuminating device 2, the half mirror 3, and the imaging device 4.
The processing on the image output from the imaging device 4 is performed by an analog circuit, and the first image memory 11, the second image memory 12, the addition operation unit 13, and the like shown in FIG.
Subtraction operation unit 14, division operation unit 15, and pupil extraction unit 16
Is replaced by an addition circuit 63, a subtraction circuit 64, a division circuit 65, and a comparator 66. Note that the pupil position detection unit 68 performs an image gravity center calculation in the same manner as in each of the above-described embodiments, and this calculation is realized by an electronic computer.

The addition circuit 63 calculates the sum of the video luminance signals S _ii of the two types of images I ₁ and I ₂ obtained from the imaging device 4. The subtraction circuit 64 calculates the difference between the video luminance signals of the two types of images I ₁ and I ₂ obtained from the imaging device 4. The division circuit 65 divides output signals of the addition circuit 63 and the subtraction circuit 64 from each other. The comparator 66 is a circuit that performs a binarization process on an output signal of the division circuit 65. As described above, since the addition operation, the subtraction operation, and the binarization process are performed by the analog circuit, the first
As compared with the embodiment, the amount of digital processing by the computer can be reduced, and a large number of image memories need not be used.

Further, the present embodiment also has a synchronizing signal generator 69 similar to that of the third embodiment.
The two image sensors 6, 7 (see FIG. 2) are driven in synchronization. Image luminance signals at the same time, which are output from the imaging device 4 and obtained by the respective imaging devices 6 and 7, are added to the addition circuit 6.
3 and the subtraction circuit 64, and the outputs of the addition circuit 63 and the subtraction circuit 64 are processed by the division circuit 65, thereby performing the operation based on the equation (1) described in the first embodiment.

The output of the division circuit 65 is
_Are stored in the image memory 67 in synchronization with the synchronizing signal _Ssy from the synchronizing signal generator 69, and the pupil position is detected by the pupil position detecting unit 68 performing image gravity center calculation.

The principle of pupil extraction is the same as in the first embodiment, and the division circuit 65 outputs a result based on equation (1).

When the face of the observer 20 at a distance of 2 m from the viewpoint position detecting device is photographed using the viewpoint position detecting device of the present embodiment, the pupil components (coordinates (x _P , y
_P)), and the face component (coordinates (x _F, y _F)) each brightness at that _{I 1 (x P, y P} ) = 50 ( arbitrary units; hereinafter the _{same), I 2 (x P,} y P) = 0, I ₁ (x _F , y _F ) =
40, I ₂ (X _F , Y _F ) = 30. At this time,
The result of the luminance comparison operation of the pupil component and the face component is I (x
_{P, y P) = 1.00,} I (x F, a y _F) = 0.14, the signal-to-noise ratio is 7. Therefore, the threshold value of the binarization process was set to 0.5, and only the pupil could be completely extracted.

Here, when the observer 20 uses glasses, the luminance of the pupil component decreases from 50 to 10, and each luminance becomes I _{1
(X P, y P) =} 10, I 2 (x P, y P) = 0, I 1
_{(X F, y F) =} 40, I 2 (x F, y F) became = 30. In this case, the result of the luminance comparison operation of each component is I (x
_P , y _P ) = 1.00, I (x _F , y _F ) = 0.14
And the signal-to-noise ratio is 7
It was still. Since there was no change in the signal-to-noise ratio, the binarization process was performed with the threshold value set to 0.5 in the same manner as in the case where no glasses were used, and only the pupil could be completely extracted.

(Sixth Embodiment) FIG. 11 is a block diagram of a viewpoint position detecting apparatus according to a sixth embodiment of the present invention.

The viewpoint position detecting device according to the present embodiment has a fifth
The subtraction circuit 64 (see FIG. 10) described in the embodiment is omitted.
Obtained by using the first lighting device 1 of the imaging device 4
Image I ₁ Image luminance signal S_iiDirectly into the division circuit 65.
It has been modified to work. Therefore, this implementation
In the embodiment, the adding circuit 63 and the dividing circuit 65
As in the case of the second embodiment, the calculation result shown in the equation (2) is output.
can do. Fifth implementation for other configurations
Since it is the same as the embodiment, the same reference numerals as those in FIG.
Their description is omitted.

The principle and effect of detecting the viewpoint position are the same as in the fifth embodiment, and a description thereof will be omitted.

(Example of Application to Stereoscopic Display Device) The viewpoint position detecting device of the present invention can be applied to a viewpoint-following type stereoscopic display device.

FIG. 12 is a diagram showing the overall configuration of a viewpoint following type stereoscopic display device using the viewpoint position detecting device of the present invention. This viewpoint-following type stereoscopic display device projects an image having binocular parallax from a liquid crystal projector 81 onto a lenticular screen 82, and spatially separates and presents images having binocular parallax to the left and right eyes. It realizes the visual.

The lenticular screen 82 is provided with a viewpoint position detecting device 80, and the viewpoint position detecting device 80 detects the viewpoint position of the observer 20. As the viewpoint position detecting device 80, any of the viewpoint position detecting devices of the present invention described in the first to sixth embodiments is used. An optical system control unit 83 is connected to the viewpoint position detecting device 80, and the viewpoint following optical system mechanism unit 8 in the liquid crystal projector 81 according to the viewpoint position of the observer 20.
4 is controlled. Although two human pupils are extracted by the viewpoint position detecting device 80, when applied to a viewpoint-following stereoscopic display device, the midpoint of the two pupils is set as the viewpoint position.

As described above, when the viewpoint position detecting device 80 according to the present invention is applied to a viewpoint-following three-dimensional display device,
It is possible to accurately detect the viewpoint position without contact,
A three-dimensional display device with a small burden on the observer 20 could be provided.

[0075]

As described above, according to the present invention, the first image, which is the image of the face of the subject illuminated by the first illuminating means, and the subject illuminated by the second illuminating means, Using the second image, which is an image of the face of, the ratio of the addition result to the subtraction result, or the ratio of the addition result to the first image, for the luminance of the corresponding pixel is determined for each of these two types of images. By obtaining for each pixel, even when the pupil reflection luminance of the subject fluctuates, the pupil of the subject can be stably extracted,
An accurate viewpoint position can be detected.

In particular, it is possible to simplify the structure of the image pickup means by irradiating light of the same wavelength to each of the illuminating means and alternately emitting light at a constant period.

Further, if the viewpoint position detecting device of the present invention is applied to a viewpoint-following type stereoscopic display device, the viewpoint position can be accurately detected in a non-contact manner, so that the burden on the observer of the stereoscopic display device is reduced. be able to.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a first embodiment of a viewpoint position detection device of the present invention.

FIG. 2 is a configuration diagram of an imaging device of the viewpoint position detection device shown in FIG.

FIG. 3 is a diagram illustrating an image captured by the imaging device illustrated in FIG. 2;

FIG. 4 is a graph showing a luminance distribution of an image between line segments AB in the image shown in FIG. 3;

FIG. 5 is a configuration diagram of a second embodiment of the viewpoint position detecting device of the present invention.

FIG. 6 is a configuration diagram of a third embodiment of a viewpoint position detecting device of the present invention.

7 is a configuration diagram of an imaging device of the viewpoint position detection device shown in FIG.

8 is a timing chart of the operation of the illumination device and the image memory with respect to a synchronization signal in the viewpoint detection device shown in FIG.

FIG. 9 is a configuration diagram of a fourth embodiment of the viewpoint position detection device of the present invention.

FIG. 10 is a configuration diagram of a fifth embodiment of a viewpoint position detecting device of the present invention.

FIG. 11 is a configuration diagram of a sixth embodiment of the viewpoint position detecting device of the present invention.

FIG. 12 is a diagram showing an overall configuration of a viewpoint tracking type stereoscopic display device to which the viewpoint position detection device of the present invention is applied.

FIG. 13 is a configuration diagram of a conventional pupil detection device.

FIG. 14 is a diagram illustrating an image captured by an imaging device of the pupil detection device illustrated in FIG. 13;

FIG. 15 is a graph showing distribution luminance of an image between line segments AB in the image shown in FIG. 14;

[Explanation of symbols]

 1,41 First illumination device 2,42 Second illumination device 3 Half mirror 4,44 Imaging device 5 Dichroic prism 6 First imaging device 7 Second imaging device 8,48 Lens 9,49 Filter 10,30 Image processing device 11, 31 First image memory 12, 32 Second image memory 13, 34 Addition operation unit 14 Subtraction operation unit 15, 35 Division operation unit 16, 36 Pupil extraction unit 17, 37, 68 Pupil position detection unit Reference Signs List 20 observer 20a pupil part 20b face part 46 image sensor 51, 69 synchronization signal generator 52 illumination controller 63 addition circuit 64 subtraction circuit 65 division circuit 66 comparator 67 image memory 80 viewpoint position detection device 81 liquid crystal projector 82 lenticular screen 83 optical System control unit 84 Viewpoint tracking optical system mechanism

Continuation of the front page (51) Int.Cl. ⁶ Identification symbol FI G06T 1/00 G06F 15/62 380 7/60 15/64 L 15/70 350Z (56) References JP-A-8-328170 (JP, A JP-A-8-313843 (JP, A) JP-A-7-311364 (JP, A) JP-A-7-13105 (JP, A) JP-A-9-230286 (JP, A) 54376 (JP, A) JP-A-8-322068 (JP, A) JP-A-2-138673 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G01B 11/00

Claims (8)

(57) [Claims] 1. An image pickup means for picking up an image of a face of a person to be detected, and a first illuminating means for illuminating the face of the person to be detected, the optical axis of which is substantially coaxial with the optical axis of the image pickup means. A second illumination unit whose optical axis is non-coaxial with the optical axis of the imaging unit; and an image captured by the imaging unit in a state where the subject is illuminated by the first illumination unit. The luminance of a pixel corresponding to a first image and a second image that is an image captured by the imaging unit in a state where the subject is illuminated by the second illumination unit is set for each pixel. Addition means for adding; subtraction means for subtracting, for each pixel, the luminance of a corresponding pixel between the first image and the second image; and calculation results obtained by the addition means and the subtraction means Dividing means for each pixel, Based on the division results in means, the viewpoint position detecting device having extraction means for extracting a portion corresponding to the pupil of the detected person. 2. The first illuminating means and the second illuminating means irradiate light of different wavelengths, and the imaging means illuminates incident light with light from the first illuminating means. A spectroscopic unit that separates the light from the second illuminating unit into light, a first imaging element to which the light from the first illuminating unit reaches among the light separated by the spectroscopic unit, A second image pickup element to which light from the illumination means reaches, an image picked up by the first image pickup element being the first image, and an image picked up by the second image pickup element The viewpoint position detecting device according to claim 1, wherein the second image is used. 3. The first illuminating means and the second illuminating means both emit light of the same wavelength and are alternately illuminated at a constant cycle, and when the first illuminating means emits light. 2. The viewpoint according to claim 1, wherein an image captured by the imaging unit is the first image, and an image captured by the imaging unit when the second illumination unit emits light is the second image. 3. Position detection device. 4. The viewpoint position detecting device according to claim 1, wherein said adding means, said subtracting means, said dividing means and said extracting means are each constituted by an analog circuit. 5. An image pickup means for picking up an image of the face of a person to be detected, and a first illuminating means for illuminating the face of the person to be detected, the optical axis of which is substantially coaxial with the optical axis of the image pickup means. A second illumination unit whose optical axis is non-coaxial with the optical axis of the imaging unit; and an image captured by the imaging unit in a state where the subject is illuminated by the first illumination unit. The luminance of a pixel corresponding to a first image and a second image that is an image captured by the imaging unit in a state where the subject is illuminated by the second illumination unit is set for each pixel. Adding means for adding, dividing means for dividing the brightness of the pixel corresponding to the first image and the calculation result obtained by the adding means for each pixel, and a dividing result based on the dividing result by the dividing means. Extracting means for extracting a portion corresponding to a pupil of the subject Viewpoint position detecting device having a. 6. The first illuminating means and the second illuminating means irradiate light of different wavelengths, and the imaging means illuminates incident light with light from the first illuminating means. A spectroscopic unit that separates the light from the second illuminating unit into light, a first imaging element to which the light from the first illuminating unit reaches among the light separated by the spectroscopic unit, A second image pickup element to which light from the illumination means reaches, an image picked up by the first image pickup element being the first image, and an image picked up by the second image pickup element The viewpoint position detecting apparatus according to claim 5, wherein the second image is used as the second image. 7. The first illuminating means and the second illuminating means both emit light of the same wavelength and emit light alternately at a constant period, and when the first illuminating means emits light, 6. The image captured by the imaging unit is the first image, and the image captured by the imaging unit when the second illumination unit emits light is the second image.
3. A viewpoint position detecting device according to claim 1. 8. The viewpoint position detecting device according to claim 5, wherein said adding means, said dividing means and said extracting means are each constituted by an analog circuit.