JP2000059666A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce fatigue to user of a head-mount display by making more speedy and comfortable the change in an image pickup direction concerning a head-mount display image pickup device. SOLUTION: A head-mount display 100 is composed of a CCD camera 3 (image pickup means) for picking up the image in a visual range 2 of a user 1 mounting the display, a display device 5 (image display means) for forming the image picked up by this CCD camera 3 on the retina 4a of the user 1 of the display, a detecting means 6 for detecting a glance direction 2 of the user 1 of the display, and a drive means 7 for making the image pickup direction of the CCD camera 3 follow the glance direction 2 detected by the detecting means 6.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus configured to image a visual field range of an adherend and to form an image on the retina of the adherend. The present invention relates to a head-mounted display that is mounted to correct visual impairments such as the above.

[0002]

2. Description of the Related Art Conventionally, a head-mounted display type device has been developed as a device for correcting visual impairment such as low vision or poor color vision. FIG. 12 is a plan view showing a conventional head mounted display 110 for correcting visual impairment.

In FIG. 1, a head-mounted display 110 includes a CCD camera 113 that captures an image of a field of view 112 of an adherend 111, and an image captured by the CCD camera 113 is formed on a retina 114 of the adherend 111. It comprises a finder 115 to be operated and a main body 116 for integrally fixing these devices to the head of the adherend 111.

More specifically, a CCD camera 113 captures an image of an area 117 equivalent to the visual field area 112 of the adherend 111, and corrects the captured image to a color LCD.
The image is projected by 118, reflected by the mirror 119, and projected on a finder 115 facing the retina 114 of the adherend 111.

[0005]

However, in the conventional head mounted display 110 described above, the CC
Since the D camera 113 is fixed at the front position of the main body 116, the imaging direction of the CCD camera 113 is always only the front of the main body 116.

Therefore, when trying to change the imaging direction of the CCD camera 113, the adherend 111 turns his / her head in the direction in which the imaging is desired, and
It is necessary to change the direction of the CCD camera 113 every time, and it is difficult to change the imaging direction instantly, and there is a possibility that the fatigue at the time of mounting the head mounted display 110 is large and the practicality is reduced.

Accordingly, the present invention has been made in view of the above circumstances, and an object of the present invention is to make the change of the imaging direction in a head-mounted display type imaging apparatus quicker and more comfortable. I do.

[0008]

In order to achieve the above object, the invention according to claim 1 of the present application is directed to an image pickup means for picking up an image of a field of view of a person to be adhered, and an image picked up by the image pickup means. Image display means for forming an image on the retina of a wearer, detection means for detecting the line of sight of the adherend, and drive means for following the image pickup direction of the image pickup means to the line of sight detected by the detection means It consists of:

According to the first aspect of the present invention,
In order to detect the direction of the line of sight of the adherend and make the imaging direction of the imaging unit follow the direction of the line of sight, the adherend can project the desired direction on the image display unit without changing the direction of the entire apparatus. Can be.

According to a second aspect of the present invention, in the first aspect, the image pickup means, the image display means, the detection means, and the drive means are integrally held near an eyeball of the adherend. It has holding means.

[0011] According to the second aspect of the present invention,
Since the respective units constituting the imaging apparatus can be integrally mounted, it is easy to attach and detach the units.

According to a third aspect of the present invention, in the image pickup apparatus according to the first or second aspect, the image display means moves the center of the image picked up by the image pickup means in the line of sight of the adherend. It has a center moving means.

According to the third aspect of the present invention,
By moving the center of the image displayed by the image display means in the direction of the line of sight of the adherend, the adherent pays attention when changing the imaging direction of the imaging means to follow the direction of the line of sight of the adherend. It is possible to prevent a shift from occurring between the target object and the target object displayed on the image display means.

According to a fourth aspect of the present invention, in the image pickup apparatus according to the first, second or third aspect, a designation means for designating a subject displayed on the image display means, and the subject designated by the designation means And color notifying means for notifying the user of the color.

According to the fourth aspect of the invention,
Even if the adherend has a visual impairment such as color weakness, the color of the imaged object can be easily recognized.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment A first embodiment of the present invention will be described below in detail. FIG. 1 schematically shows the internal configuration of a head-mounted display 100 according to the present embodiment, and FIGS. 2 and 3 are perspective views showing the appearance thereof.

1 to 3, a head-mounted display 100 includes a CCD camera 3, 3 for capturing an image of a visual field 2 of a person 1, and an image captured by the CCD cameras 3, 3. Display devices 5 and 5 for forming an image on the retina 4a, detecting means 6 and 6 for detecting the line of sight of the adherend 1, and the CCD cameras 3 and 3 for detecting the line of sight detected by the detecting means 6 and 6. And driving means 7 for following the imaging direction. Note that, in the present embodiment, these units are operated by the power supplied from the power supply device 9.

The CCD cameras 3, 3 are imaging means having an imaging area substantially equal to the visual field areas 2, 2 of the adherend 1, and their driving directions are changed up, down, left and right by driving means 7, 7. is there. And this CCD camera 3,
In the present embodiment, two are provided on the front surface of the main body 8 and are covered with a transparent hemispherical hood 3a. Also, these two CCD cameras 3 and 3
The interval is set substantially equal to the interval between the eyeballs 4 and 4 of the adherend 1, and the arrangement is such that the line of sight of the adherend 1 and the imaging direction coincide. The CCD cameras 3 and 3 may be provided with an interval adjusting means so that the intervals between the CCD cameras 3 and 3 are adapted to the intervals between the eyes 4 and 4 of the adherend 1.

The display devices 5, 5 include a backlight 51, a color LCD 52, and a focus adjuster 53 built in the side of the main body 8, a reflecting mirror 54, a lens 56 disposed on the optical axis of the focus adjust 53, It comprises a reflecting mirror 55 and a finder 57 arranged to face the eyeball 4 of the adherend 1.

By the display devices 5 and 5, C
The image picked up by the CD camera 3 is displayed by a backlight 51 and a color LCD 52, and the focus is adjusted by a focus adjuster 53, and then displayed on a finder 57 through a reflecting mirror 54, a lens 56, and a reflecting mirror 55. Is done.

As the detection means 6, 6, a known technique can be used. In this embodiment, a system using infrared rays is employed. FIG. 4 illustrates the detecting means 6 and 6 according to the present embodiment in more detail.

As shown in FIGS. 4A to 4C, the detecting means 6 and 6 are provided with an infrared light source 60 disposed close to the eyeball 4.
The eye 4 is irradiated with infrared light from X and 60Y, and the reflected light is received by the photoelectric conversion elements 61R, 61L, 61U and 61D, and the movement of the iris to the left and right and up and down is detected.

More specifically, the photoelectric conversion elements 61R and 61L for detecting the left and right movement of the iris are arranged so as to receive the reflected light from the right and left sides of the iris, respectively, with the iris facing the front. The output is amplified by the amplifier 63. Further, the photoelectric conversion elements 61U and 61D for detecting the vertical movement of the iris are arranged so as to receive the reflected light from the diagonally lower position of the iris in a state where the iris faces the front. Amplified by The outputs from the amplifiers 62 and 63 are analyzed by an analysis circuit (not shown), and the line of sight of the adherend 1 is specified.

The driving units 7 and 7 are arranged so that the imaging directions of the CCD cameras 3 and 3 follow the line of sight of the adherend 1 detected by the detection units 6 and 6.
Is turned up and down and left and right. FIGS. 5 and 6 show a specific configuration of the driving units 7 and 7. 5 is a perspective view of the driving means 7, FIG. 6 (a) shows a stopped state of the driving means 7, and FIG. 6 (b) shows a rotating state thereof.

As shown in FIG. 5 and FIG.
The camera 3 is protruded from a cubic base 70, and the base 70 is mounted on an inner frame 71 by a horizontal shaft 70a.
Is mounted so as to be able to rotate vertically. A motor 73 is provided on the base 70, whereby the motor 73 is vertically rotated with respect to the internal frame 71, and this vertical rotation is braked by a brake mechanism 75.

The brake mechanism 75 includes a horizontal shaft 70a
And a permanent magnet 75a inserted into the base 70 and fixed to the base 70 side,
It comprises an electromagnet 75b slidably inserted into the horizontal shaft 70a and opposed to the permanent magnet 75a so as to magnetically repel it, and a disc spring 75c for pressing the electromagnet 75b against the base 70 side.

To rotate the base 70 up and down,
As shown in FIG. 6B, the electromagnet 75b is energized to magnetically repel it against the permanent magnet 75a.
The brake is released with the 5b separated from the permanent magnet 75a, and the motor 73 rotates. When the power supply to the electromagnet 75b is cut off, the magnetic repulsion disappears, and the disc spring 75c
The electromagnet 75b is pressed against the permanent magnet 75a by the elastic force of the above, and the rotation of the base 70 is braked.

Further, similarly to the base 70, the inner frame 71 is also attached to the outer frame 72 by a vertical shaft 71a so as to be rotatable left and right. Further, a motor 74 is also provided on the inner frame 71, whereby the inner frame 71 is rotated left and right with respect to the outer frame 72, and a brake mechanism 76 having the same configuration as the brake mechanism 75 is used. The rotation is controlled.

In this embodiment, the CCD cameras 3 and 3, the display devices 5 and 5, and the detecting means 6 and 6 are used.
The driving means 7, 7 are held integrally with the main body 8 in the vicinity of the eyeball 4 of the adherend 1.

The main body 8 is provided with fixing belts 81 and 83 for holding the main body 8 on the head of the adherend 1. The fixed belt 81 fixes the main body 8 at the upper head of the adherend 1, and the fixed belt 83 fixes the main body 8 at the back of the adherent 1. Also,
These fixed belts 81, 83 are provided with dial type adjusters 81a, 83a for adjusting the tightening thereof.

According to the head-mounted display 100 according to this embodiment, the direction of the line of sight of the adherend 1 can be detected, and the imaging direction of the CCD cameras 3 and 3 can be made to follow the direction of the line of sight. The adherend 1 can project a desired direction on the display device 5 without changing the direction of the entire device.

In this embodiment, the CCD camera 3
Since the interval of 3 and the interval of the eyes 4 and 4 of the adherend 1 are provided to be substantially equal, the images displayed by the display devices 5 and 5 can be more realistic. .

Further, in the present embodiment, since the respective means constituting the head mounted display device 100 are integrally attached to the main body 8, it is easy to attach and detach them.

Second Embodiment Hereinafter, a second embodiment of the present invention will be described in detail. In the present embodiment, a method of displaying an image displayed on the display device 5 in the first embodiment will be described in detail. The overall configuration of the head mounted display according to the second embodiment is the same as that of the head mounted display 100 according to the first embodiment.

In particular, the display device according to the second embodiment is an image center moving means (not shown) for moving the center of the image picked up by the CCD cameras 3 and 3 in the line of sight of the adherend 1. It has.

In the present embodiment, the image center moving means is an arithmetic means for processing an image displayed on the display device 5, and this arithmetic means can be realized by, for example, a CPU built in the main body 8. it can.

The processing of an image by the image center moving means will be described below with reference to FIG. In addition, FIG.
3 shows an image captured by the CCD camera 3 and an image displayed by the display device 5.

Now, as shown in FIG. 7 (a), the CCD camera 3 captures a cylinder 701 in the foreground at the center of the image. At this time, as shown in FIG. 7B, the column 701 is projected on the center of the finder 57 of the display device 5, and the adherend 1 pays attention to the center P1 of the finder 57.

Then, as shown in FIG. 4B, the line of sight is set to P so that the adherend 1 can pay attention to the cube 702 located behind.
Suppose you moved to 2. Then, the detecting means 6 detects a change in the line of sight of the adherend, and changes the imaging direction of the CCD camera 3 by the driving means 7.

When the imaging direction of the CCD camera 3 is changed, a cube 702 is captured at the center of the CCD camera 3 as shown in FIG. According to this, FIG.
As shown in (d), in the finder 57, the cube 70
2 is projected at the center P1 of the image, and a deviation occurs from P2 which the adherend is paying attention to.

At this time, the image center moving means calculates the displacement from the center P1 of the display image to the point of interest P2, and processes the image to be displayed. That is, as shown in FIG.
The center P1 of the displayed image is moved to the position P2,
The object captured by the camera 3 at its center is matched with the position P2 to which the adherend pays attention.

In this embodiment, for convenience, P1
After the shift between P2 and P2, the center of the display image is moved in order to correct this, but the change in the line of sight of the adherend 1 is detected and the imaging direction is changed. At the same time, if the display image is processed and the center of the display image is moved in parallel with the driving of the driving means 7, the display object can be smoothly projected in the line of sight of the adherend.
Can be avoided.

According to the head-mounted display according to the second embodiment having such an image center moving means, the center P1 of the image displayed on the display device 5 is displayed.
When the imaging direction of the CCD camera 3 is changed following the direction of the line of sight of the adherend to move the point in the line of sight P2 of the adherend, the point of interest of the adherend is displayed on the image display means. It is possible to prevent a shift from occurring with the object to be performed.

In the present embodiment, the image is processed by the arithmetic means as the means for moving the image center, but the present invention is not limited to this. For example, the finder 57 of the display device 5 is configured to be movable up, down, left, and right, and is made to follow the line of sight of the adherend,
The position of the finder 57 may be changed.

Further, the display devices 5 and 5 according to the second embodiment have image correction means for emphasizing and displaying the outline of the imaged subject. This image correction means,
For example, it can be realized by an arithmetic device such as a CPU provided in the main body 8.

As a method of such image correction, for example, C
A device that enhances the contrast of an image captured by the CD camera 3, inverts the hue, or extracts the contour of the subject to enhance the contour of the subject can be adopted.

As a method of extracting the contour of the subject, for example, a difference between a newly captured image and an image captured before a predetermined period (for example, one frame period) is obtained, or a background image is stored in advance. By taking the difference between this and the input image, it is possible to use a method of specifying a moving part in the image and performing processing such as noise removal, labeling, and feature extraction on this part.

Further, the head mounted display according to the second embodiment is provided with a color notifying means for notifying the color of the subject projected on the display device 5.
Specifically, as shown in FIG. 8A, for example, the color notification unit detects the line-of-sight direction P3 of the adherend 1 by the detection unit 6 and detects the subject 801 that the adherend 1 is paying attention to. Is identified and its color is analyzed.
The notification is made by displaying characters or the like on the display means 5 such as D ".

In addition, as a method of notifying the adherend of this color by using characters or the like as described above, it is also possible to notify the color by a speaker (not shown) incorporated in the main body 8, for example. .

Further, in the present embodiment, the subject whose color is to be identified is specified by detecting the direction of the line of sight of the adherend by the detecting means 6, but the present invention is not limited to this. Pointer device 9 as shown in FIG.
It can also be specified manually using 0.

That is, for example, by moving a knob 91 provided above a grip 93 of the pointer device 90 up, down, left and right, the finder 57 of the display device 5 is moved.
Is moved to an arbitrary subject 801 and the switch 92 is pressed, so that the point P3 is displayed.
Can be locked, the color of the subject is analyzed, and the color is notified by text, voice, or the like.

As an example for realizing such a function, a circuit block as shown in FIG. 9 will be described.

After the output from the CCD camera 3 as the imaging means is separated into three colors, the R, G, and B signals converted into digital signals by the A / D converters 94, 94, 94 are respectively level-converted. The signals are supplied to circuits 95, 95, 95. The outputs of the level circuits 95, 95, and 95 are supplied to a control unit 96, where the respective signal levels are compared, and the result is output from a speaker 99 via an amplifier 98 through a voice synthesis circuit 97 as necessary. By producing sound, the color information of the subject can be recognized by voice.
Also, information such as the state of the subject can be appropriately pronounced.

On the other hand, in order to output the recognized color information as character information as text information, the output from the control unit 96 is converted into a predetermined character code by a character code conversion unit 100, and then converted by a conversion unit 101. Is converted into a predetermined pattern character and output to the display device 5.

According to the head mounted display device provided with such a color notifying means, even if the adherend has a visual impairment such as color weakness, the obstruction can be compensated.

Third Embodiment Next, a third embodiment of the present invention will be described with reference to FIG. As shown in the figure, a head-mounted display 200 according to the present embodiment has a third imaging means 201 provided at the front of the main body 8 in the head-mounted display 100 according to the above-described first embodiment.

As the third imaging means 201, for example, a wide-angle camera capable of photographing a wider area than the above-mentioned CCD camera 3 can be used. Then, by appropriately switching between the image by the CCD camera 3 and the image by the second imaging means 201 and displaying them on the display device 5, it is possible to display an image according to the preference of the adherend.

Fourth Embodiment A fourth embodiment of the present invention will be described with reference to FIG. As shown in the figure, a head-mounted display 300 according to the present embodiment is provided with a CCD camera 3 and a driving means 7 provided separately from a main body 301, and these are imaged or driven by remote control. .

That is, the display device 5 and the detecting means 6 described above are integrally attached to the main body 301 which is a fixing means, and these means are integrally fixed near the eyeball of the adherend. And the driving means 7 are attached to a self-propelled means 302 provided separately and independently from the main body 301.

The self-propelled means 302 transmits an image taken by the imaging means 3 attached to the self-propelled part, which is remotely operated by the adherend. In this embodiment, the self-propelled means 302 is provided with a wide-angle camera 303 rotatably mounted on an upper part thereof, and an image capturing a wide range by this and an image of an adherend by the CCD camera 3 are provided. It is configured to be able to appropriately switch between an image that follows the line of sight.

According to the head mounted display 300 according to the fourth embodiment, since the image pickup position by the CCD camera 3 and the display position by the display device 5 can be separated, the position where the adherend is present The image of a place different from the above can be seen by the adherend. Also at this time, since the CCD camera 3 performs imaging in the direction of the line of sight of the adherend, the head-mounted display 300 provides realism in a small place where humans cannot enter or in a dangerous place. A rich image can be captured.

[0062]

As described above, according to the image pickup apparatus of the present invention, the change of the image pickup direction in the head-mounted display type image pickup apparatus can be made quicker and more comfortable, and the sense of reality is rich. Images can be captured, and fatigue at the time of mounting can be reduced.

[Brief description of the drawings]

FIG. 1 is a plan view of a head mounted display according to a first embodiment of the present invention.

FIG. 2 is a perspective view illustrating an appearance of a head mounted display according to the first embodiment.

FIG. 3 is a perspective view illustrating an appearance of a head mounted display according to the first embodiment.

FIG. 4 is an explanatory diagram illustrating the principle of a detection device in a head mounted display according to a second embodiment of the present invention.

FIG. 5 is an enlarged perspective view showing an imaging unit in a head mounted display according to a second embodiment of the present invention.

FIG. 6 is an explanatory diagram illustrating a driving principle of an imaging unit and a driving unit in a head mounted display according to a second embodiment of the present invention.

FIG. 7 is an explanatory diagram showing a driving principle of image display means in a head mounted display according to a second embodiment of the present invention.

FIG. 8 is an explanatory diagram of a color notifying unit in a head mounted display according to a second embodiment of the present invention.

FIG. 9 is a circuit block diagram for realizing a function of a color notifying unit in a head mounted display according to a second embodiment of the present invention.

FIG. 10 shows a plan view of a head mounted display according to a third embodiment of the present invention.

FIG. 11 shows a plan view of a head mounted display according to a fourth embodiment of the present invention.

FIG. 12 is a diagram showing a conventional head mounted display.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Adherent 2 Field of view 3 CCD camera (imaging means) 4 Eyeball 4a Retina 5 Display device (Image display means) 6 Detecting means 7 Driving means 8 Main body 9 Power supply device 100 Head mounted display

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yukito Yamada 3-12 Moriyacho, Kanagawa-ku, Yokohama-shi, Kanagawa Japan F-term (reference) 5C022 AA01 AB51 AB62 AB65 AC01 AC31 AC51 AC77 AC78 5C065 AA07 BB48 CC01 CC08 DD02 EE02 EE12 FF02 FF03 FF08

Claims (4)

[Claims] 1. An image pickup means for picking up an image of a field of view of an adherend; an image display means for forming an image picked up by the image pickup means on a retina of the adherend; An imaging apparatus comprising: a detection unit configured to detect a direction of a line of sight detected by the detection unit; 2. The imaging apparatus according to claim 1, further comprising a holding unit that integrally holds the imaging unit, the image display unit, the detection unit, and the driving unit near an eyeball of the adherend. An imaging device characterized by the above-mentioned. 3. The image pickup apparatus according to claim 1, wherein said image display means has an image center moving means for moving a center of an image picked up by said image pickup means in a direction of a line of sight of a wearer. An imaging device characterized by the above-mentioned. 4. An image pickup apparatus according to claim 1, 2 or 3, wherein a designation means for designating a subject displayed on said image display means, and a color notification for notifying a color of said subject designated by said designation means. And an imaging device.