JP2005172851A - Image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display apparatus which has an advantage of user friendliness by enabling a user to easily and securely check the user's environment. <P>SOLUTION: The image display apparatus 10 is equipped with a wearing section 12, a display unit 14, a plurality of cameras 20, a headphone 21, a controller 22, or the like. In the state that the wearing section 12 is worn on the head of the user, a screen 15 of the display unit 14 is arranged within a user's visual field and an image is displayed on the screen 15. The cameras 20 are composed of first to seventh cameras and are arranged so that the image of approximately all areas surrounding the user can be imaged and each image signal acquired by imaging is supplied to the controller 22 as an environment image signal which shows the user's environment. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image display device mounted on a head or a face.

Conventionally, an image display device called a head mounted display has been used.
Such an image display device includes a mounting unit that is mounted on the user's head or face, a screen that is supported by the mounting unit and that is mounted in the user's field of view while the mounting unit is mounted on the user, and an input video Image display means for displaying an image on a screen based on a signal, and using a liquid crystal panel as the screen (for example, see Patent Document 1) or using a holographic optical element (for example, see Patent Document 2) ) Has been proposed.
Such an image display apparatus is configured to input a video signal supplied from, for example, a video player, a DVD player, a tuner that receives a television broadcast, or a game device and display an image on the screen. .
JP-A-6-78248 JP 10-301055 A

By the way, when such an image display device is mounted and an image is being viewed, if the user needs to check the situation of his / her surroundings, for example, the optical system including the liquid crystal panel constituting the screen is in a transparent state. The display is switched to (see-through state), and an image of the outside world is visually recognized through the optical system.
However, when such an optical system is used, the light is reduced and the image of the outside world becomes dark, which is disadvantageous in clearly visualizing the outside world. In addition, in order to confirm the surrounding situation, it is necessary to move around the head and observe the surroundings. While observing the image displayed on the screen based on the input video signal, It was difficult to check the situation, and it was not easy to use.
The present invention has been made in view of such circumstances, and the object of the present invention is to allow the user to easily and reliably confirm the situation of his / her surroundings without moving his / her head, and to be displayed on the screen. It is an object of the present invention to provide an image display device that is advantageous in improving usability, such as being able to confirm the surrounding situation while visually recognizing an image.

  In order to achieve the above-described object, an imaging apparatus of the present invention includes a mounting unit that is mounted on a user's head or face, and the user's head in a state that the mounting unit is supported by the mounting unit and the mounting unit is mounted on the user. An image display device having a screen arranged in a field of view and an image display means for displaying an image on the screen based on an input video signal, wherein the mounting unit or the mounting unit supported by the screen is provided to the user. An image pickup means for picking up an image of a part or almost the entire area of the user in a mounted state and generating a surrounding video signal representing the situation of the user; and an externally supplied video signal supplied from the outside of the image display device And generating the input video signal based on both the ambient video signal supplied from the imaging means and supplying the input video signal to the image display means, A first image by Kigaibu supply video signals, characterized in that it comprises a control means for displaying a second image by the surrounding video signal signal to the screen.

  Therefore, according to the present invention, an input video signal is generated based on both an externally supplied video signal supplied from the outside of the image display device and an ambient video signal supplied from the photographing means and supplied to the image display means. As a result, the first image based on the externally supplied video signal and the second image based on the ambient video signal signal are displayed on the screen, so the user needs to check his / her surrounding conditions. In such a case, if the second image is displayed on the screen, the surrounding situation can be surely visually recognized without moving the head. Further, if the second image is displayed while displaying the first image on the screen, the user can check the surrounding situation while viewing the first image, and the usability can be improved.

  For the purpose of enabling the user to easily and reliably confirm the surrounding situation of the user, an imaging means for generating a surrounding video signal representing the surrounding situation of the user, a first image by an externally supplied video signal, This is realized by providing control means for displaying the second image based on the ambient video signal signal on the screen.

Next, Embodiment 1 of the present invention will be described with reference to the drawings.
1 is a perspective view showing a configuration of an image display apparatus according to a first embodiment, FIG. 2 is an explanatory diagram showing a state in which the image display apparatus according to the first embodiment is mounted, and FIG. 3 is an imaging unit in the image display apparatus according to the first embodiment. 4 is a block diagram showing the configuration of the image display apparatus according to the first embodiment, FIG. 5 is an explanatory diagram showing the configuration of the image display apparatus according to the first embodiment, and FIG. 6 is an image display according to the first embodiment. It is a block diagram of the optical system of an apparatus.
As shown in FIGS. 1 and 2, the image display apparatus 10 includes a mounting unit 12, a display unit 14, a plurality of cameras 20, headphones 21, a controller 22, and the like.
The mounting portion 12 includes a strip-shaped front plate 1202, two strip-shaped side plates 1204 that are connected to both longitudinal ends of the front plate 1202 in a bendable manner, and an expansion / contraction that connects the tips of the two side plates 1204. The belt 1206 is configured in a ring shape. Then, as shown in FIG. 2, the user U's head is positioned inside the mounting portion 12, and the front plate 1202, the two side plates 1204, and the belt 1206 are respectively placed on the front head, the left and right side heads, and the back head. The mounting portion 12 is mounted on the head by adjusting the expansion and contraction of the belt 1206 in the applied state.

The display unit 14 has a support body 1402 formed in a goggle shape, and the upper center of the support body 1402 is connected to the front center of the front plate 1202 of the mounting part 12 via a connection part 1404.
As shown in FIGS. 5 and 6, the support 1402 includes a backlight 1406, an LCD 1408 that is a liquid crystal display panel for left and right eyes, a left and right eye half mirror 1410, a left and right eye concave half mirror 1412, and a liquid crystal shutter. 1413 and a lens 1414 are provided.
The backlight 1406 is configured to illuminate from above the left-and-right LCD 1408. Based on an input video signal from the controller 22 described later, the left-eye LCD 1408 displays a left-eye image and the right-eye LCD 1408. The right eye image is displayed on the LCD 1408. That is, the backlight 1406 and the LCD 1408 constitute image display means.
The liquid crystal shutter 1413 constitutes shutter means, and is disposed so as to cover the entire area of the concave half mirror 1412 in front of the concave half mirror 1412. At least a light transmitting state and a light non-transmitting state (not transmitting light) It is configured to be switchable to a shielding state. A mechanical shutter can also be used as the shutter means. Further, it is optional to provide a protective cover made of a material that can transmit light so as to cover the front of the liquid crystal shutter 1413.
The mounting unit 12 is mounted on the head of the user U, the liquid crystal shutter 1413 is impermeable, and the images displayed on the LCDs 1408 and 1408 are reflected by the half mirrors 1410 and 1410, and further enlarged by the concave half mirrors 1412 and 1412. Then, the light is reflected again, passes through the half mirrors 1410 and 1410, and forms an image on the eyes of the user via the lens 1414. That is, the screen 15 is constituted by the half mirror 1410 and the concave half mirror 1412, and this screen 15 is arranged in the field of view of the user U with the mounting portion 12 mounted on the user U.
In this embodiment, the image formed by the user's eyes through the screen 15 is a virtual image.
In this way, the size of the screen viewed by the user's eyes can be represented by the ratio of the viewing angle when the image on the LCD 1408 enters the eye and the total viewing angle viewed by the eye. For example, if the viewing angle is set to 30 degrees, it can be converted into a virtual image distance of 2 meters, which corresponds to viewing a large 52-inch screen.
In addition, since the concave half mirror 1412 has a see-through function to transmit light, when the liquid crystal shutter 1413 is in a transmission state, the concave half mirror can be used to view the external environment at the same time while viewing an image from the LCD 1408. 1412 and the liquid crystal shutter 1413 can be seen.

As shown in FIGS. 1 and 2, the camera 20 is composed of a total of seven cameras, ie, first to seventh cameras 2002, 2004, 2006, 2008, 2010, 2012, and 2014, and constitutes imaging means. Each camera 20 is based on an imaging optical system (not shown), an imaging element such as a CCD or CMOS sensor that captures an image formed by the imaging optical system, and an imaging signal output from the imaging element. A signal processing circuit for generating a video signal is provided.
Each camera 20 is arranged as follows in a state in which the mounting unit 12 is mounted on the user U.
The first camera 2002 is provided at the front left end of the support 1402 and its optical axis is directed forward.
The second camera 2004 is provided at the front right end of the support 1402 and its optical axis is directed forward.
The third camera 2006 is provided at the front portion of the connecting portion 1404 and its optical axis is directed forward.
The fourth camera 2008 is provided near the front side of the left side plate 1204, and its optical axis is directed to the left side.
The fifth camera 2010 is provided near the front side of the right side plate 1204 and its optical axis is directed to the right side.
The sixth camera 2012 is provided near the rear side of the left side plate 1204 and its optical axis is directed rearward.
The seventh camera 2014 is provided near the rear side of the right side plate 1204 and its optical axis is directed rearward.
In this embodiment, as shown in FIG. 3, the angle of view (viewing angle) of the photographing optical system of each camera 20 is set as follows. In FIG. 3, symbol F indicates the front, symbol L indicates the left, symbol R indicates the right, symbol B indicates the rear, symbol W indicates the wide angle, symbol S indicates the standard, and symbol Z indicates the telephoto.
1st camera 2002 (FW): Wide angle 2nd camera 2004 (FZ): Telephoto 3rd camera 2006 (FS): Standard 4th camera 2008 (LW): Wide angle 5th camera 2010 (RW): Wide angle 6th camera 2012 ( BW): Wide angle Seventh camera 2014 (BZ): Telephoto A part or all of the plurality of cameras 20 may be configured by a zoom-type photographing optical system capable of adjusting a photographing magnification (lens magnification). The magnification adjustment (zoom adjustment) may be performed manually, or may be performed by providing the camera 20 with a zoom adjustment mechanism having a drive source such as a motor and controlling the zoom adjustment mechanism from the central control unit 2202 controller 22. Also good.
Further, a pan head may be provided in a part or all of the plurality of cameras 20 so that the directing direction (shooting direction) can be changed. The directivity direction may be changed manually, or by providing a direction adjustment mechanism having a drive source such as a motor on the pan head and controlling the direction adjustment mechanism from the controller 22.
Further, if a camera 20 that is directed upward in the vertical direction is provided, the situation above the user U can be imaged, and for example, the direction in which the head is facing when the head is tilted nearly 90 degrees This situation can be imaged by the camera 20, which is advantageous for imaging a wide range of situations.
These seven cameras 20 are arranged so as to be able to image almost the entire area around the user U, and each video signal obtained by imaging is a controller 22 as an ambient video signal indicating a situation around the user U. It is comprised so that it may be supplied to.

  The headphones 21 are configured to be attachable to the left and right ears of the user U from below the left and right side plates 1204, respectively, and are configured to reproduce audio signals supplied from the controller 22.

Next, the configuration of the controller 22 will be described with reference to FIG.
The controller 22 includes a central control unit 2202 (corresponding to the control means in the claims), a RAM 2204, an interface unit 2206, an audio processing unit 2208, an operation unit 2209, a GPS sensor 2210, a walking sensor 2212, an Ethernet interface 2214, and the like. I have.
The interface unit 2206 has an input function for inputting an externally supplied video signal and an externally supplied audio signal supplied from an external device (TV tuner, DVD player, video player, etc.) 30, an ambient video signal from the camera 20, and a microphone 33. And an output function for outputting the audio signal to a recording device (recorder) 32.
The audio processing unit 2208 amplifies an audio signal supplied from the external device 30 via the interface unit 2206 and supplies the amplified audio signal to the headphones 21, supplies it to the speaker via the external terminal 34, or via the LINE terminal 36. It has an output function for outputting to an external device, and an output function for inputting an audio signal via the microphone 38 and outputting it to the recording device 32 via the interface unit 2206.
The operation unit 2209 includes a switch for instructing the central control unit 2202 to operate.
The GPS sensor unit 2210 has a function of measuring the position on the earth where the image display device 10 is present by receiving radio waves from a satellite in GPS (Global Positioning System) and generating the positioning data (position information). have.
The walking sensor 2212 is attached to the body of the user U and has a function of detecting whether the user U is in a walking state or a stationary state.
As such a walking sensor 2212, for example, there is one that includes a first acceleration sensor that detects vertical acceleration and a second acceleration sensor that detects acceleration in a traveling direction.
Specifically, as shown in FIG. 7, when vertical / traveling direction acceleration is detected, attention is paid to acceleration in the traveling direction in addition to the acceleration in the vertical direction, and estimation is performed by peak detection of the acceleration in the vertical direction. The acceleration in the direction of travel in the period of half a cycle from the start time of the walking motion that is performed is integrated, and if this value is equal to or greater than a certain value, it is detected that there was a walking motion (for example, the 12th IEICE Tech Report Times mixed reality study group, PRMU 2002-180, pp. 67-72 (2003)).
The Ethernet interface 2214 controls data communication between the central control unit 2202 and an external information processing apparatus.
The central control unit 2202 includes a microcomputer, a video signal processing circuit, and the like.
The central control unit 2202 has a function of inputting an ambient video signal supplied from each camera 20 and performing image processing, a function of supplying a video signal to the LCD 1408 based on an operation of the operation unit 2209, and a function of controlling the audio processing unit 2208. , A function of processing detection signals from the GPS sensor 2210 and the walking sensor 2212, a function of performing data communication with an external information processing apparatus via the Ethernet interface 2214, and the like.
More specifically, the central control unit 2202 generates an input video signal based on both the externally supplied video signal supplied from the external device 30 and the surrounding video signal supplied from each camera 20 and supplies the input video signal to the LCD 1408. Thus, a control unit is configured to display the first image based on the externally supplied video signal and the second image based on the ambient video signal signal on the screen 15.
Further, the central control unit 2202 is configured to select and supply the surrounding video signal supplied from each camera 20 to the LCD 1408 based on the operation of the operation unit 2209.
The RAM 2204 constitutes a working area required when the central control unit 2202 performs data processing and image processing.

Next, the operation of the image display device 10 configured as described above will be described.
First, an operation at the time of content viewing for viewing content (image or sound) will be described.
The user U connects an external device 30 such as a DVD player that reproduces content to the interface unit 2206, wears the mounting unit 12 of the image display device 10 on the head, and places the screen 15 in his field of view. Moreover, the headphones 21 are worn on the ears.
Next, when the user U performs an operation for selecting an externally supplied video signal supplied from the external device 30 on the operation unit 2209, the central control unit 2202 is based on the externally supplied video signal supplied from the external device 30. By generating the input video signal and supplying it to the LCD 1408, the first image based on the externally supplied video signal is displayed on the screen 15, and the externally supplied audio signal supplied from the external device 30 is sent via the audio processing unit 2208. To the headphones 21. Note that the liquid crystal shutter 1413 is in an opaque state.
As a result, the user U views the first image displayed on the screen 15 and listens to the sound output from the headphones 21.
Here, when the user U needs to check the surrounding situation, when the user U performs an operation for selecting the surrounding video signal supplied from each camera 20 on the operation unit 2209, the central control unit 2202 Then, an input video signal is generated based on the ambient video signal supplied from the camera 20 and supplied to the LCD 1408, whereby the second image based on the ambient video signal is displayed on the screen 15.
The second image can be displayed on the screen 15 as follows.
(1) Only an image based on one system of surrounding video signals selected from the seven systems of surrounding video signals supplied from each camera 20 is displayed on the screen 15.
(2) Each of the images of a plurality of surrounding video signals selected from the seven systems of surrounding video signals supplied from each camera 20 is simultaneously displayed on the screen 15.
Further, in each of the cases (1) and (2), the second image and the first image may be displayed on the screen 15 at the same time.
Further, the magnification of the first and second images displayed on the screen 15 may be arbitrarily set, and can be enlarged or reduced.
In addition, the central control unit 2202 constantly monitors whether the user U is in a walking state or a stationary state by the walking sensor 2212, and when determining that the user U has shifted from a stationary state to a walking state, Even if the first image is displayed on the screen 15 in the stationary state, the second image is forcibly displayed on the screen 15.

As described above, according to this embodiment, the input video signal is generated based on both the externally supplied video signal supplied from the outside of the image display device 10 and the ambient video signal supplied from the camera 20. Since the first image based on the externally supplied video signal and the second image based on the ambient video signal signal are displayed on the screen 15 by being supplied to the LCD 1408, the user U can change his / her surrounding situation. If the second image is displayed on the screen 15 when it is necessary to check the situation, it is possible to immediately and reliably visually recognize the surrounding situation without moving the head.
In addition, when the user U needs to check the situation around him / her, even if the liquid crystal shutter 1413 is in the transmissive state, the user U can visually recognize the situation in front. However, in this case, since the appearance of the outside world is viewed through the optical system including the concave half mirror 1412 and the liquid crystal shutter 1413, it is difficult to secure a sufficient field of view, and light transmitted by the optical system is reduced. Therefore, the external world which can be visually recognized becomes dark, and it becomes disadvantageous when ensuring visibility. On the other hand, in the present embodiment, since the second image displayed on the screen 15 can visually recognize the periphery of the user U over the entire circumference, the brightness of the image in the outside world can be ensured even in ensuring the field of view. This is also advantageous.
In addition, when the second image is displayed while displaying the first image on the screen 15, the user can check the surrounding situation while viewing the first image, thereby improving usability. Is advantageous.
Further, when the walking sensor 2212 determines that the user U has shifted from the stationary state to the walking state, even if the first image is displayed on the screen 15 in the stationary state, the second image is forcibly displayed. When configured to display on the screen 15, the user U can visually recognize the surrounding situation without performing a special operation, which is further advantageous in improving usability.

In addition, the photographic magnification of the camera 20 is set larger than that of the user U's naked eye, that is, the viewing direction is observed by setting the camera 20 to the telephoto side rather than viewing the external environment through the concave half mirror 1412 and the liquid crystal shutter 1413. It is possible to observe in detail the target object to be magnified from the naked eye.
In addition, the photographic magnification of the camera 20 is set to be smaller than that of the user U's naked eye, that is, to the wide angle side as compared with the case where the external environment is viewed through the concave half mirror 1412 and the liquid crystal shutter 1413, so It is possible to observe the desired visual field range at a wider angle than the naked eye.
Further, by setting the photographing magnification of the camera 20 to be the same as that of the user U's naked eye, that is, the same size as when viewing the external environment via the concave half mirror 1412 and the liquid crystal shutter 1413, the concave half mirror The problem that the amount of light is reduced when the front is visually recognized through the liquid crystal shutter 1413 and the liquid crystal shutter 1413 can be solved.
Furthermore, if the camera 20 has an infrared night vision camera function, the visibility of surrounding scenery at night can be improved.
Of course, if the plurality of cameras 20 are arranged so that the entire area around the user U, that is, 360 degrees can be imaged without gaps, the periphery of the user U can be continuously observed.
Further, in addition to the camera 20 that is oriented in the horizontal direction so that the entire celestial sphere around the user U can be photographed, the camera 20 that is oriented in the direction inclined up and down with respect to the horizontal direction, or in the vertical direction and in the vertical direction. May be provided.
Of the plurality of cameras 20, two cameras that are substantially parallel to each other and arranged at intervals in a direction orthogonal to the optical axis, for example, first and second cameras 2002 and 2004 directed in the front direction. Can be displayed on the basis of images obtained by these two cameras.
Further, another image pickup device such as a video camera can be connected to the image display device 10, and a video signal supplied from the image pickup device can be displayed on the screen 15 in the same manner as the surrounding video signal from the camera 20.
In the embodiment described above, the number of cameras 20 is not limited to seven, and the number of cameras 20 can be increased or decreased according to the imaging range and angle of view of each camera 20.
In addition, when the walking sensor 2212 determines that the user U has transitioned from the stationary state to the walking state, display of both the first image and the second image may be stopped.
Further, in this embodiment, the plurality of cameras 20 constituting the photographing unit is configured to be able to capture almost the entire area of the user U. However, it is only necessary that the situation around the user U can be captured. Of course, it may be configured to image a part of the periphery of U.

Next, a second embodiment of the present invention will be described with reference to the drawings.
The second embodiment is different from the first embodiment in that the imaging unit is composed of a single camera 20 </ b> A and a reflecting mirror 2016.
FIG. 8 is a perspective view showing the configuration of the image display apparatus according to the second embodiment. Hereinafter, the same or similar parts as those in the first embodiment are denoted by the same reference numerals.
As shown in FIG. 8, the camera 20 </ b> A is fixed to the tip of a support portion 1216 that extends obliquely upward and rearward from the center of the front plate 1202 of the mounting portion 12. More specifically, the camera 20A is disposed via the support portion 1216 so that the optical axis of the photographing optical system faces upward in the vertical direction with the mounting portion 12 mounted on the head.
A reflecting mirror 2016 is provided on the optical axis of the photographing optical system of the camera 20A, and the camera 20A is configured to capture an image reflected by the reflecting mirror 2016.
The reflecting mirror 2016 is formed in a conical shape having a central axis that coincides with the optical axis, and a cross section of which forms a quadratic curve, and extends 360 degrees around the central axis. In other words, the reflecting mirror 2016 is formed so as to face almost the entire circumference of the user U. It is configured to guide the situation in the range of 360 degrees around the central axis to the photographing optical system of the camera 20A.
An image captured by the camera 20A is an annularly curved image. Therefore, the central control unit 2202 performs predetermined image processing on the captured image. The image is converted into a visually natural image extending in a panoramic shape (a horizontally long rectangular shape).
The central control unit 2202 treats the visually natural image obtained by the predetermined image processing as the second image as in the case of the first embodiment, and displays it on the screen 15.
However, since the second image becomes a rectangular image extending 360 degrees around the central axis, if the second image is displayed on the screen 15 as it is, it becomes visually unnatural. Therefore, if the second image is divided into, for example, every 90 degrees about the central axis and divided into four images of the front, left and right sides, and the rear, and the images are switched appropriately and displayed visually. Natural and preferable.
It should be noted that a camera 20A using such a reflecting mirror 2016 and a technique for obtaining a visually natural image by performing the predetermined image processing on an image captured by such a camera 20A are conventionally known. .
According to the second embodiment, since the number of cameras 20A is only one in addition to the function and effect of the first embodiment, it is advantageous for downsizing and cost reduction.
Further, as the camera 20, a camera provided with a photographing optical system incorporating a fisheye lens having a viewing angle of about 180 degrees can be used. In this case, one of the two cameras is arranged so that its optical axis is directed forward, and the other camera is arranged so that its optical axis is directed rearward. Images captured by these two cameras are taken. A visually natural image may be generated by image processing and handled in the same manner as in the second embodiment. In this case, since the number of cameras 20A is only two in addition to the operational effects of the first embodiment, it is advantageous in reducing the size and cost.

Next, a third embodiment of the present invention will be described.
In the third embodiment, when it is necessary for the user U to check the surrounding situation, the reduction of the environment displayed on the screen 15 at the same time as the situation of the outside world is visually recognized via the liquid crystal shutter 1413 and the concave half mirror 1412 shown in FIG. The images are visible.
The liquid crystal shutter 1413 according to the third embodiment is configured so that the density, that is, the light transmittance can be changed under the control of the central control unit 2202. Therefore, by changing the transmittance, the contrast ratio between the external image transmitted through the liquid crystal shutter 1413 and the image by the screen 15 can be changed. Further, the liquid crystal shutter 1413 is configured not to change the light transmittance uniformly over the entire area but to control the transmittance of the divided areas independently for each area.
Next, functions and effects of the image display device 10 according to the third embodiment will be described.
FIG. 9A shows a state in which the first image based on the externally supplied video signal is displayed on the screen 15 with the liquid crystal shutter 1413 in a transmissive state or a semi-transmissive state. In this case, the external image 102 is visually recognized through the liquid crystal shutter 1413 and the concave half mirror 1412, and the first image 104 is displayed in a state where the first image 104 is superimposed on the central portion (the visual center) of the external image 102. .
In such a display state, it is extremely difficult to observe the details of the external image 102 and the first image 104 at the same time, and generally one of the images displayed twice is consciously concentrated and observed. In some cases, it is known that the recognition ability of the other video is significantly reduced.
Therefore, when it becomes necessary to observe the external image 102 in detail, only the external image 102 is displayed by temporarily stopping the display of the first image 104 on the screen 15 as shown in FIG. 9B. It can be made visible.
Further, as shown in FIG. 9C, if the first image 104 is configured to be reduced and displayed at the end of the field of view of the user U, the observation of the external image 102 is not hindered, and the line of sight Is moved to the first image 104 displayed in a reduced size, the outline of the image can be recognized. The visual field of the user U will be described in detail next.
The first image 104 to be reduced may be displayed in a see-through state in which the external image 102 is transmitted through the background, but the region of the first image 104 to be reduced and displayed by the liquid crystal shutter 1413 is partially opaque. Alternatively, the first image 104 can be easily viewed by reducing the transmittance.
On the contrary, by controlling the backlight 1406 and the LCD 1408 to reduce the luminance of the first image 104 to be reduced and displayed, it is possible to make the background external image 102 easier to see.
As shown in FIGS. 9B and 9C, if the user U can easily visually recognize the external image 102, it is advantageous for the user U to easily check the surrounding situation, and the user U It becomes possible for U to walk without hindering the observation of the external image 102 while wearing the image display device 10 on the head.

Next, the visual field of the user U will be described.
FIG. 10 shows a standard human visual field characteristic, according to which the normal field of view is 10 ° to 15 ° below the horizontal direction H (shown, normal line of sight when standing) To the normal line of sight when sitting.
FIG. 11 shows a standard human visual field characteristic. According to this figure, it can be seen that the visual field capable of recognizing characters is about 20 ° (shown in the character recognition limit range).
Therefore, in this embodiment, when the first image 104 is reduced and displayed at the end of the field of view of the user U, as shown in FIG. 12, a solid angle with a normal line of sight (10 ° below the horizontal direction) as the center C is obtained. The first image 104 is not displayed within 20 °. In other words, the first image 104 is reduced and displayed in a range excluding the range of the solid angle of 20 ° (FIG. 9C).
By doing so, as shown in FIG. 9C, the external image 102 can be viewed without being obstructed by the display image in the normal viewing direction, and when the first image 104 is desired to be viewed, the first viewing line is displayed. By moving to the image 104, the first image 104 and the external image 102 can be merged and viewed.
The range in which the first image 104 is not displayed is minimized to the above-described range of the solid angle of 20 °, and if the range in which the first image 104 is not displayed is widened, the user U's field of view can be ensured and the user U can walk.
Further, as shown in FIG. 9B, if the first image 104 is not displayed at all and only the external image 102 can be displayed, the user U can walk while viewing the external image 102.
In addition, as described above, the normal display state in which the first image 104 is displayed in the normal size of the user U without being reduced, and the first image 104 is reduced to the end of the visual field of the user U. The switching operation between the displayed state or the reduced or non-displayed state in which the first image 104 is hidden may be performed by manually operating a video switch provided in the operation unit 2209 or walking. The detection may be performed by detecting the walking of the user U by the sensor 2212.
When the video changeover switch and the walking sensor 2212 are used in combination, even if the user U operates the video changeover switch during walking, the switching operation based on the walking detection of the walking sensor 2212 is prioritized so that the normal display state cannot be changed. In addition, if the video switch can be operated after recognizing the stop of walking by the walking sensor 2212, the user U can surely see the external image 102 during walking.

Next, a fourth embodiment of the present invention will be described.
In the fourth embodiment, a switching operation between the normal display state and the reduced or non-display state is performed using the user U's line-of-sight detection.
FIG. 13 is a configuration diagram illustrating a simplified configuration of the image display apparatus 10 according to the fourth embodiment.
As shown in FIG. 13, a line-of-sight detection sensor 1416 is provided at a location of the half mirror 1410 facing the user U, and the line-of-sight detection sensor 1416 indicates in which region of the half mirror 1410 the line of sight of the user U is directed. In other words, it is configured to detect which area the user U is gazing at and to supply gazing area data indicating the detected area to the controller 22 (central control unit 2202). Such a line-of-sight detection method is conventionally known as a method for recognizing where in the viewfinder the user is gazing when looking through the viewfinder of the camera system.
In addition, a brightness sensor 1418 (for example, an optical sensor or a photodiode) is provided on the outer surface of a liquid crystal shutter 1413 formed of a liquid crystal shutter, and the brightness sensor 1418 detects the brightness of the outside world and detects the brightness indicating the brightness. The signal is configured to be supplied to the central control unit 2202.
Further, the liquid crystal shutter driver 1413A is configured to adjust the transmittance of the liquid crystal liquid crystal shutter 1413 and the operation range of the liquid crystal shutter 1413 (range in which the transmittance is adjusted) according to a command from the central control unit 2202.
The central control unit 2202 adjusts the operation range of the liquid crystal shutter 1413 so that only the portion of the first image 104 to be reduced is put on the shutter (decreasing the transmittance), thereby improving the visibility of the first image 104. It is configured so that it can be raised.
Further, the user U automatically adjusts the transmittance of the liquid crystal shutter 1413 via the liquid crystal shutter driver 1413A in accordance with the brightness detected by the brightness sensor 1418 by the central control unit 2202 so that the user U and the first image 102 The ratio of these brightnesses can be maintained in an appropriate range so that the image 104 can be seen in combination.
Further, as shown in FIG. 15, in the reduced or non-display state, an icon 108 indicating the re-display operation area 106 is displayed at the end of the field of view of the user U in the screen 15. When the gaze detection sensor 1416 detects that the user U is gazing at the icon 108, that is, the re-display operation area 106 for a predetermined time, the control unit 2202 moves the screen 15 from the reduced or non-display state to the normal state. In other words, the first image 104 is enlarged and displayed so as to switch to the display state.

Next, the operation of the fourth embodiment will be described with reference to the flowchart of FIG.
The central control unit 2202 monitors whether or not the user U is walking based on the detection signal of the walking sensor 2212 (step S10).
If no walking is detected, it is determined by the gaze detection sensor 1416 whether or not the user U gazes at the icon 108, that is, the redisplay operation area 106 for a predetermined time, that is, whether or not there is an image enlargement request (step S12). If there is no image enlargement request, the process returns to step S10, and if there is an image enlargement request, it is determined whether or not the normal display state is present, that is, whether or not the first image 104 is being enlarged (step S14). . If the enlarged display is being performed, the process returns to step S10. If the enlarged display is not being performed, the first image 104 is enlarged and displayed on the screen 15 (step S16).
If walking is detected in step S10, it is determined whether or not the first image 104 is being enlarged and displayed (step S18). If enlarged and displayed, the screen 15 is reduced or not displayed. The state is set (step S20), and if not enlarged display, the process returns to step S10.
Therefore, according to the fourth embodiment, since the switching operation between the normal display state and the reduced or non-display state is performed using the user's U line of sight detection, the user U can perform the switching operation without operating the switch. This is advantageous in improving operability.
Furthermore, when the user U is walking, the enlarged display of the first image 104 is stopped and no enlargement request is accepted, so that the visibility of the external image 102 can be ensured when the user U is walking.
Note that the redisplay operation area 106 may be represented by a reduced first image 104 instead of the icon 108.

Next, a fifth embodiment of the present invention will be described.
In the fifth embodiment, both the first image and the second image are displayed on the screen 15 at the same time.
As shown in FIG. 16, the external image 102 is visually recognized through the liquid crystal shutter 1413 and the concave half mirror 1412 when the liquid crystal shutter 1413 is set to the transmissive state. Then, the first image 104 based on the externally supplied video signal is displayed in a superimposed state at the center location (view center) of the external image 102, and the second image is displayed at the end of the external image 102 (the end of the field of view of the user U). The image 110 is displayed. The second image 110 corresponds to almost the entire area of the external image 102 and is supplied from, for example, the first camera 2002 (FW) of the camera 20 that is set to a wide angle that faces the front. This is based on the surrounding video signal, and is reduced and displayed so as to be displayed on the screen 15.
Therefore, according to the present embodiment, since the second image 110 corresponding to the external image 102 is reduced and displayed on the edge of the screen 15, part or all of the external image 102 is hidden by the first image 104. Even if it is, the surrounding situation can be confirmed by moving the line of sight to the second image 110, and the surrounding situation can be visually recognized without the user U performing a special operation. Is advantageous.
In addition, the second image 110 displayed on the edge of the screen 15 is not limited to an image corresponding to almost the entire area of the external image 102 as described above, and is not limited to the external image hidden by the first image 104. Even if only the image corresponding to the portion of the image 102 is reduced or displayed at the same magnification as that of the external image 102, the same effect as described above can be obtained.

Next, a sixth embodiment of the present invention will be described.
In the sixth embodiment, the second image 110 is superimposed on the external image 102 and displayed.
As shown in FIG. 17A, the external image 102 is visually recognized through the liquid crystal shutter 1413 and the concave half mirror 1412 when the liquid crystal shutter 1413 is set to the transmissive state. At this time, it is assumed that the user U is stationary.
Here, when the user U operates a video switching switch provided in the operation unit 2209, as shown in FIG. 17B, for example, the second camera 2004 ( FZ) is displayed in a state where the second image 110 based on the surrounding video signal is superimposed on the external image 102 at the center of the screen 15.
Then, when the user U starts walking from a stationary state, walking is detected by the walking sensor 2212, whereby the central control unit 2202 displays the second image 110 on the end of the screen 15 (see FIG. 17C). The image is reduced and displayed on the edge of the visual field of the user U.
Therefore, according to the present embodiment, the object that the user U wants to observe can be enlarged and displayed as the second image 110 by the naked eye and can be viewed in detail. In addition, when the user U is walking, the field of view can be ensured by reducing the second image 110 at the end of the field of view without troublesome operations.

It should be noted that the following configuration can be used for displaying the second image 110.
(1) The camera 20 is configured to be a zoom type whose shooting magnification can be adjusted, and the shooting magnification can be adjusted by the central control unit 2202. If the line of sight of the user U is directed in a certain direction for a predetermined time by the line of sight detection sensor 1416, it is determined that the user U is gazing at an object in that direction, and enlargement by the camera 20 zoom is automatically performed.
If comprised in this way, the user U can enlarge and visually recognize the image of an object, without performing troublesome operation.
(2) The camera 20 is provided with a so-called pantilter function that can change the imaging direction in an arbitrary direction, and is configured to display an image in an arbitrary direction on the screen 15 under the control of the central control unit 2202. The pantilter direction control can be performed by operating a joystick or a cross key provided in the operation unit 2209, for example, or can be performed by a voice recognition device or the like provided by a voice recognition device. Alternatively, the above-described line-of-sight detection sensor 1416 may be used to control the direction of the pantilter so that the center of the image displayed in the second image 110 matches the line-of-sight direction of the user U.
(3) The designated object may be recognized by a method such as image recognition, and so-called automatic tracking may be performed in which the camera 20 automatically faces the object.
(4) The camera 20 may have different shooting magnifications as described above. For example, if the surrounding video signals from the cameras 20 with different shooting magnifications directed in the same direction are selected and displayed on the screen 15, the same video display magnification can be switched and observed.

(5) The second image 110 displayed on the screen 15 is not limited to the image of one camera 20 and may be images of a plurality of cameras 20.
For example, as shown in FIG. 18, the first camera 2002 (FW), the second camera 2004 (FZ), the third camera 2006 (FS), the fourth camera 2008 (LW), the fifth camera 2010 (RW), the first camera Seven types of second images 110 from the six cameras 2012 (BW) and the seventh camera 2014 (BZ) are simultaneously displayed on the screen 15.
Then, the second image 110 by the selected second camera 2004 (FZ) is enlarged and displayed at the center of the screen 15, and the names of the remaining six cameras 20 (FW, FS, LW, RW, BW, BZ) are displayed. Each icon 110 </ b> A is displayed at the end of the screen 15 (the end of the field of view) instead of the second image 110 by each camera 20. Of course, the second image 110 from each camera 20 may be reduced and displayed instead of each icon 110A.
Of the second images 110 by the plurality of cameras 20 displayed in a reduced scale, selection of which camera 20 the second image 110 is enlarged and displayed in the center is performed by selecting a switch provided in the operation unit 2209 or performing voice recognition. The selection may be performed by a system or the like, but the above-described line-of-sight detection sensor 1416 may be used. That is, when the user U gazes at the icon 110A to be enlarged and displayed or the reduced second image 110 for a predetermined time, the second of the camera 20 corresponding to the icon 110A or the reduced second image 110 is displayed. The image 110 may be enlarged and displayed. According to such a configuration, the selection operation of the plurality of cameras 20 can be easily performed, which is advantageous in improving operability.

(6) As shown in FIG. 19, on the screen 15, one second image 110 is enlarged and displayed, and only one second image 110 is reduced and displayed. The second image 110 is selected by a switch of the operation unit 2209, a voice recognition system, or the like. Then, execution or non-execution of the enlarged display of the selected second reduced image 110 may be performed using the above-described line-of-sight detection sensor 1416. That is, when the user U gazes at the reduced second image 110 that he / she wants to enlarge and displays for a predetermined time, the second image 110 of the camera 20 corresponding to the reduced second image 110 is displayed. An enlarged display may be performed.
According to such a configuration, the number of second images 110 displayed in a reduced size can be reduced as compared with FIG. 18, so that the portion of the external image 102 hidden by the second image 110 displayed in a reduced size can be reduced. This is advantageous in securing the area of the external image 102 that can be reduced.

(7) The second image 110 to be enlarged and displayed on the screen 15 without performing the reduced display of the second image 110 shown in FIGS. 18 and 19 is selected by a changeover switch of the operation unit 2209 or an instruction by voice recognition. If the second image 110 is switched to the second image 110 by the camera 20, the portion of the external image 102 hidden by the reduced second image 110 is eliminated, so that the area of the external image 102 that can be visually recognized is secured. Further advantageous.
Of the second images 110 taken by the plurality of cameras 20 that capture the surroundings of the user U, the selection of which camera 20 the second image 110 to be displayed on the screen 15 is enlarged is, for example, a joystick provided on the operation unit 2209 It is possible by an operation in the left and right directions by the user U using an operation such as a cross key or a voice recognition device. Or you may make it select the 2nd image 110 by the camera 20 which orient | assigns the same direction as the gaze direction of the user U using the above-mentioned gaze detection sensor 1416. FIG. In this case, it is desirable that image areas that overlap each other in the second images 110 by the cameras 20 displayed on the screen 15 are continuously displayed by image processing.
In any of the configurations (1) to (7) described above, when the walking of the user U is detected, at least the display of the image in the normal visual field center range should be stopped. This is preferable for ensuring visual recognition.

Next, Example 7 will be described.
In the seventh embodiment, an image captured by the camera 20 is recorded in a recording device.
As shown in FIG. 4, in this embodiment, the recording device 32 is connected to the interface unit 2206 of the controller 22, and the ambient video signal from the camera 20 and the audio signal from the microphone 33 are output to the recording device 32. 32 is configured so that video and audio can be recorded.
The recording device 32 may be any of a magnetic recording device, an optical disk recording device, a semiconductor memory recording device, etc. used in a video camera.
The recording device 32 may be integrated with the controller 22 of the image display device 10, may be connected to the controller 22 by a connection cable or wirelessly, or may be carried by the user U.
In order to record the surrounding video signals of the plurality of cameras 20 on the recording device 32, the surrounding video signals from the plurality of cameras 20 and the audio signals from the microphones 33 may be simultaneously or within the range of the recording rate of the recording device 32. It is assumed that recording is performed in a time-sharing manner, and the recorded surrounding video signal of each camera 20 can be selectively reproduced during reproduction.
The recording device 32 may record all the surrounding video signals from the plurality of cameras 20, or may record a selected part. The recording device 32 is not limited to one, and a plurality of recording devices 32 may be used. .

Next, the operation will be described.
First, the external image 102 is made visible through the concave half mirror 1412 and the half mirror 1410 by adjusting the transmittance of the liquid crystal shutter 1413.
In this state, the operation unit is operated to output the ambient video signal from the camera 20 and the audio signal from the microphone 33 to the recording device 32, and the recording device 32 starts recording video and audio.
Then, with the movement of the user U, an ambient video signal and an ambient audio signal indicating a situation outside the user U are recorded in the recording device 32.
According to such a configuration, as the user U moves, the ambient video signal and the ambient audio signal indicating the situation outside the user U are recorded in the recording device 32, so that the ambient video recorded in the recording device 32 is recorded. By reproducing the signal and the ambient audio signal, the visual and auditory experience of the user U can be reproduced.
Further, as described in the third, fourth, fifth, and sixth embodiments, the user U reduces the outline of the second image 110 by the camera 20 with the screen 15 as necessary while observing an object to be recorded with the naked eye. You can check on the screen or instantly zoom in and check the details of the recording screen.
As a result, it is possible to simultaneously perform the video shooting and the observation of the shooting target without feeling the stress that “the shooting target cannot be observed due to the video shooting”.
Also in the seventh embodiment, the configuration of the image display device 10 described in the first to sixth embodiments can be applied, and the ambient video signal generated by the camera 20 in each of these embodiments, that is, the second image is recorded. can do.

In addition, the image display apparatus 10 according to the present embodiment can be used not only as a video camera that records the situation around the user U but also as a recording apparatus for the user U's action history.
While the user U observes the external scenery and acts normally, the recording device 32 can record the surrounding video signal from the camera 20 and record it naturally without being aware of the contents experienced by the user U.
For example, the first camera 2002 (FW) set to a wide angle that captures the front direction of the user U will be described. Since the surrounding video signal recorded by the first camera 2002 is the front direction of the user U's face, the user is generally the user. The video information observed by the user U can be recorded without being aware of the operation of the video camera at the same time as the voice information from the microphone, which coincides with the line of sight of U.
However, the video information normally observed by the user U is not limited to the front direction of the user U's face, but can be moved vertically and horizontally from the front direction of the face by moving the line of sight.
For this reason, in order to record the video information generated by the movement of the user's line of sight with the first camera 2002 without missing the angle of view of the first camera 2002, the upper limit of the line of sight of the user U to the lower limit of the line of sight and the right eye It is desirable to set to include the visible range of the left eye to the visible range of the left eye.
Further, the line-of-sight detection sensor 1416 is used to detect the line-of-sight direction of the user U, and the line-of-sight information of the user U is recorded on the recording device 32 at the same time. ) Superimposed on the second image 110, the viewpoint of the user U at the time of recording, that is, what the user U was gazing at can be recognized from the reproduced second image 110.

In addition, the first acceleration sensor and the second acceleration sensor that constitute the walking sensor 2212 described above have a so-called camera shake correction function so as to recognize the shaking of the imaging device due to the action of the user U and prevent the captured image from blurring. Is possible.
This camera shake correction may be performed when an image is recorded by the camera 20, but the output information of the acceleration sensor is recorded together with the image without performing the correction, and whether or not to perform the blur correction at the time of reproduction is selected. You may be able to do it.
This reduces power consumption during blur correction during recording, and reproduces images with a sense of realism according to the movement of the user U by selecting not to perform correction selection during playback or to reduce the correction amount. It is also possible to do.
If such a plurality of cameras 20 can record all directions around the user U at the same time, it is possible to record the surrounding situation during the action of the user U without any omission and the convenience is further improved.
It is possible for the viewer to select which of the surrounding video signals is selected and displayed during playback. However, if an image in the line-of-sight direction is selected from the recorded line-of-sight information of the user U, the experience video of the user U can be obtained. Can be reproduced.
As the action history of the user U, more detailed information can be stored if the following information can be recorded in addition to the surrounding video information, audio information, and line-of-sight information of the user U.
(1) If the positioning data generated by the GPS sensor 2210 is recorded in the recording device 32, the movement status of the user U and the information on the place where the user U has acted can be recorded.
(2) Using an acceleration sensor of two or more axes, it determines and records behavior modes such as walking start / stop, walking speed, ascending / descending stairs, head variation, etc., and uses it as image blur correction data.
(3) Various sensors for detecting body temperature, heart rate, respiration rate, sweating amount, etc. are further provided, and physical condition data obtained from these sensors is recorded to determine changes in exercise status, physical condition, and emotion. .

In each of the above-described embodiments, the image display device 10 is configured to display an image for both eyes of the user U, but may be configured to display an image for one eye.
Further, the screen 15 is not limited to a configuration having a so-called see-through function for visually recognizing the external image 102 through the screen 15 by using a half mirror as shown in FIG. It may have a configured see-through function. In addition, when a field of view corresponding to the external image 102 can be secured based on the surrounding video signal of the camera 20, a so-called free-shaped prism type that does not have a see-through function may be used.
In addition, the configuration of the mounting unit 12 of the image display device 10 is not limited to the configuration illustrated in FIG. 1, and may be a configuration that is supported by both ears and nose bridge portions of the user U like glasses. The structure supported by covering a head like a helmet may be sufficient.

1 is a perspective view illustrating a configuration of an image display device according to a first embodiment. It is explanatory drawing which shows the state with which the image display apparatus of Example 1 was mounted | worn. FIG. 3 is an explanatory diagram illustrating an arrangement of imaging units in the image display apparatus according to the first embodiment. 1 is a block diagram illustrating a configuration of an image display device according to a first embodiment. 1 is an explanatory diagram illustrating a configuration of an image display device according to Embodiment 1. FIG. 1 is a configuration diagram of an optical system of an image display apparatus according to Embodiment 1. FIG. It is principle explanatory drawing of a walk sensor. FIG. 6 is a perspective view illustrating a configuration of an image display device according to a second embodiment. FIG. 10 is an explanatory diagram of an image displayed on the screen 15. It is explanatory drawing which shows the visual field characteristic of a normal human perpendicular | vertical direction. It is explanatory drawing which shows the visual field characteristic of a standard human horizontal direction. It is explanatory drawing which shows a standard human visual field characteristic. The block diagram which simplifies and shows the structure of the image display apparatus 10 in Example 4. FIG. 10 is an operation flowchart according to the fourth embodiment. 14 is an explanatory diagram of an image displayed on a screen 15 in Embodiment 4. FIG. FIG. 10 is an explanatory diagram of an image displayed on a screen 15 in the fifth embodiment. FIG. 10 is an explanatory diagram of an image displayed on a screen 15 in Example 6. FIG. 10 is an explanatory diagram of an image displayed on a screen 15 in Example 6. FIG. 10 is an explanatory diagram of an image displayed on a screen 15 in Example 6.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Image display apparatus, 12 ... Mounting part, 14 ... Display unit, 1406 ... Backlight, 1408 ... LCD, 1410 ... Half mirror, 1412 ... Concave half mirror, 1413 ... Liquid crystal shutter, 15 ... screen, 20 ... camera, 22 ... controller.

Claims (10)

A wearing part to be worn on the user's head or face;
A screen that is supported by the mounting portion and disposed within the field of view of the user in a state where the mounting portion is mounted on the user;
An image display device having image display means for displaying an image on the screen based on an input video signal,
An imaging unit that is supported by the mounting unit or the screen and that captures an image of a part or almost the entire area around the user in a state where the mounting unit is mounted on the user, and generates an ambient video signal that represents the situation around the user; ,
By generating the input video signal based on both the externally supplied video signal supplied from the outside of the image display device and the ambient video signal supplied from the imaging means, and supplying the input video signal to the image display means Control means for displaying on the screen a first image based on the externally supplied video signal and a second image based on the ambient video signal signal;
An image display device comprising:   The display of the first image and the second image by the control means is performed by selecting any one of the first image and the second image and displaying the selected image on the image display means. The image display device according to claim 1.   The display of the first image and the second image by the control means is performed by causing the image display means to display both the first image and the second image at the same time. 1. The image display device according to 1.   The screen includes shutter means extending over the entire area, and the shutter means is configured to be switchable between a transmissive state that transmits at least part of light and a non-transmissive state that does not transmit light. The image display device according to claim 1.   The imaging unit includes a plurality of cameras, and imaging of almost the entire area around the user by the imaging unit is performed by arranging the plurality of cameras so that their directing directions are different from each other. Item 2. The image display device according to Item 1.   The image display apparatus according to claim 1, wherein the imaging unit includes at least two cameras arranged so that their directivity directions are the same.   The imaging means has a single camera and a reflecting mirror provided so as to face almost the whole area around the user. The imaging means picks up the whole area around the user by the imaging means. The image display device according to claim 1, wherein the image display device is made by taking a picture through the camera.   The image display apparatus according to claim 1, wherein the imaging unit is configured to be capable of adjusting a shooting magnification.   The image display apparatus according to claim 1, wherein the imaging unit is configured to be capable of adjusting a shooting direction.   The image display apparatus according to claim 1, wherein the imaging unit is configured to output the ambient video signal in a form connectable to an external recording apparatus.

Images