JP6500382B2 - Head-mounted display device, control method therefor, and computer program - Google Patents

Description

  The present invention relates to a head-mounted display device, a control method therefor, and a computer program.

  In recent years, a head mounted display that is a display mounted on the head is known. This head-mounted display device is also called a head mounted display (HMD: Head Mounted Display), and there is a see-through type that allows the user to see the outside scene in a state where the HMD is mounted. The see-through HMD reflects the image light generated by a light modulation element such as a liquid crystal panel with an optical system or the like placed in front of the user's eyes to create a virtual image with the outside view (real image) in the user's field of view. Is displayed (for example, Patent Document 1).

JP, 2010-139901, A

  Conventionally, in the see-through type HMD, those having various functions, including the function of displaying the image, have been proposed. As various functions, there are various functions such as a photographing function using a camera and a function to output sound. There has been a problem that it has not been sufficiently studied as to whether these functions may be operated at all times. In addition, in the conventional head-mounted display device, the convenience of the user is improved, the detection accuracy is improved, the malicious use such as theft photographing is prevented, the compactness of the device configuration, the cost reduction, the resource saving, For example, facilitation of production has been desired.

The present invention has been made to solve at least a part of the above-mentioned problems, and can be realized as the following modes.
One embodiment of the present invention is a transmissive head-mounted display device. The head-mounted display device displays an image to cause a user wearing the head-mounted display device to visually recognize the image, and an image display unit capable of transmitting an outside view, and the head-mounted display device And a processing control unit that changes at least a part of a predetermined function mounted on the head-mounted display device according to the determined usage environment. . The predetermined function is a telephoto photographing function of photographing a telephoto by the camera unit. The use environment includes at least a specific seating environment in the stadium. The processing control unit turns on the telephoto imaging function to perform telephoto imaging with the camera unit when the usage environment determination unit determines that the seat environment is the specific one. The head-mounted display device further includes an erasing unit that erases a captured image recorded by the telephoto imaging when it is detected that the head-mounted display device has moved out of the stadium.
According to the head-mounted display device of this aspect, at least a part of the predetermined function mounted on the head-mounted display device is selected according to the usage environment of the head-mounted display device mounted on the user. You may change it. For this purpose, changes are made such that at least a part of the predetermined functions mounted on the head-mounted display device is suppressed or expanded in accordance with the usage environment. Therefore, according to the head-mounted display device of this embodiment, the function corresponding to the usage environment can be easily realized. Moreover, according to the head mounted display of this aspect, it is possible to suppress or expand the telephoto imaging function by the camera unit. Therefore, the mode can be easily switched to the telephoto shooting mode depending on the usage environment. In addition, according to the head-mounted display device of this embodiment, since the telephoto shooting is automatically performed in a specific seat environment in the stadium, the user is free to take a shot image recorded by the telephoto shooting. It becomes possible to reproduce. Therefore, the convenience of the user in the specific seating environment in the stadium can be enhanced. Moreover, according to the head-mounted display device of this aspect, it is possible to prevent the taken image taken in a specific seat environment in the stadium from being taken out of the stadium.
Besides, the present invention can also be realized as the following embodiments.

(1) One embodiment of the present invention is a transmissive head-mounted display device. The head-mounted display device displays an image to cause a user wearing the head-mounted display device to visually recognize the image, and an image display unit capable of transmitting an outside view, and the head-mounted display device And a processing control unit that changes at least a part of a predetermined function mounted on the head-mounted display device according to the determined usage environment. You may According to the head-mounted display device of this aspect, at least a part of the predetermined function mounted on the head-mounted display device is selected according to the usage environment of the head-mounted display device mounted on the user. You may change it. For this purpose, changes are made such that at least a part of the predetermined functions mounted on the head-mounted display device is suppressed or expanded in accordance with the usage environment. Therefore, according to the head-mounted display device of this embodiment, the function corresponding to the usage environment can be easily realized.

(2) In the head mounted display according to the above aspect, the usage environment may be a usage environment switched by the movement of the head mounted display. According to the transmission-type head-mounted display device, the user can move freely while being worn, and the usage environment is likely to change. Therefore, according to the head-mounted display device of this embodiment, it is possible to easily realize the function corresponding to the changing usage environment.

(3) The head mounted display according to the above aspect, further comprising: an imaging unit that performs imaging, wherein the use environment determination unit determines the use environment based on a captured image obtained by the imaging unit. May be Since the imaging unit is provided in the head-mounted display device, the imaging range also moves as the head of the user moves. For this reason, it is possible to easily detect a change in the use environment from the photographed image obtained by the photographing unit.

(4) In the head mounted display according to the above aspect, the use environment determining unit may include a marker for recognizing a specific use environment in the captured image obtained by the imaging unit. It may be determined that it is the specific usage environment. According to the head-mounted display device of this aspect, a change in the usage environment can be easily detected by photographing the marker.

(5) In the head mounted display according to the above aspect, the use environment determining unit may determine the use environment based on a signal from an external wireless communication terminal. According to the head-mounted display device of this aspect, the usage environment can be easily detected.

(6) In the head mounted display according to the aspect described above, the predetermined function may be a telephoto imaging function of performing telephoto imaging by the camera unit. According to the head-mounted display device of this aspect, the telephoto imaging function of the camera unit can be suppressed or expanded. Therefore, the mode can be easily switched to the telephoto shooting mode depending on the usage environment.

(7) In the head mounted display according to the above aspect, the use environment includes at least a specific seat environment in a stadium, and the processing control unit determines that the use environment determination unit is the specific seat environment. When determined, the telephoto imaging function may be turned on to perform telephoto imaging with the camera unit. According to the head-mounted display device of this aspect, the user can freely reproduce the photographed image recorded by the telephoto photographing since the telephoto photographing is automatically performed in a specific seat environment in the stadium. It becomes possible. Therefore, the convenience of the user in the specific seating environment in the stadium can be enhanced.

(8) In the head mounted display according to the above aspect, the erasing unit is configured to erase the photographed image recorded by the telephoto photographing when it is detected that the head mounted display has moved out of the stadium. May be provided. According to the head-mounted display device of this aspect, it is possible to prevent the taken image taken in a specific seat environment in the stadium from being taken out of the stadium.

(9) In the head mounted display according to the above aspect, the processing control unit acquires predetermined data and stores the data in the storage unit when the usage environment determination unit determines that the usage environment is the predetermined usage environment. The predetermined data stored in the storage unit in the predetermined usage environment may be erased when the usage environment determination unit determines that the predetermined usage environment is left. According to the head-mounted display device of this aspect, the predetermined data stored in the predetermined usage environment can be erased when it is out of the predetermined usage environment. For this reason, according to the head-mounted display device of this embodiment, security can be improved because the use range of predetermined data can be limited to a predetermined use environment.

(10) In the head mounted display according to the above aspect, the predetermined function may be an information presenting function that causes the image display unit to display predetermined information. According to the head-mounted display device of this aspect, it is possible to suppress or expand the information presentation function of causing the image display unit to display predetermined information. Therefore, it is possible to easily present information appropriate for the occasion.

(11) In the head mounted display according to the above aspect, the use environment includes at least a specific seat environment in a theater, and the processing control unit determines that the use environment determination unit determines the specific seat environment. When determined, guidance information on a play being played in the theater may be displayed as the predetermined information. According to the head-mounted display device of this aspect, when transitioning to a specific seating environment in the theater, it is possible to show the user, by image display, guidance information on a play being played in the theater. Therefore, information on the play can be easily presented.

(12) In the head mounted display according to the above aspect, the use environment includes at least a specific seat environment in a movie theater, and the processing control unit determines that the use environment determination unit is the specific seat environment. When determined, the subtitle may be displayed as the predetermined information. According to the head-mounted display device of this aspect, when transitioning to a specific seating environment in a movie theater, subtitles of a movie being shown can be shown to the user by image display. Therefore, information presentation on subtitles can be easily performed.

(13) In the head mounted display according to the above aspect, the specific seating environment in the movie theater includes a first seating environment and a second seating environment, and the processing control unit determines the usage environment. When it is determined by the unit that the seat environment is the first, the subtitles described in the first language are displayed, and when the use environment determination unit determines that the seat environment is the second Alternatively, the subtitles written in a second language different from the first language may be displayed. According to the head-mounted display device of this aspect, when transitioning to the first seating environment in the movie theater, subtitles can be received in the first language, and transitioning to the second seating environment in the movie theater Subtitles can be received by the second language. Therefore, the subtitle language can be easily switched according to the seating environment.

(14) In the head mounted display according to the above aspect, the use environment includes at least a security environment requiring a high level of security, and the process control unit is the security environment by the use environment determining unit. When it is determined that the predetermined application is executed, execution of the predetermined application may be prohibited when the usage environment determination unit determines that the security environment is not set. According to the head-mounted display device of this aspect, when transitioning to a security environment requiring a high level of security, execution of a predetermined application is permitted, and when not in the security environment, the predetermined application of Execution is prohibited. Therefore, depending on the security environment, permission / prohibition of execution of a predetermined application can be easily switched.

  The plurality of components included in each aspect of the present invention described above are not all essential, and some or all of the effects described in this specification may be solved in order to solve some or all of the problems described above. In order to achieve the above, it is possible to appropriately change, delete, replace with another new component, and partially delete limited content for some components of the plurality of components. In addition, in order to solve some or all of the problems described above, or to achieve some or all of the effects described in the present specification, the technical features included in one embodiment of the present invention described above It is also possible to combine some or all of the technical features included in the other aspects of the present invention described above into one independent aspect of the present invention.

  For example, one aspect of the present invention can be realized as a device including one or more or all of the three elements of an image display unit, a usage environment determination unit, and a processing control unit. That is, this device may or may not have the image display unit. The device may or may not have the usage environment determination unit. Further, the apparatus may or may not have the process control unit. For example, the image display unit may display an image to allow a user wearing the head-mounted display device to visually recognize the image and transmit an outside scene. The use environment determination unit may determine, for example, the use environment of the head-mounted display device. The process control unit may change, for example, at least a part of the predetermined function mounted on the head-mounted display device according to the determined usage environment. Such a device can be realized, for example, as a head-mounted display device, but can also be realized as another device other than the head-mounted display device. According to such a mode, the convenience of the user is improved, the detection accuracy is improved, the malicious use such as theft shooting is prevented, the device configuration is compacted, the cost is reduced, the resource is saved, the manufacturing is facilitated, etc. At least one of the various problems of can be solved. Some or all of the technical features of each form of the head mounted display described above can be applied to this device.

  The present invention can also be realized in various forms other than the head mounted display. For example, a display device, a head-mounted display device and a control method of the display device, a head-mounted display system, a display device, a head-mounted display system, a computer program for realizing the functions of the display device, and the computer program And a computer program and a data signal embodied in a carrier wave.

BRIEF DESCRIPTION OF THE DRAWINGS It is explanatory drawing which shows schematic structure of the head mounted display (HMD) in 1st Embodiment. It is a block diagram which shows the structure of HMD functionally. It is explanatory drawing which shows an example of the virtual image visually recognized by the user. It is explanatory drawing of the platform of HMD. It is an explanatory view showing a ticket for watching a football concerning a 1st embodiment. It is explanatory drawing which shows an example of the information visually recognized by a user, when a gaze for a soccer watching ticket is made and a QR code is read. It is a flow chart which shows the details of a watching aiding routine. It is an explanatory view showing an example of information visually recognized by a user when reproducing a photography animation. It is explanatory drawing which shows the use form of HMD which concerns on 2nd Embodiment. It is a flowchart which shows the detail of the viewing assistance routine in 2nd Embodiment. It is explanatory drawing which shows the use form of HMD which concerns on 3rd Embodiment. It is a flowchart which shows the detail of the viewing assistance routine in 3rd Embodiment. It is explanatory drawing which shows the use form of HMD which concerns on 4th Embodiment. It is explanatory drawing which shows the structure of the external appearance of HMD in a modification.

A. First embodiment:
A-1. Configuration of head mounted display:
FIG. 1 is an explanatory view showing a schematic configuration of a head mounted display according to a first embodiment of the present invention. The head-mounted display device 100 is a display device mounted on the head, and is also called a head mounted display (HMD). The HMD 100 is an optical transmission head mounted display that allows a user to view a virtual image and also directly view the outside scene, and in the present embodiment, is used for watching a game (for example, soccer).

  The HMD 100 includes an image display unit 20 that allows a user to visually recognize a virtual image in a state of being worn on the head of the user, and a control unit (controller) 10 that controls the image display unit 20.

  The image display unit 20 is a mounting body to be mounted on the head of the user, and in the present embodiment, has an eyeglass shape. Image display unit 20 includes right holding unit 21, right display driving unit 22, left holding unit 23, left display driving unit 24, right optical image display unit 26, and left optical image display unit 28. It is. The right optical image display unit 26 and the left optical image display unit 28 are disposed so as to be positioned in front of the user's right and left eyes, respectively, when the user wears the image display unit 20. One end of the right optical image display unit 26 and one end of the left optical image display unit 28 are connected to each other at a position corresponding to the eyelid of the user when the user wears the image display unit 20.

  The right holding unit 21 extends from an end ER, which is the other end of the right optical image display unit 26, to a position corresponding to the side of the user when the user wears the image display unit 20. Member. Similarly, the left holding unit 23 extends from the end EL, which is the other end of the left optical image display unit 28, to a position corresponding to the user's temporal region when the user wears the image display unit 20. It is a member provided. The right holding unit 21 and the left holding unit 23 hold the image display unit 20 on the head of the user, like temples of glasses.

  The right display drive unit 22 is disposed inside the right holding unit 21, in other words, on the side facing the head of the user when the user wears the image display unit 20. Further, the left display drive unit 24 is disposed inside the left holding unit 23. Note that, hereinafter, the right holding unit 21 and the left holding unit 23 will be described as “holding units” without distinction. Similarly, the right display drive unit 22 and the left display drive unit 24 are described as “display drive unit” without distinction, and the right optical image display unit 26 and left optical image display unit 28 are not distinguished “optical image display unit” Explain as.

  The display driving unit includes liquid crystal displays (Liquid Crystal Display, hereinafter referred to as “LCD”) 241 and 242, projection optical systems 251 and 252, and the like (see FIG. 2). Details of the configuration of the display drive unit will be described later. An optical image display unit as an optical member includes light guide plates 261 and 262 (see FIG. 2) and a light control plate. The light guide plates 261 and 262 are made of a light transmissive resin material or the like, and guide the image light output from the display drive unit to the eyes of the user. The light control plate is a thin plate-like optical element, and is disposed so as to cover the front side of the image display unit 20 (the side opposite to the side of the user's eye). The light control plate protects the light guide plates 261 and 262, and suppresses damage to the light guide plates 261 and 262 and adhesion of dirt and the like. Further, by adjusting the light transmittance of the light control plate, it is possible to adjust the amount of external light entering the user's eyes and adjust the ease of visual recognition of the virtual image. The light control plate can be omitted.

  The image display unit 20 further includes a connection unit 40 for connecting the image display unit 20 to the control unit 10. The connection portion 40 includes a main body cord 48 connected to the control portion 10, a right cord 42 and a left cord 44 in which the main body cord 48 is branched into two, and a connecting member 46 provided at the branch point. . The right cord 42 is connected to the right display drive unit 22, and the left cord 44 is connected to the left display drive unit 24. The connection member 46 is provided with a jack for connecting the earphone plug 30. The right earphone 32 and the left earphone 34 extend from the earphone plug 30.

  The image display unit 20 and the control unit 10 transmit various signals via the connection unit 40. A connector (not shown) which is fitted to each other is provided at each end of the main body cord 48 opposite to the connecting member 46 and the control portion 10, and the connector of the main body cord 48 and the connector of the control portion 10 The control unit 10 and the image display unit 20 are connected or disconnected due to the engagement / disengagement with. For the right cord 42, the left cord 44, and the main body cord 48, for example, a metal cable or an optical fiber can be adopted.

  The control unit 10 is a device for controlling the HMD 100. The control unit 10 includes a lighting unit 12, a touch pad 14, a cross key 16, and a power switch 18. The lighting unit 12 notifies the operating state of the HMD 100 (for example, power ON / OFF, etc.) by the light emission mode. For example, an LED (Light Emitting Diode) can be used as the lighting unit 12. The touch pad 14 detects a touch operation on the operation surface of the touch pad 14 and outputs a signal corresponding to the content of the detection. As the touch pad 14, various touch pads such as an electrostatic type, a pressure detection type, and an optical type can be adopted. The cross key 16 detects the pressing operation on the keys corresponding to the up, down, left, and right directions, and outputs a signal according to the detection content. The power switch 18 switches the state of the power of the HMD 100 by detecting the sliding operation of the switch.

  FIG. 2 is a block diagram functionally showing the configuration of HMD 100. Referring to FIG. The control unit 10 includes an input information acquisition unit 110, a storage unit 120, a power supply 130, a wireless communication unit 132, a GPS module 134, a CPU 140, an interface 180, and transmission units (Tx) 51 and 52. Each part is mutually connected by the bus which is not illustrated.

  The input information acquisition unit 110 acquires, for example, a signal corresponding to an operation input to the touch pad 14, the cross key 16, the power switch 18, and the like. The storage unit 120 is configured by a ROM, a RAM, a DRAM, a hard disk, and the like.

  The power supply 130 supplies power to each part of the HMD 100. As the power supply 130, for example, a secondary battery such as a lithium polymer battery or a lithium ion battery can be used. Furthermore, instead of the secondary battery, a primary battery or a fuel cell may be used, or wireless power may be supplied to operate. Furthermore, power may be supplied from the solar cell and the capacitor. The wireless communication unit 132 performs wireless communication with another device in accordance with a predetermined wireless communication standard such as wireless LAN, Bluetooth (registered trademark), or iBeacon (registered trademark). The GPS module 134 detects its current position by receiving signals from GPS satellites.

  The CPU 140 reads out and executes a computer program stored in the storage unit 120, thereby operating system (OS) 150, image processing unit 160, display control unit 162, usage environment determination unit 164, processing control unit 166, voice The processing unit 170 functions as an iBeacon processing unit 172.

  The image processing unit 160 generates a signal based on content (video) input via the interface 180 or the wireless communication unit 132. Then, the image processing unit 160 controls the image display unit 20 by supplying the generated signal to the image display unit 20 via the connection unit 40. The signals to be supplied to the image display unit 20 differ between the analog form and the digital form. In the case of the analog format, the image processing unit 160 generates and transmits the clock signal PCLK, the vertical synchronization signal VSync, the horizontal synchronization signal HSync, and the image data Data. Specifically, the image processing unit 160 acquires an image signal included in the content. For example, in the case of a moving image, the acquired image signal is an analog signal generally composed of 30 frame images per second. The image processing unit 160 separates synchronization signals such as the vertical synchronization signal VSync and the horizontal synchronization signal HSync from the acquired image signal, and generates a clock signal PCLK by a PLL circuit or the like according to the period thereof. The image processing unit 160 converts the analog image signal from which the synchronization signal is separated into a digital image signal using an A / D conversion circuit or the like. The image processing unit 160 stores the converted digital image signal in the DRAM in the storage unit 120 frame by frame as image data Data of RGB data.

  On the other hand, in the case of digital format, the image processing unit 160 generates and transmits the clock signal PCLK and the image data Data. Specifically, when the content is in digital format, the clock signal PCLK is output in synchronization with the image signal, so generation of the vertical synchronization signal VSync and the horizontal synchronization signal HSync and A / D conversion of the analog image signal are performed. It becomes unnecessary. The image processing unit 160 performs image processing such as resolution conversion processing, various color tone correction processing such as adjustment of luminance and saturation, keystone correction processing, and the like on the image data Data stored in the storage unit 120. It may be executed.

  The image processing unit 160 transmits the generated clock signal PCLK, the vertical synchronization signal VSync, the horizontal synchronization signal HSync, and the image data Data stored in the DRAM in the storage unit 120 through the transmission units 51 and 52, respectively. Do. The image data Data transmitted through the transmission unit 51 is also referred to as “right-eye image data Data 1”, and the image data Data transmitted through the transmission unit 52 is also referred to as “left-eye image data Data 2”. The transmission units 51 and 52 function as transceivers for serial transmission between the control unit 10 and the image display unit 20.

  The display control unit 162 generates control signals for controlling the right display drive unit 22 and the left display drive unit 24. Specifically, the display control unit 162 controls the drive on / off of the right LCD 241 by the right LCD control unit 211, the drive on / off of the right backlight 221 by the right backlight control unit 201, and the left LCD control according to control signals. Each of the right display drive unit 22 and the left display drive unit 24 is controlled by individually controlling the drive ON / OFF of the left LCD 242 by the unit 212 and the drive ON / OFF of the left backlight 222 by the left backlight control unit 202. Control the generation and emission of image light by The display control unit 162 transmits control signals to the right LCD control unit 211 and the left LCD control unit 212 via the transmission units 51 and 52, respectively. Similarly, the display control unit 162 transmits control signals to the right backlight control unit 201 and the left backlight control unit 202, respectively.

  The usage environment determining unit 164 determines a moving environment which is switched by the movement of the HMD 100 mounted on the head of the user. The processing control unit 166 changes at least a part of predetermined functions among various functions mounted on the HMD 100. The predetermined function may be one function or a plurality of functions, but in the first embodiment, it is one function. Details of the usage environment determination unit 164 and the process control unit 166 will be described later.

  The audio processing unit 170 acquires an audio signal included in the content, amplifies the acquired audio signal, and outputs a speaker (not shown) in the right earphone 32 connected to the connecting member 46 and a speaker (not shown) in the left earphone 34. Supply against. Note that, for example, when the Dolby (registered trademark) system is adopted, processing is performed on the audio signal, and the right earphone 32 and the left earphone 34 respectively output different sounds whose frequency or the like is changed.

  The iBeacon processing unit 172 receives the signal from the BLE (Bluetooth Low Energy) terminal provided outside the HMD 100 using the technology of iBeacon (registered trademark), thereby determining the distance between the BLE terminal and the HMD 100. Ask.

  The interface 180 is an interface for connecting to the control unit 10 various external devices OA as a content supply source. Examples of the external device AA include a personal computer PC, a mobile phone terminal, a game terminal and the like. As the interface 180, for example, a USB interface, a micro USB interface, an interface for a memory card, or the like can be used.

  The image display unit 20 includes a right display drive unit 22, a left display drive unit 24, a right light guide plate 261 as the right optical image display unit 26, a left light guide plate 262 as the left optical image display unit 28, and an outside view photography A camera 61 (see also FIG. 1) and a nine-axis sensor 66 are provided.

  The outside scene photographing camera 61 is disposed at a position corresponding to the gap between the user when the user wears the image display unit 20. Therefore, the outside view photographing camera 61 captures an outside view which is an external scene in the direction in which the user is facing, in a state where the user wears the image display unit 20 on the head. The outside view photographing camera 61 is a monocular camera, but may be a stereo camera. In the present embodiment, the outside-view shooting camera 61 has a telephoto function, and can perform shooting under telephoto. The telephoto function may be either optical zoom or digital zoom. Further, the outside view photographing camera 61 may be replaced by an external camera such as a USB connection.

  The 9-axis sensor 66 is a motion sensor that detects acceleration (3 axes), angular velocity (3 axes), and geomagnetism (3 axes), and is arranged at a position corresponding to the user's right temple in this embodiment. . Since the 9-axis sensor 66 is provided in the image display unit 20, when the image display unit 20 is mounted on the head of the user, the movement of the head of the user is detected. The orientation of the image display unit 20 is specified from the detected movement of the head of the user.

  The right display driving unit 22 includes a receiving unit (Rx) 53, a right backlight (BL) control unit 201 functioning as a light source and a right backlight (BL) 221, a right LCD control unit 211 functioning as a display element, and a right An LCD 241 and a right projection optical system 251 are included. The right backlight control unit 201, the right LCD control unit 211, the right backlight 221, and the right LCD 241 are collectively referred to as an "image light generation unit".

  The receiving unit 53 functions as a receiver for serial transmission between the control unit 10 and the image display unit 20. The right backlight control unit 201 drives the right backlight 221 based on the input control signal. The right backlight 221 is, for example, a light emitter such as an LED or electroluminescence (EL). The right LCD control unit 211 drives the right LCD 241 based on the clock signal PCLK input via the reception unit 53, the vertical synchronization signal VSync, the horizontal synchronization signal HSync, and the right-eye image data Data1. The right LCD 241 is a transmissive liquid crystal panel in which a plurality of pixels are arranged in a matrix. The right LCD 241 changes the transmittance of light transmitted through the right LCD 241 by driving the liquid crystal at each pixel position arranged in a matrix, thereby converting the illumination light emitted from the right backlight 221 into an image. Modulate into useful image light.

  The right projection optical system 251 is configured of a collimator lens that converts the image light emitted from the right LCD 241 into parallel light beams. The right light guide plate 261 as the right optical image display unit 26 guides the image light output from the right projection optical system 251 to the right eye RE of the user while reflecting it along a predetermined optical path. The optical image display unit can use any method as long as a virtual image is formed in front of the user using image light, and for example, a diffraction grating may be used or a semi-transmissive reflective film may be used. Good. The emission of the image light by the HMD 100 is also referred to as “displaying an image” in the present specification.

  The left display drive unit 24 has a configuration similar to that of the right display drive unit 22. That is, the left display driving unit 24 includes a receiving unit (Rx) 54, a left backlight (BL) control unit 202 functioning as a light source and a left backlight (BL) 222, and a left LCD control unit 212 functioning as a display element. And a left LCD 242 and a left projection optical system 252. Similar to the right LCD 241, the left LCD 242 is illuminated from the left backlight 222 by changing the transmittance of light transmitted through the left LCD 242 by driving the liquid crystal at each pixel position arranged in a matrix. The illumination light is modulated into useful image light representing an image. Although the backlight method is adopted in this embodiment, the image light may be emitted using a front light method or a reflection method.

  FIG. 3 is an explanatory view showing an example of a virtual image visually recognized by the user. In FIG. 3, the visual field VR of the user is illustrated. As described above, the image light guided to the eyes of the user of the HMD 100 forms an image on the user's retina, whereby the user views the virtual image (image) VI. In the example of FIG. 3, the virtual image VI is a standby screen of the OS of the HMD 100. In addition, the user passes through the right optical image display unit 26 and the left optical image display unit 28 to visually recognize the outside scene SC. As described above, the user of the HMD according to the present embodiment can view the virtual image VI and the outside scene SC behind the virtual image VI for the portion of the visual field VR where the virtual image VI is displayed. Further, with respect to the portion of the visual field VR1 where the virtual image VI is not displayed, the outside scene SC can be directly viewed through the optical image display unit.

A-2. HMD platform:
FIG. 4 is an explanatory diagram of a platform of the HMD 100. The platform is a collection of underlying hardware resources, an OS, middleware, etc. necessary to operate an application installed in the HMD 100. The platform 500 of the present embodiment includes an application layer 510, a framework layer 520, a library layer 530, a kernel layer 540, and a hardware layer 550. Each layer 510 to 550 conceptually divides hardware resources, an OS, middleware, and the like included in the platform 500. The framework layer 520, the library layer 530, and the kernel layer 540 implement the functions of the OS 150 (FIG. 2). In FIG. 4, components not necessary for the description are not shown.

  The application layer 510 is a set of application software for executing predetermined processing on the OS 150. Each application software included in the application layer 510 is hereinafter also referred to as “app” or “application”. The application layer 510 includes both an application pre-installed in the HMD 100 and an application installed by the user of the HMD 100.

  In the example of FIG. 4, the application layer 510 includes a watching support application 511, a game application 512, a camera application 513, a code reader application 514, and the like. The watching support application 511 provides a watching support function suitable for watching a game on a large stadium (stadium). The game application 512 provides a game function. The camera application 513 provides a photographing function.

  The framework layer 520 is a set of programs having a basic program structure common to the application software of the application layer 510 and a function set implemented. In the present embodiment, an image processing unit frame 521, a display control unit frame 522, an audio processing unit frame 523, an iBeacon processing unit frame 524 and the like are included. The image processing unit frame 521 implements the function of the image processing unit 160 (FIG. 2). The display control unit frame 522 implements the function of the display control unit 162 (FIG. 2). The audio processing unit frame 523 implements the function of the audio processing unit 170 (FIG. 2). The iBeacon processing unit frame 524 implements the function of the iBeacon processing unit 172 (FIG. 2).

  The library layer 530 is a collection of library software that has been componentized so that a program for realizing a specific function can be used by another program (for example, an application of the application layer 510). Each library software contained in the library layer 530 is hereinafter also referred to as a "library". The library can not be executed alone, but is executed as called from other programs.

  In the example of FIG. 4, the library layer 530 includes a display library 533, an audio library 534, a sensor library 535, a camera library 536, an HTML (Hyper Text Markup Language) library 537, etc. include. The display library 533 drives the right LCD 241 and the left LCD 242 (FIG. 2). The audio library 534 drives an audio integrated circuit (IC) built in the right earphone 32 and the left earphone 34 (FIG. 2). The sensor library 535 drives the 9-axis sensor 66 (FIG. 2), acquires detection values from the 9-axis sensor 66, and processes the detection values into information for providing the application. The camera library 536 drives the outside view photographing camera 61 (FIG. 2), acquires a detection value by the outside view photographing camera 61, and generates an outside view image from the detection value. The HTML library 537 interprets the data described in the web page description language to calculate the arrangement of characters and images for screen display.

  The kernel layer 540 is a collection of programs implementing the basic functions of the OS 150. The kernel layer 540 manages exchange between software (library layer 530) and hardware (hardware layer 550), and functions as a bridge between the two.

  In the example of FIG. 4, the kernel layer 540 includes an LCD driver 541 for the right LCD 241 and the left LCD 242, an audio IC driver 542 for an audio IC, a sensor driver 543 for the 9-axis sensor 66, and a camera 61 for taking an outside view. And an image sensor driver 544 for an image sensor, which is built in.

  The hardware layer 550 is an actual hardware resource incorporated in the HMD 100. In the present embodiment, “hardware resources” means a device connected to the HMD 100 and incorporated in the HMD 100. That is, hardware resources include devices (for example, sensor device of 9-axis sensor 66, image sensor device of camera 61, sensor device of touch pad 14, etc.) internally connected to the motherboard of HMD 100, and interface 180. And the devices externally connected to the HMD 100 (for example, an external motion sensor device, an external USB device, etc.).

  In the example of FIG. 4, the hardware layer 550 includes an LCD device 551 as the right LCD 241 and the left LCD 242, an audio IC device 552, a sensor device 553 of the 9-axis sensor 66, and an image sensor device 554 of the camera 61. It is included.

  The library, the driver, and the device surrounded by a broken line in FIG. 4 correspond to each other and operate in cooperation with each other. For example, sensor library 535, sensor driver 543, and sensor device 553 work in concert to achieve the functionality of nine-axis sensor 66. That is, the sensor library 535 of the library layer 530 and the sensor driver 543 of the kernel layer 540 are programs (software) for the application to use the sensor device 553 as a hardware resource (hardware layer 550). It can be said that there is. A plurality of libraries may be assigned to the sensor device 553 in order to enable the sensor device 553 as one hardware resource.

  On the other hand, in FIG. 4, for example, the HTML library 537 of the library layer 530 has no correspondence with hardware resources and does not depend on hardware resources. As described above, a program (software) incorporated in the HMD 100 and not dependent on hardware resources is also referred to as “software resource” in the present embodiment. As software resources, various programs included in each layer of the framework layer 520, the library layer 530, and the kernel layer 540 are assumed.

A-3. How to launch the watching ticket and app:
FIG. 5 is an explanatory view showing a ticket for watching soccer according to the present embodiment. On a ticket TC for watching soccer, a venue name C1, seat information C2 indicating a seat position, a QR code (registered trademark) C3 and the like are printed. In the QR code C3, start information for starting the watching support application 511 (FIG. 4) (hereinafter referred to as "watching support application start information") and seat information are coded and recorded.

The user wearing the HMD 100 activates the code reader application 514 in the stadium to look at the ticket TC presented at the entrance. The code reader application 514 is activated when the input information acquisition unit 110 receives a predetermined operation for selecting the [code reader] icon IC arranged on the standby screen (see FIG. 3) of the HMD 100. The HMD 100 that executes the code reader application 514 captures an image with the outside view shooting camera 61, and when the obtained captured image includes a QR code, restores the QR code to the original information, and the original information Display That is, when the captured image includes the QR code, the CPU 140 causes the image display unit 20 to display the information encoded as the QR code.

FIG. 6 is an explanatory view showing an example of information visually recognized by the user when the QR code C3 is read with the line of sight toward the ticket watching ticket TC. In FIG. 6, the visual field VR1 of the user is illustrated. As shown in the figure, the visual field VR1 includes the outside scene SC1 in the stadium. Further, the code information display area AR1 is displayed superimposed on the outside scene SC1. The code information display area AR1 includes the watching support application start information D1 and the seat information D2 which are the original information of the QR code C3 .

The user uses the touch pad 14 or the cross key 16 to perform a predetermined operation of selecting the watching support application start information D1. When the input information acquisition unit 110 receives the predetermined operation, the HMD 100 can start the watching support application 511 (FIG. 4). Specifically, the HMD 100 follows the URL described in the watching support application activation information D1, installs the watching support application, and starts the installed watching support application 511 (FIG. 4). The watching support application 511 may be built in advance. In the present embodiment, the QR code C3 is described on the ticket TC for watching soccer, but instead, it may be pasted on each seat in the stadium. That is, a QR code having seat information D2 corresponding to each seat can be attached to each seat.

A-4. Processing by the watching aid application:
FIG. 7 is a flowchart showing the details of the watching assistance routine. The watching assistance routine is a processing routine according to the watching assistance application 511, and is executed by the CPU 140 of the HMD 100. The watching support routine is started upon receiving the start of the watching support application 511. Once the watching support application 511 has been installed in the HMD 100, the watching support routine is started by directly instructing the installed watching support application 511, not the ticket TC, to be installed. be able to.

When the process is started, the CPU 140 first stores the seat information D2 encoded as described above, obtained by reading the QR code C3 , in a predetermined area of the storage unit 120 (FIG. 2) Step S110). When the watching support application 511 is activated, when the QR code C3 is not read, the seat information D2 may be input using the touch pad 14 or the cross key 16.

  Next, the CPU 140 determines whether the seat determined by the seat information D2 is included in the upper stage area of the stadium (step S120). The upper stage area is an area preset on the upper stage side of the stadium, and in the present embodiment, the upper stage at step S120 can provide a special service to the audience included in the upper stage area. It is judged whether it is included in the area. If it is determined in step S120 that the seat determined by the seat information D2 is not included in the upper area, the CPU 140 ends the watching support routine.

  On the other hand, when it is determined in step S120 that the seat determined by the seat information D2 is included in the upper area, the CPU 140 detects the current position of the HMD 100 by the GPS module 134 (step S140). It is judged whether it is in the stadium which is performing (step S150). Here, when it is determined that the current position is in the stadium, the CPU 140 turns on the telephoto function of the outside view shooting camera 61 and starts shooting (step S160). In addition, at the time of the imaging | photography, an imaging | photography moving image is preserve | saved at the memory | storage part 120 (step S170). Also, at the time of shooting, the processes of steps S140 and S150 are repeated, and when it is determined at step S150 that the current position of the HMD 100 is within the stadium, the shooting is continued at steps S160 and S170.

  As described above, since the outside view photographing camera 61 can capture an outside view in the direction in which the user is facing, according to the HMD 100, the pattern of the competition in the direction in which the user is facing is automatically Can be shot in telephoto. Note that the HND 100 can reproduce the photographed content, that is, the photographed moving image stored in the storage unit 120 using a separately prepared reproduction application in the middle of telephoto photographing.

  FIG. 8 is an explanatory view showing an example of information visually recognized by the user when the photographed moving image is reproduced. FIG. 8 illustrates the visual field VR2 of the user. The visual field VR2 includes the outside scene SC2 in the stadium as illustrated. Further, the reproduction area AR2 is displayed superimposed on the outside scene SC2. The playback area AR2 includes the captured moving image captured at the above-described telephoto.

  When it is determined in step S150 of FIG. 7 that the current position of the HMD 100 is not within the stadium, that is, the HMD 100 has moved to the outside of the stadium, the CPU 140 advances the process to step S130 and stores the information in the storage unit 120 according to step S170. Delete stored recorded videos. Thereafter, the CPU 140 ends the watching support routine.

  The processing of step S110 and step S120 in the watching support routine corresponds to the usage environment determination unit (FIG. 2), and the processing of step S160 and step S170 corresponds to the processing control unit (FIG. 2).

  According to the HMD 100 of the first embodiment configured as described above, when the use environment in the stadium is in the seat environment such as the upper stage area, the pattern in the direction in which the user is facing is automatically photographed in telephoto. Ru. For this reason, the user automatically reproduces a captured moving image photographed and stored in telephoto when "I can not see well" or "I want to play again again" because of the upper area. Can be seen again. Therefore, according to the HMD 100, the imaging function suitable for the seat environment of the upper stage area can be easily realized. Moreover, the convenience of the user in the seating environment of the upper stage area can be improved. In addition, since the captured image is deleted when it goes out of the stadium, it is possible to prevent the captured image from being taken out of the stadium.

  As a modification of the first embodiment, when it is determined that the HMD 100 has moved to the outside of the stadium, it may be configured to lock the reproduction of the captured moving image instead of the configuration for deleting the captured moving image. By locking the outside of the stadium, it is possible to prohibit the reproduction of the above-mentioned photographed moving image outside the stadium, and when returning to the stadium, it is possible to reproduce the past photographed moving image. Further, as another modified example of the first embodiment, the processes of steps S110 and S120 in FIG. 7 may be deleted. According to this modification, it is possible to perform telephoto shooting at any position in the stadium, and when moving outside the stadium, the shot image is deleted. Also in this modification, in place of the configuration for deleting the captured moving image, the playback of the captured moving image may be locked.

B. Second embodiment:
FIG. 9 is an explanatory view showing a usage pattern of the HMD according to the second embodiment of the present invention. The HMD 600 according to the second embodiment is different from the HMD 100 according to the first embodiment in that it has an opera (Opera) operation assistance application, and is the same in the remaining points. The same parts as in the first embodiment are given the same reference numerals as in the first embodiment, and the following description will be made. As illustrated, the HMD 600 according to the second embodiment is used in a theater performing an opera. The opera viewing assistance application is, for example, downloaded from the website of the theater to be used and installed in advance in the HMD 600.

  The BLE terminal 610 is disposed behind the last seat of the theater, and the BLE terminal 610 outputs an iBeacon signal. The iBeacon signal holds at least the distance to the BLE terminal 610. When the user wearing the HMD 600 enters the theater, the user first activates the opera viewing app.

  FIG. 10 is a flowchart showing details of the viewing assistance routine. The viewing assistance routine is a processing routine according to the opera viewing assistance application, and is executed by the CPU 140 of the HMD 600. In this viewing assistance routine, an iBeacon processing unit 172 (FIG. 2) is used.

When the process is started, the CPU of the HMD 600 first determines whether a signal of iBeacon from the BLE terminal 610 has been detected (step S210). The CPU 140 repeatedly executes the process of step S210 until it detects an iBeacon signal, and if it is determined in step S210 that the distance is held by the sensed signal is less than a predetermined value (for example, 7 m) It is determined whether or not it is (step S220). The determination as to whether the distance is less than a predetermined value is to determine whether the seat of the user wearing the HMD 600 is included in a predetermined range on the rear side of the theater. That is, the determination of whether the distance is less than the predetermined value determines whether the usage environment of the HMD 600 is in a seat environment of the rear seat. Here, if it is determined that the distance is not less than the predetermined value, the CPU 140 returns the process to step S210.

If it is determined in step S220 that the distance is less than the predetermined value, the CPU 140 determines that the use environment of the HMD 600 has shifted to the rear seat environment, and the voice processing unit included in the application layer 510 (FIG. 4). The frame 523 and the camera application 513 are locked (steps S230 and S240). Since the audio processing unit frame 523 bears all of the audio output of the HMD 600 , all the audio output from the HMD 600 is extinguished (muted) by locking the audio processing unit frame 523.

On the other hand, with regard to the photographing function using the outside view photographing camera 61, marker recognition processing (processing to recognize the QR code in the first embodiment, etc.) by other applications included in the application layer 510 besides the camera application 513 Although it is also included, in the present embodiment, shooting by the camera application 513 is prohibited, and shooting according to the marker recognition process is not prohibited (that is, permitted). That is, in the present embodiment, the photographing by the camera application 513 available for the purpose of the voyage is prohibited, and the photographing according to the marker recognition process which can not be used as the voyage is permitted. As a modification, the camera library 536 included in the library layer 530 may be locked to prohibit all shooting.

  After execution of step S240, the CPU 140 starts display of guidance information (step S250). That is, the CPU 140 starts displaying the guidance information on the image display unit 20. As illustrated in FIG. 9, the user's visual field VR3 includes the outside scene SC3 in the theater, and is superimposed on the outside scene SC3 to display a guidance information display area AR3. The guidance information GI is included in the guidance information display area AR3. The guidance information GI is detailed information on techniques, backgrounds, highlights, etc. about the opera (performing work) being performed. In the present embodiment, the HMD 600 detects the start of the performance from the start time, and switches the guidance information included in the guidance information display area AR3 in order according to the performance time. Further, in the present embodiment, in the visual field VR3, the guidance information display area AR3 is disposed at a position shifted from the center thereof. As a result, the user can view the guidance information GI without disturbing the performance contents. After the execution of step S250, the viewing assistance routine is ended, but the display of the started guidance information is continued until the performance is ended.

  The processing of steps S210 and S220 in the viewing assistance routine corresponds to the usage environment determination unit, and the processing from steps S230 to S250 corresponds to the processing control unit. The process control unit may be configured to execute only one of steps S230 to S250 instead of the configuration for executing all the processes from steps S230 to S250, or any two steps. It may be configured to execute.

  According to the HMD 600 of the second embodiment configured as described above, when the use environment in the theater is in a seat environment such as a rear seat, guidance information for the opera being performed is included along with the performance content visually recognized as the outside scene SC3. I can see it. In the backseat seating environment in theaters (especially large theaters), the objects to be viewed are far away and occupy less of the field of view VR. For this reason, sufficient space can be provided to display the guidance information GI in the field of view VR. Therefore, according to the HMD 600, it is possible to easily realize the display function suitable for the seat environment of the rear seat. In addition, the convenience of the user in the seat environment of the rear seat can be improved.

C. Third embodiment:
FIG. 11 is an explanatory view showing a use mode of the HMD according to the third embodiment of the present invention. The HMD 700 according to the third embodiment is different from the HMD 100 according to the first embodiment in that a movie viewing assistance application is provided, and the remaining points are the same. The same parts as in the first embodiment are given the same reference numerals as in the first embodiment, and the following description will be made. As illustrated, the HMD 700 according to the third embodiment is used in a movie theater. The movie viewing assistance application is, for example, downloaded from the home page of the movie theater to be used and installed in advance in the HMD 700.

A first BLE terminal 710 is disposed at a first position in the movie hall, and a second BLE terminal 720 is disposed at a second position different from the first position in the movie theater. A signal of iBeacon is output from each of the BLE terminals 710 and 720. The iBeacon signal holds at least the BLE terminal identification number for identifying the BLE terminals 710 and 720, and the distance to each BLE terminal 710 and 720. When a user wearing the HMD 700 enters a movie theater, the user first activates a movie viewing assistance application.

  FIG. 12 is a flowchart showing details of the viewing assistance routine. The viewing assistance routine is a processing routine according to the movie viewing assistance application, and is executed by the CPU 140 of the HMD 700. In this viewing assistance routine, an iBeacon processing unit 172 (FIG. 2) is used.

When the process is started, the CPU of the HMD 700 first determines whether or not a signal of iBeacon is sensed from at least one of the two BLE terminals 710 and 720 (step S310). The CPU 140 repeatedly executes the process of step S310 until it senses the iBeacon signal, and if it is determined in step S310 that it is sensed, whether the distance held by the sensed signal is less than a predetermined value (for example, 5 m) It is determined whether or not (step S320). When a plurality of iBeacon signals are sensed, the signal with the shorter distance to be held is regarded as the “sensed signal”. Whether or not the distance is less than a predetermined value is determined by setting the radius of the seat of the user wearing the HMD 700 centered on the first position where the first BLE terminal 710 is disposed, to the predetermined value. Whether the radius is included in one of the first range E1 and the second range E2 having the predetermined value and centered at the second position where the second BLE terminal 720 is disposed. To determine the That is, the determination as to whether the distance is less than the predetermined value determines whether the usage environment of the HMD 700 is included in the first range E1 or the second range E2. Here, if it is determined that the distance is not less than the predetermined value, the CPU 140 returns the process to step S310.

If it is determined in step S320 that the distance is less than the predetermined value, the CPU 140 determines that the usage environment of the HMD 700 has shifted to the environment included in the first range E1 or the second range E2, and the CPU 140 determines The audio processing unit frame 523 included in 510 (FIG. 4) and the camera application 513 are locked (steps S330 and S340). Steps S330 and S340 are the same as steps S230 and S240 (FIG. 10) of the second embodiment.

After execution of step S330, CPU 140 starts displaying subtitles of the language corresponding to the BLE terminal identification number held in the sensed signal (step S350 ). In this embodiment, subtitles are displayed in Japanese in a first range E1 in a movie theater, and subtitles are displayed in Chinese in a second range E2 in a movie theater. Therefore, in step S350 , when the BLE terminal identification number held in the sensed signal indicates the first BLE terminal 710, display of Japanese subtitles is started, and BLE held in the sensed signal. When the terminal identification number indicates the second BLE terminal 710, display of Chinese subtitles is started. After the execution of step S350 , the viewing assistance routine is ended, but the display of the started subtitles is continued until the end of the screening.

  The processing of steps S310 and S320 in the viewing assistance routine corresponds to the usage environment determination unit, and the processing from steps S330 to S350 corresponds to the processing control unit. The process control unit may be configured to execute only one of steps S330 to S350 instead of the configuration for executing all the processes from steps S330 to S350, or any two steps. It may be configured to execute.

  According to the HMD 700 of the third embodiment configured as described above, when the use environment in the movie theater is in the seat environment such as the first range E1, Japanese with the screen SR viewed through as the outside scene SC4 You can see the subtitles of. In addition, when the use environment in the movie theater is in the seat environment such as the second range E2, Chinese subtitles can be viewed together with the screen SR visually recognized as the outside scene SC5. Therefore, according to the seating environment, the HMD 600 can easily switch the subtitle language. In addition, the convenience of the user in the seat environment can be improved.

D. Fourth Embodiment:
FIG. 13 is an explanatory view showing a usage pattern of the HMD according to the fourth embodiment of the present invention. The HMD 800 according to the fourth embodiment is different from the HMD 100 according to the first embodiment in that the application execution restriction application is included, and the remaining points are the same. The HMD 800 according to the fourth embodiment is used in an office 810 that includes a security room 820 that requires a high level of security. At the center of the security room 820, a BLE terminal 830 is provided. When the distance between the BLE terminal 830 and the HMD 800 becomes less than a predetermined value (for example, 5 m), it is considered that the user wearing the HMD 800 has entered the security room 820. HMD800, when the distance from the BLE terminal 830 is determined to be less than a predetermined value, thereby permitting execution of a predetermined application program, so as to allow connections to predetermined network. On the other hand, when it is determined that the distance from the BLE terminal 830 is equal to or greater than a predetermined value, or when the iBeacon signal from the BLE terminal 830 can not be sensed, execution of the predetermined application program is prohibited and Prohibit connection.

  According to the HMD 800 according to the fourth embodiment configured as described above, execution of a predetermined application is permitted or prohibited depending on whether the usage environment is in a security environment requiring a high level of security. Connection to the network is permitted or prohibited. Therefore, according to the security environment, the HMD 800 can easily switch between permission / prohibition of execution of a predetermined application and permission / inhibition of connection to a predetermined network. In the present embodiment, although both the execution of the predetermined application and the connection to the predetermined network are permitted or prohibited, instead of this, the execution of the predetermined application and the connection to the predetermined network Only one of them may be performed.

E. Modification:
The present invention is not limited to the above first to fourth embodiments and their modifications, and can be implemented in various embodiments without departing from the scope of the present invention, for example, as follows: Variations are also possible.

E-1. Modification 1:
The differences in usage environment include the following.
[1] Geographic differences such as countries, provinces and provinces, cities and administrative districts.
[2] Specific facilities such as stadiums, theaters, museums, museums, cinemas, concert halls, research institutes, companies, department stores, factories, schools, hospitals, highways, stadiums, construction sites, zoos, botanical gardens, etc. Inside or outside? In this case, the distinction between inside and outside may be judged as stadium general and theater general. ○ ○ ○ It may be determined for a specific facility such as an art museum.
[3] In or out of a specific area, such as sand dunes, wetlands, beaches, mountainous areas, etc. In this case, the distinction between inside and outside may be judged as dunes in general, and wetlands in general. It may be judged for specific facilities such as ○ wetlands.
[4] Specific transportation means such as airplanes, trains, ships, inside and outside of vehicles. In this case, the distinction between inside and outside may be determined as a general plane or a general train. It may be determined for a specific transportation means such as ○ flight.
[5] A specific position inside the above [2] to [4], for example, the first to fourth embodiments.
[6] Elements felt by the user's five senses (vision, hearing, smell, taste, touch) at the user's location, such as illuminance, temperature, humidity, noise level, environmental sound difference, odor, air volume, air direction, You may recognize differences such as snowfall and rainfall as differences in usage environment. The difference in the usage environment may be recognized by elements of the environment that can not be felt by the five senses, such as low frequency sound, infrared radiation, ultraviolet radiation, radio wave intensity, magnetic field intensity, and the like.
[7] The differences in usage environment may be distinguished beforehand by statistical or statistical methods. For example, it can be indexed by the difference in safety (which can be indexed by crime rate, accident rate, damage rate etc.) of the place where the user is, convenience (distance to station, number of convenience stores, distance to school etc) ), And differences in sociality (public space, private space, and so on). Such distinction may be made into an index in advance and associated with position information, or may be obtained each time via a network such as the Internet.

Specifically, for example, as a configuration in which the camera application 513 is locked when a transition to a public space such as a train, a car, or an airplane is detected, it is possible to prevent spycams. Good. For example, when movement to a company is detected, execution of a business application program may be permitted. For example, in an environment inside an aircraft, it is possible to prohibit the use of a cell phone and a wireless LAN which are the basis of jamming, and to enable Bluetooth and iBeacon which have less influence on electronic devices of the aircraft. For example, when it is detected by the GPS or the like that the usage environment of the HMD is a specific country, the function defined according to the country may be changed. For example, it is possible to change the language used in the HMD, change the use unit such as the length and mass displayed in the HMD, and change the output restriction of the wireless device of the HMD.

E-2. Modification 2:
The determination of the use environment includes the following.
[1] Judgment based on captured image <1> A method of directly determining from the captured outside scene. For example, landscapes, landscapes, buildings, etc. are determined using image matching.
<2> A method of judging based on markers included in the captured outside scene. The marker includes a bar code, a QR code (registered trademark), a signboard of a specific form, a specific sign (for example, a sign such as “shooting prohibited” or “flash prohibited”) mark, a character string, and the like. The markers may be fixed objects (signboards, road marks, etc.) of land or facilities, or markers (placards, posters, etc.) that can be easily installed or carried.
<3> Other than the outside view, it is judged from a specific captured image such as an endoscopic image in surgery. A method of judging based on markers contained in a specific captured image.
[2] Judgment based on recorded sound <1> Analysis of the sound to judge beach, city, etc.
<2> Judgment based on a specific pattern included in the sound. Music flowing, beacons in the audible range, etc.
[3] Judgment by signals such as radio waves and magnetic fields. For example, wireless LAN, Bluetooth, iBeacon etc.

E-3. Modification 3:
The predetermined functions to be changed include the following.
[1] Shooting Function As shown in the first embodiment, the function for shooting a moving image may be replaced with a function for shooting a still image. Also, as shown in the first embodiment, it may be a function of telephoto shooting, or may be a function of infrared shooting, or another type of shooting function such as panoramic shooting. Further, in the first embodiment, switching of driving on / off is performed as the change of the photographing function by the camera, but instead, change of resolution, switching of infrared photographing on / off, video function On / off switching, color / black and white switching, continuous shooting on / off switching, split shooting on / off switching, storage format (low data) switching, switching of whether to save the captured image, shooting It is also possible to switch the assignment of a password to an image, switch various filter photography, or the like. That is, in addition to the on / off of the imaging function, the imaging function may be partially stopped or partially activated. For example, when the transition to the public space is detected, limiting the resolution of the camera, for example, to drop the camera of 12 million pixels to a function of 300,000 pixels at a level that can not recognize human face, for example Good.

[2] Recording Function As shown in the first embodiment, the recording function may be used instead of the recording function. For example, the recording function may be prohibited in the use environment of the concert hall. Further, as a change of the recording function, a change of on / off of the microphone, a change of on / off of the sound recording function itself, a change of presence / absence of the recording voice, a change of presence / absence of the bass word may be made.

[3] Voice Function In the second and third embodiments, as the process control unit, the voice processing unit frame 523 is locked to suppress the voice function, but instead, the voice output level is changed. The configuration may be reduced (that is, the volume is reduced). For example, the configuration may be configured to promote the degree of audio output (that is, increase the volume). That is, in addition to the on / off of the imaging function, the imaging function may be partially stopped or partially activated. In addition, as change of sound function, change of sound range, change of bit rate (such as AM broadcast, high quality, etc.), change of Dolby on / off, change of number and kind of usable sound source (synthesizer, voice synthesis), stereo The present invention can be applied to various switches such as monophonic change and left / right change.

[4] Information Presentation Function As the information presentation function, in the second embodiment, guidance information is displayed, and in the third embodiment, subtitles are displayed, and it is switched whether to display these or not. However, in place of this, the display capability may be switched. For example, switching between binocular display using the right display drive unit 22 and the left display drive unit 24 and monocular display using one of them, switching of resolution, switching of 3D display and 2D display, switching of color / black and white, The transmittance may be switched, the display luminance may be switched, the number of moving image frames may be switched, the display language may be switched, or the like. Moreover, it is good also as on-off switching of an advertisement display. In addition, as a configuration for suppressing the information presentation function, a configuration for suppressing a predetermined function, such as a configuration for reducing the amount of information of guidance information to be displayed, may be employed. In addition, for example, the information presentation function may be promoted, such as a configuration in which the information amount of the guidance information to be displayed is increased.

[5] Change of the function that HMD cooperates with the outside For example, as change of communication function, switching of communication function ON / OFF, switching of communication speed, switching of communication distance, switching of connection destination network range, automatic connection It is also possible to switch the availability or the like. In addition, switching on / off of Bluetooth and iBeacom, switching on / off of Bluetooth multilink, switching on / off of reading function of RF tag, switching on memory access range, switching on / off of access to external memory, live It is also possible to switch the rally or the driver's access range. Besides, various sensors may be switched on / off.

  The predetermined functions to be changed include various things other than the above [1] to [5]. In addition, the change in function includes on, off, suppression, promotion, partial stop, and partial activation. Furthermore, the change of function includes addition or removal of the function.

E-4. Modification 4:
In each of the above embodiments, as a method of changing the function, an element included in the application layer 510 (for example, the camera application 513 ) is changed, or an element included in the framework layer 520, for example, the audio processing unit frame 523). In the above, the elements included in the library layer 530 are changed, the elements included in the kernel layer 540 are changed, and the elements included in the hardware layer 550 are changed. May be

E-5. Modification 5:
In each of the above embodiments, the usage environment of the HMD is a usage environment switched by moving the HMD, but instead, it may be a usage environment switched according to a change in the HMD surrounding regardless of the movement of the HMD. . For example, a change in season may be detected, and in summer, the brightness of display by the image display unit 20 may be made higher than that in winter.

E-6. Other variations:
In the above embodiment, the configuration of the head mounted display is illustrated. However, the configuration of the head mounted display can be arbitrarily determined without departing from the scope of the present invention. For example, addition, deletion, conversion, and the like of each component can be performed.

  The allocation of the component to the control unit and the image display unit in the above embodiment is just an example, and various aspects can be adopted. For example, the following embodiment may be used. (I) A control unit equipped with processing functions such as CPU and memory, an image display unit equipped only with a display function, and (ii) Both control unit and image display unit with processing functions such as CPU and memory A mode of mounting, (iii) a mode in which a control unit and an image display unit are integrated (for example, a mode in which the control unit is included in the image display unit and functions as a glasses-type wearable computer), (iv) instead of the control unit The aspect which uses a smart phone or a portable game machine, the aspect which eliminated the connection part (cord | chord) by making the control part and the image display part into the structure which can be wirelessly communicated and wirelessly supplied.

  In the above embodiment, for convenience of explanation, the control unit includes the transmission unit, and the image display unit includes the reception unit. However, each of the transmission unit and the reception unit in the above-described embodiment has a function capable of bi-directional communication, and can function as a transmission / reception unit. Also, for example, the control unit illustrated in FIG. 2 is connected to the image display unit via a wired signal transmission path. However, the control unit and the image display unit may be connected by connection via a wireless LAN, infrared communication, or a wireless signal transmission path such as Bluetooth (registered trademark).

  For example, the configurations of the control unit and the image display unit described in the above embodiment can be arbitrarily changed. Specifically, for example, the touch pad may be omitted from the control unit and operated with only the cross key. In addition, the control unit may be provided with another operation interface such as an operation stick. The control unit may be connected to a device such as a keyboard and a mouse, and may receive an input from the keyboard or the mouse. For example, in addition to the operation input by the touch pad and the cross key, the operation input by the foot switch (a switch operated by the user's foot) may be acquired. For example, after providing a line-of-sight detection unit such as an infrared sensor in the image display unit, the line-of-sight of the user may be detected, and an operation input by a command associated with the movement of the line of sight may be acquired. For example, a gesture of the user may be detected using a camera, and an operation input by a command associated with the gesture may be acquired. In the case of gesture detection, the user's fingertip, a ring attached to the user's hand, a medical instrument used by the user's hand, etc. can be used as a mark for motion detection. If it is possible to acquire an operation input by a foot switch or a line of sight, the input information acquisition unit can acquire an operation input from the user even in an operation where it is difficult for the user to release the hand.

  FIG. 14 is an explanatory view showing the configuration of the appearance of the HMD in the modification. In the example of FIG. 14A, the image display unit 20x includes a right optical image display unit 26x instead of the right optical image display unit 26, and replaces the left optical image display unit 28x instead of the left optical image display unit 28. Have. The right optical image display unit 26 x and the left optical image display unit 28 x are formed smaller than the optical members of the above embodiment, and are disposed obliquely above the right eye and the left eye of the user when the HMD is attached. . In the example of FIG. 14B, the image display unit 20y includes the right optical image display unit 26y in place of the right optical image display unit 26, and the left optical image display unit 28y in place of the left optical image display unit 28. Have. The right optical image display unit 26y and the left optical image display unit 28y are smaller than the optical members of the above-described embodiment, and are disposed obliquely below the right eye and the left eye of the user when the HMD is attached. . Thus, it is sufficient if the optical image display unit is disposed in the vicinity of the user's eye. In addition, the size of the optical member forming the optical image display unit is also arbitrary, and the optical image display unit covers only a part of the user's eye, in other words, the optical image display unit completely covers the user's eye. It can also be realized as an HMD in an uncovered manner.

  For example, although the head mounted display is a binocular transmission type head mounted display, it may be a monocular head mounted display. In addition, the head mount display may be configured as a non-transmission type head mount display in which the transmission of the outside scene is blocked when the user wears the head mount display.

  For example, functional units such as an image processing unit, a display control unit, and an audio processing unit are described as being realized by the CPU expanding a computer program stored in a ROM or a hard disk into a RAM and executing the computer program. However, these functional units may be configured using an application specific integrated circuit (ASIC) designed to realize the function.

  For example, in the above embodiment, the head mounted display in which the image display unit is mounted like glasses is a flat mount type display device having a normal image display unit (liquid crystal display device, plasma display device, organic EL display device, etc. ) May be. Also in this case, the connection between the control unit and the image display unit may be a connection via a wired signal transmission path, or may be a connection via a wireless signal transmission path. In this way, the control unit can also be used as a remote control of a conventional flat display device.

  Further, as the image display unit, instead of the image display unit worn like glasses, an image display unit with another shape such as an image display unit worn like a hat may be adopted. Further, the earphone may be of an ear hook type or a head band type, or may be omitted. In addition, for example, it may be configured as a head-up display (HUD, Head-Up Display) mounted on a vehicle such as an automobile or an airplane. Also, for example, it may be configured as a head mounted display incorporated in a body protector such as a helmet.

  For example, in the above embodiment, the display driving unit is configured using a backlight, a backlight control unit, an LCD, an LCD control unit, and a projection optical system. However, the above aspect is merely an example. The display drive unit may be equipped with a configuration unit for implementing another scheme, in addition to or instead of these components. For example, the display driving unit may be configured to include an organic EL (organic electro luminescence, organic electro luminescence) display, an organic EL control unit, and a projection optical system. For example, the display driver can use DMD (digital micro mirror device) or the like instead of the LCD. For example, the display drive unit may include a signal light modulation unit including a color light source for generating RGB color light and a relay lens, a scanning optical system including a MEMS mirror, and a drive control circuit for driving these. May be configured. As described above, even if the organic EL, the DMD, or the MEMS mirror is used, the “emission area in the display drive unit” does not change in that the image light is actually emitted from the display drive unit, and each device By controlling the emission area in (display drive unit) in the same manner as in the above embodiment, the same effect as in the above embodiment can be obtained. Also, for example, the display driving unit may be configured to include one or more lasers that emit a laser having an intensity corresponding to the pixel signal to the user's retina. In this case, the “emission area in the display drive unit” represents an area where the laser light representing an image is actually emitted from the display drive unit. By controlling the emission area of the laser light in the laser (display drive unit) in the same manner as in the above embodiment, the same effect as in the above embodiment can be obtained.

  The present invention is not limited to the above-described embodiments, examples, and modifications, and can be implemented with various configurations without departing from the scope of the invention. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in the respective forms described in the section of the summary of the invention are for solving some or all of the problems described above, or In order to achieve part or all of the above-described effects, replacements or combinations can be made as appropriate. Also, if the technical features are not described as essential in the present specification, they can be deleted as appropriate.

10: Control unit (controller)
12 Lighting part 14 Touch pad 16 Cross key 18 Power switch 20 Image display part 21 Right holding part 22 Right display drive part 23 Left holding part 24 Left display drive part 26 Right optical image display part 28 ... left optical image display unit 30 ... earphone plug 32 ... right earphone 34 ... left earphone 40 ... connection unit 42 ... right code 44 ... left code 46 ... connecting member 48 ... main body code 51 ... transmission unit 52 ... transmission unit 53 ... reception Unit 54: Reception unit 61: Camera for photographing outside view 110: Input information acquisition unit 100: Head mounted display (HMD)
120 ... storage unit 130 ... power supply 132 ... wireless communication unit 140 ... CPU
160 ... image processing unit 162 ... display control unit 164 ... usage environment determination unit 166 ... processing control unit 170 ... voice processing unit 180 ... interface 201 ... right backlight control unit 202 ... left backlight control unit 211 ... right LCD control unit 212 ... Left LCD control unit 221 ... Right backlight 222 ... Left backlight 241 ... Right LCD
242: Left LCD
251 ... right projection optical system 252 ... left projection optical system 261 ... right light guide plate 262 ... left light guide plate 500 ... platform 510 ... application layer 511 ... watching support application 512 ... game application 513 ... camera application 514 ... code reader application 520 ... frame Work layer 530 ... library layer 533 ... display library 534 ... audio library 535 ... sensor library 536 ... camera library 537 ... library 540 ... kernel layer 542 ... voice IC driver 543 ... sensor driver 544 ... image sensor driver 550 ... Hardware layer 552 ... voice IC device 553 ... sensor device 554 ... image sensor device 600, 700, 800 ... head mounted display (HMD)
610, 710, 720 ... BLE terminal 820 ... security room

Claims (7)

A transmissive head-mounted display device,
An image display unit capable of displaying an image to allow a user wearing the head-mounted display device to view the image and to transmit an outside scene;
A use environment determination unit that determines a use environment of the head-mounted display device;
A processing control unit that changes at least a part of a predetermined function mounted on the head-mounted display device according to the determined usage environment;
Equipped with
The predetermined function is a telephoto photographing function of photographing a telephoto by a camera unit,
The use environment includes at least a specific seating environment in the stadium,
The processing control unit turns on the telephoto imaging function and performs telephoto imaging with the camera unit when the usage environment determination unit determines that the seat environment is the specific one.
The head-mounted display device further includes an erasing unit that erases a captured image recorded by the telephoto imaging when it is detected that the head-mounted display device has moved out of the stadium. Head-mounted display. The head mounted display according to claim 1, wherein
The head-mounted display device, wherein the usage environment is a usage environment switched by moving the head-mounted display device. The head mounted display according to claim 1 or 2, wherein
It has a shooting unit that shoots,
The use environment determination unit
The head-mounted display device, wherein the usage environment is determined based on a photographed image obtained by the photographing unit. The head mounted display according to claim 3, wherein
The use environment determination unit
The head-mounted display device that determines that the specific usage environment is when the photographed image obtained by the photographing unit includes a marker for recognizing the specific usage environment. The head mounted display according to claim 1 or 2, wherein
The use environment determination unit
A head-mounted display device that determines the usage environment based on a signal from an external wireless communication terminal. A control method of a transmission-type head-mounted display device having an image display unit capable of displaying an image to allow a user to visually recognize the image and transmitting an outside scene.
Determining the usage environment of the head mounted display;
Changing at least a part of a predetermined function mounted on the head-mounted display device according to the determined use environment;
The predetermined function is a telephoto photographing function of photographing a telephoto by a camera unit,
The use environment includes at least a specific seating environment in the stadium,
When it is determined that the particular seat environment is, the telephoto shooting function is turned on to perform a telephoto shooting with the camera unit,
The control method of the head-mounted type display device which erase | eliminates the picked-up image recorded by the said telephoto imaging when it is detected that the said head-mounted type display apparatus moved out of the said stadium . A computer program for controlling a transmission-type head-mounted display device having an image display unit capable of displaying an image to allow a user to view the image and transmitting an outside scene,
On the computer
A function of determining a use environment of the head mounted display device;
A function of changing at least a part of a predetermined function mounted on the head mounted display device according to the determined use environment;
To achieve,
The predetermined function is a telephoto photographing function of photographing a telephoto by a camera unit,
The use environment includes at least a specific seating environment in the stadium,
On the computer
A function of turning on the telephoto imaging function and performing telephoto imaging with the camera unit when it is determined that the specific seating environment is selected;
A function of deleting a photographed image recorded by the telephoto photographing when it is detected that the head-mounted display device has moved out of the stadium.
A computer program that makes you realize .

Images