JP2016040865A - Head mounted display and control method for the same, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To mitigate a burden on a user's eye.SOLUTION: A transmission type head mounted display comprises: an image display unit capable of transmitting outside scenery while displaying an image to make a user mounting the head mounted display visually recognize the image; a movement detection unit for detecting the head mounted display having moved to a specific position (steps S110, S120); and a processing control unit for changing at least part of a predetermined function mounted on the head mounted display (steps S170, S180).SELECTED DRAWING: Figure 7

Description

  The present invention relates to a head-mounted display device, a control method thereof, and a computer program.

  In recent years, a head-mounted display device that is a display device mounted on the head is known. This head-mounted display device is also called a head mounted display (HMD), and there is a see-through type device that allows a user to see the outside scene while wearing the HMD. The see-through type HMD reflects image light generated by a light modulation element such as a liquid crystal panel with an optical system or the like disposed in front of the user's eyes, thereby creating a virtual image along with the outside scene (real image) in the user's field of view. Are displayed (for example, Patent Document 1).

JP 2010-139901 A

  Conventionally, see-through HMDs have been proposed that are equipped with various functions including the function of displaying the image. As various functions, there are various functions such as a photographing function using a camera and a function of outputting sound. There has been a problem that sufficient consideration has not been given to whether these functions can be operated at all times. In addition, in the conventional head-mounted display device, the convenience of the user is improved, the detection accuracy is improved, the malicious use such as voyeurism is prevented, the device configuration is made compact, the cost is reduced, the resource is reduced, It has been desired to facilitate the production.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms.

(1) One embodiment of the present invention is a transmissive head-mounted display device. The head-mounted display device includes: an image display unit that displays an image to allow a user wearing the head-mounted display device to visually recognize the image and transmit an outside scene; and the head-mounted display device A movement detecting unit for detecting that the robot has moved to a specific location; and changing at least a part of a predetermined function mounted on the head-mounted display device when movement to the specific location is detected; And a processing control unit. According to the head-mounted display device of this embodiment, when it is detected that the head-mounted display device attached to the user has moved to a specific location, the head-mounted display device is mounted on the head-mounted display device. At least some of the predetermined functions are changed. For this reason, a change is made such that at least a part of a predetermined function mounted on the head-mounted display device is suppressed or expanded in a specific place. Therefore, according to the head-mounted display device of this embodiment, it is possible to easily realize a function suitable for the occasion without a person switching the setting of the function one by one.

(2) The head-mounted display device according to the aspect described above includes an outside scene photographing unit that photographs the outside scene, and the movement detection unit moves to the specific place based on a photographed image obtained by the outside scene photographing unit. You may make it detect the movement of. Since the outside scene photographing unit is mounted on the head-mounted display device, the photographing range moves reliably with the movement of the user's head. For this reason, according to the head-mounted display device of the present embodiment, movement to a specific place can be detected with high accuracy.

(3) In the head-mounted display device according to the above aspect, the movement detection unit may move to the specific location when a marker for recognizing the specific location is included in the captured image. It may be assumed that it has been detected. According to this form of the head-mounted display device, movement to a specific location can be detected with high accuracy by photographing the marker.

(4) In the head-mounted display device of the above aspect, the movement detection unit may detect movement to the specific location based on a signal from an external wireless communication terminal. According to this form of the head-mounted display device, movement to a specific location can be detected with high accuracy.

(5) In the head-mounted display device according to the above aspect, the predetermined function may be an information presentation function for displaying predetermined information on the image display unit. According to this form of the head-mounted display device, an information presentation function for displaying predetermined information on the image display unit can be suppressed or expanded. Therefore, it is possible to easily present information suitable for the situation.

(6) In the head-mounted display device according to the above aspect, the specific location is around an exhibit or a building, and the processing control unit detects when movement to the specific location is detected. Information about the exhibit or building may be displayed on the image display unit. According to the head-mounted display device of this form, when moving to the periphery of an exhibit or building, information about the exhibit or building can be displayed to the user by image display. Therefore, it is possible to easily present information suitable for an exhibit or a building.

(7) In the head-mounted display device of the above aspect, the predetermined function is an imaging function including at least a marker imaging function performed for recognizing a predetermined marker, and the processing control unit When movement to a place is detected, the marker photographing function among the photographing functions may be permitted, and the photographing function excluding the marker photographing function among the photographing functions may be prohibited. According to this form of the head-mounted display device, since the photographing function excluding the marker photographing function is prohibited only by being moved to a specific place, it is possible to reliably prevent unauthorized use such as voyeurism. Can do.

(8) In the head-mounted display device according to the above aspect, when the movement to the specific place is detected, the processing control unit stops the predetermined function and moves from the specific place to another place. When the movement of the head-mounted display device is detected, the stop of the predetermined function may be released. According to the head-mounted display device of this aspect, the predetermined processing once prohibited can be automatically restored, so that the convenience of the user can be improved.

(9) In the head-mounted display device according to the aspect described above, when the escape of the head-mounted display device is detected from the specific location, the processing control unit notifies the user that the user has escaped. You may make it alert | report. According to this form of the head-mounted display device, the user can know that he / she has escaped from a specific place, and the convenience of the user can be improved.

  A plurality of constituent elements of each embodiment of the present invention described above are not essential, and some or all of the effects described in the present specification are to be solved to solve part or all of the above-described problems. In order to achieve the above, it is possible to appropriately change, delete, replace with another new component, and partially delete the limited contents of some of the plurality of components. In order to solve some or all of the above-described problems or achieve some or all of the effects described in this specification, technical features included in one embodiment of the present invention described above. A part or all of the technical features included in the other aspects of the present invention described above may be combined to form an independent form of the present invention.

  For example, one embodiment of the present invention can be realized as an apparatus including one or more or all of the three elements of the image display unit, the movement detection unit, and the processing control unit. That is, this apparatus may or may not have an image display unit. Further, the apparatus may or may not have a movement detection unit. Further, the apparatus may or may not have a processing control unit. For example, the image display unit may display an image so that a user who wears the head-mounted display device visually recognizes the image and can transmit the outside scene. For example, the movement detection unit may detect that the head-mounted display device has moved to a specific location. For example, when the movement to a specific place is detected, the processing control unit may change at least a part of a predetermined function mounted on the head-mounted display device. Such a device can be realized as, for example, a head-mounted display device, but can also be realized as a device other than the head-mounted display device. According to such a form, improvement of user convenience, improvement of detection accuracy, prevention of malicious use such as voyeurism, downsizing of device configuration, cost reduction, resource saving, ease of manufacture, etc. At least one of the various problems can be solved. Any or all of the technical features of each form of the head-mounted display device described above can be applied to this device.

  The present invention can also be realized in various forms other than the head-mounted display device. For example, display device, head-mounted display device and display device control method, head-mounted display system, display device, head-mounted display system, and computer program for realizing the functions of the display device, and the computer program Can be realized in the form of a data signal or the like embodied in a carrier wave including the computer program.

It is explanatory drawing which shows schematic structure of the head mounted display apparatus (HMD) in 1st Embodiment of this invention. It is a block diagram which shows the structure of HMD functionally. It is explanatory drawing which shows a mode that image light is inject | emitted by the image light production | generation part. It is explanatory drawing of the platform of HMD. It is explanatory drawing which shows the form of use of HMD in a museum. It is explanatory drawing which shows a gate and a central monitoring apparatus. It is a flowchart which shows the detail of an entrance process routine. It is a flowchart which shows the detail of a regular route guidance process routine. It is explanatory drawing which shows an example of the route guidance message visually recognized by the user. It is a flowchart which shows the detail of an exhibit description routine. It is explanatory drawing which shows an example of the exhibit information visually recognized by the user. It is explanatory drawing which shows the structure of the external appearance of HMD in a modification.

A. First embodiment:
A-1. Configuration of head mounted display device:
FIG. 1 is an explanatory diagram showing a schematic configuration of a head-mounted display device according to the first embodiment of the present invention. The head-mounted display device 100 is a display device mounted on the head, and is also called a head mounted display (HMD). The HMD 100 is an optically transmissive head-mounted display that allows a user to visually recognize a virtual image and at the same time directly view an outside scene. In the present embodiment, the HMD 100 is used for museum guidance.

  The HMD 100 includes an image display unit 20 that allows a user to visually recognize a virtual image when attached to the user's head, and a control unit (controller) 10 that controls the image display unit 20.

  The image display unit 20 is a mounting body that is mounted on the user's head, and has a glasses shape in the present embodiment. The image display unit 20 includes a right holding unit 21, a right display driving unit 22, a left holding unit 23, a left display driving unit 24, a right optical image display unit 26, and a left optical image display unit 28. It is out. The right optical image display unit 26 and the left optical image display unit 28 are arranged so as to be positioned in front of the right and left eyes of the user when the user wears the image display unit 20, respectively. One end of the right optical image display unit 26 and one end of the left optical image display unit 28 are connected to each other at a position corresponding to the eyebrow of the user when the user wears the image display unit 20.

  The right holding unit 21 extends from the end ER which is the other end of the right optical image display unit 26 to a position corresponding to the user's temporal region when the user wears the image display unit 20. It is a member. Similarly, the left holding unit 23 extends from the end EL which is the other end of the left optical image display unit 28 to a position corresponding to the user's temporal region when the user wears the image display unit 20. It is a member provided. The right holding unit 21 and the left holding unit 23 hold the image display unit 20 on the user's head like a temple of glasses.

  The right display drive unit 22 is disposed inside the right holding unit 21, in other words, on the side facing the user's head when the user wears the image display unit 20. Further, the left display driving unit 24 is disposed inside the left holding unit 23. Hereinafter, the right holding unit 21 and the left holding unit 23 will be described as “holding units” without being distinguished from each other. Similarly, the right display drive unit 22 and the left display drive unit 24 are described as “display drive units” without being distinguished from each other, and the right optical image display unit 26 and the left optical image display unit 28 are not distinguished from each other as “optical image display units”. Will be described.

  The display driving unit includes a liquid crystal display (hereinafter referred to as “LCD”) 241, 242, projection optical systems 251, 252, and the like (see FIG. 2). Details of the configuration of the display driving unit will be described later. The optical image display unit as an optical member includes light guide plates 261 and 262 (see FIG. 2) and a light control plate. The light guide plates 261 and 262 are formed of a light transmissive resin material or the like, and guide the image light output from the display driving unit to the user's eyes. The light control plate is a thin plate-like optical element, and is disposed so as to cover the front side of the image display unit 20 (the side opposite to the user's eye side). The light control plate protects the light guide plates 261 and 262 and suppresses damage to the light guide plates 261 and 262 and adhesion of dirt. Further, by adjusting the light transmittance of the light control plate, it is possible to adjust the amount of external light entering the user's eyes and adjust the ease of visual recognition of the virtual image. The light control plate can be omitted.

  The image display unit 20 further includes a connection unit 40 for connecting the image display unit 20 to the control unit 10. The connection unit 40 includes a main body cord 48 connected to the control unit 10, a right cord 42 and a left cord 44 in which the main body cord 48 branches into two, and a connecting member 46 provided at the branch point. . The connecting member 46 is provided with a jack for connecting the earphone plug 30. A right earphone 32 and a left earphone 34 extend from the earphone plug 30.

  The image display unit 20 and the control unit 10 transmit various signals via the connection unit 40. Each end of the main body cord 48 opposite to the connecting member 46 and the control unit 10 are provided with connectors (not shown) that are fitted to each other. The connector of the main body cord 48 and the control unit 10 The control unit 10 and the image display unit 20 are connected to or disconnected from each other by the fitting / releasing of the connector. For the right cord 42, the left cord 44, and the main body cord 48, for example, a metal cable or an optical fiber can be adopted.

  The control unit 10 is a device for controlling the HMD 100. The control unit 10 includes a lighting unit 12, a touch pad 14, a cross key 16, and a power switch 18. The lighting unit 12 notifies the operation state of the HMD 100 (for example, ON / OFF of the power supply) by its light emission mode. For example, an LED (Light Emitting Diode) can be used as the lighting unit 12. The touch pad 14 detects a contact operation on the operation surface of the touch pad 14 and outputs a signal corresponding to the detected content. As the touch pad 14, various touch pads such as an electrostatic type, a pressure detection type, and an optical type can be adopted. The cross key 16 detects a pressing operation on a key corresponding to the up / down / left / right direction, and outputs a signal corresponding to the detected content. The power switch 18 switches the power state of the HMD 100 by detecting a slide operation of the switch.

  FIG. 2 is a block diagram functionally showing the configuration of the HMD 100. The control unit 10 includes an input information acquisition unit 110, a storage unit 120, a power supply 130, a wireless communication unit 132, a GPS module 134, a CPU 140, an interface 180, and transmission units (Tx) 51 and 52. The parts are connected to each other by a bus (not shown).

  The input information acquisition unit 110 acquires a signal corresponding to an operation input to the touch pad 14, the cross key 16, the power switch 18, and the like, for example. The storage unit 120 includes a ROM, a RAM, a DRAM, a hard disk, and the like.

  The power supply 130 supplies power to each part of the HMD 100. As the power source 130, for example, a secondary battery such as a lithium polymer battery or a lithium ion battery can be used. Further, instead of the secondary battery, a primary battery or a fuel cell may be used, or the wireless battery may be operated by receiving wireless power feeding. Furthermore, you may make it receive electric power supply from a solar cell and a capacitor. The wireless communication unit 132 performs wireless communication with other devices in accordance with a predetermined wireless communication standard such as a wireless LAN, Bluetooth (registered trademark), or iBeacon (registered trademark). The GPS module 134 detects its current position by receiving a signal from a GPS satellite.

  The CPU 140 reads out and executes a computer program stored in the storage unit 120, thereby executing an operating system (OS) 150, an image processing unit 160, a display control unit 162, a movement detection unit 164, a processing control unit 166, and an audio processing. Unit 170 and iBeacon processing unit 172.

  The image processing unit 160 generates a signal based on content (video) input via the interface 180 or the wireless communication unit 132. Then, the image processing unit 160 controls the image display unit 20 by supplying the generated signal to the image display unit 20 via the connection unit 40. The signal supplied to the image display unit 20 differs between the analog format and the digital format. In the case of the analog format, the image processing unit 160 generates and transmits a clock signal PCLK, a vertical synchronization signal VSync, a horizontal synchronization signal HSync, and image data Data. Specifically, the image processing unit 160 acquires an image signal included in the content. For example, in the case of a moving image, the acquired image signal is an analog signal generally composed of 30 frame images per second. The image processing unit 160 separates a synchronization signal such as a vertical synchronization signal VSync and a horizontal synchronization signal HSync from the acquired image signal, and generates a clock signal PCLK by a PLL circuit or the like according to the period. The image processing unit 160 converts the analog image signal from which the synchronization signal is separated into a digital image signal using an A / D conversion circuit or the like. The image processing unit 160 stores the converted digital image signal as image data Data of RGB data in the DRAM in the storage unit 120 for each frame.

  On the other hand, in the case of the digital format, the image processing unit 160 generates and transmits the clock signal PCLK and the image data Data. Specifically, when the content is in digital format, the clock signal PCLK is output in synchronization with the image signal, so that the generation of the vertical synchronization signal VSync and the horizontal synchronization signal HSync and the A / D conversion of the analog image signal are performed. It becomes unnecessary. The image processing unit 160 performs image processing such as resolution conversion processing, various tone correction processing such as adjustment of luminance and saturation, and keystone correction processing on the image data Data stored in the storage unit 120. May be executed.

  The image processing unit 160 transmits the generated clock signal PCLK, vertical synchronization signal VSync, horizontal synchronization signal HSync, and image data Data stored in the DRAM in the storage unit 120 via the transmission units 51 and 52, respectively. To do. The image data Data transmitted via the transmission unit 51 is also referred to as “right eye image data Data1”, and the image data Data transmitted via the transmission unit 52 is also referred to as “left eye image data Data2”. The transmission units 51 and 52 function as a transceiver for serial transmission between the control unit 10 and the image display unit 20.

  The display control unit 162 generates a control signal for controlling the right display drive unit 22 and the left display drive unit 24. Specifically, the display control unit 162 uses the control signal to turn on / off the right LCD 241 by the right LCD control unit 211, turn on / off the right backlight 221 by the right backlight control unit 201, and control the left LCD. Each of the right display drive unit 22 and the left display drive unit 24 is controlled by individually controlling ON / OFF driving of the left LCD 242 by the unit 212 and ON / OFF driving of the left backlight 222 by the left backlight control unit 202. Controls the generation and emission of image light. The display control unit 162 transmits control signals for the right LCD control unit 211 and the left LCD control unit 212 via the transmission units 51 and 52, respectively. Similarly, the display control unit 162 transmits control signals to the right backlight control unit 201 and the left backlight control unit 202, respectively.

  The movement detection unit 164 detects that the HMD 100 mounted on the user's head has moved to a plurality of predetermined specific locations. The process control unit 166 changes at least a part of predetermined functions among various functions mounted on the HMD 100. The predetermined function may be one function or a plurality of functions, but in the present embodiment, it is a plurality of functions. Details of the movement detection unit 164 and the processing control unit 166 will be described later.

  The audio processing unit 170 acquires an audio signal included in the content, amplifies the acquired audio signal, and transmits the acquired audio signal to a speaker (not shown) in the right earphone 32 and a speaker (not shown) in the left earphone 34 connected to the connecting member 46. To supply. For example, when a Dolby (registered trademark) system is adopted, processing is performed on an audio signal, and different sounds with different frequencies or the like are output from the right earphone 32 and the left earphone 34, respectively.

  The iBeacon processing unit 172 obtains a distance between the BLE terminal and the HMD 100 by receiving a signal from a BLE (Bluetooth Low Energy) terminal provided outside the HMD 100 by using the iBeacon (registered trademark) technology. .

  The interface 180 is an interface for connecting various external devices OA that are content supply sources to the control unit 10. Examples of the external device OA include a personal computer PC, a mobile phone terminal, and a game terminal. As the interface 180, for example, a USB interface, a micro USB interface, a memory card interface, or the like can be used.

  The image display unit 20 includes a right display drive unit 22, a left display drive unit 24, a right light guide plate 261 as a right optical image display unit 26, a left light guide plate 262 as a left optical image display unit 28, and outside scene shooting. Camera 61 (see also FIG. 1) and a nine-axis sensor 66.

  The outside scene shooting camera 61 is arranged at a position corresponding to the eyebrow of the user when the user wears the image display unit 20. For this reason, the outside scene shooting camera 61 captures an outside scene, which is an external scenery in the direction in which the user is facing, with the user wearing the image display unit 20 on the head. The outside scene shooting camera 61 is a monocular camera, but may be a stereo camera.

  The 9-axis sensor 66 is a motion sensor that detects acceleration (3 axes), angular velocity (3 axes), and geomagnetism (3 axes). Since the 9-axis sensor 66 is provided in the image display unit 20, when the image display unit 20 is mounted on the user's head, the movement of the user's head is detected. The orientation of the image display unit 20 is specified from the detected movement of the user's head.

  The right display driving unit 22 includes a receiving unit (Rx) 53, a right backlight (BL) control unit 201 and a right backlight (BL) 221 that function as a light source, a right LCD control unit 211 that functions as a display element, and a right An LCD 241 and a right projection optical system 251 are included. The right backlight control unit 201, the right LCD control unit 211, the right backlight 221 and the right LCD 241 are also collectively referred to as “image light generation unit”.

  The receiving unit 53 functions as a receiver for serial transmission between the control unit 10 and the image display unit 20. The right backlight control unit 201 drives the right backlight 221 based on the input control signal. The right backlight 221 is a light emitter such as an LED or electroluminescence (EL). The right LCD control unit 211 drives the right LCD 241 based on the clock signal PCLK, the vertical synchronization signal VSync, the horizontal synchronization signal HSync, and the right eye image data Data1 input via the reception unit 53. The right LCD 241 is a transmissive liquid crystal panel in which a plurality of pixels are arranged in a matrix.

  The right projection optical system 251 is configured by a collimator lens that converts the image light emitted from the right LCD 241 to light beams in a parallel state. The right light guide plate 261 as the right optical image display unit 26 guides the image light output from the right projection optical system 251 to the right eye RE of the user while reflecting the image light along a predetermined optical path. The optical image display unit can use any method as long as a virtual image is formed in front of the user's eyes using image light. For example, a diffraction grating or a transflective film may be used. good. Note that the emission of image light by the HMD 100 is also referred to as “displaying an image” in this specification.

  The left display drive unit 24 has the same configuration as the right display drive unit 22. That is, the left display driving unit 24 includes a receiving unit (Rx) 54, a left backlight (BL) control unit 202 and a left backlight (BL) 222 that function as a light source, and a left LCD control unit 212 that functions as a display element. And a left LCD 242 and a left projection optical system 252.

  FIG. 3 is an explanatory diagram illustrating a state in which image light is emitted by the image light generation unit. The right LCD 241 changes the transmittance of the light transmitted through the right LCD 241 by driving the liquid crystal at each pixel position arranged in a matrix, thereby changing the illumination light IL emitted from the right backlight 221 into an image. Is modulated into an effective image light PL representing The same applies to the left side. As shown in FIG. 3, the backlight system is adopted in the present embodiment, but the image light may be emitted using a front light system or a reflection system.

A-2. Head-mounted display platform:
FIG. 4 is an explanatory diagram of the platform of the HMD 100. The platform is a set of basic hardware resources, OS, middleware, and the like necessary for operating the application installed in the HMD 100. The platform 500 of this embodiment includes an application layer 510, a framework layer 520, a library layer 530, a kernel layer 540, and a hardware layer 550. Each layer 510 to 550 conceptually divides hardware resources, OS, middleware, and the like included in the platform 500. The framework layer 520, the library layer 530, and the kernel layer 540 implement the functions of the OS 150 (FIG. 2). In FIG. 4, illustration of components that are not necessary for the description is omitted.

  The application layer 510 is a set of application software for executing predetermined processing on the OS 150. Each application software included in the application layer 510 is hereinafter also referred to as “application” or “application”. The application layer 510 includes both apps installed in advance in the HMD 100 and apps installed by the user of the HMD 100.

  In the example of FIG. 4, the application layer 510 includes a museum guide application 511, a game application 512, a camera application 513, and the like. The museum guide application 511 provides a guide function suitable for museum tours. The game application 512 provides a game function. The camera application 513 provides a shooting function.

  The framework layer 520 is a set of programs that implement a basic program structure and function set common to the application software of the application layer 510. In the present embodiment, an image processing unit frame 521, a display control unit frame 522, an audio processing unit frame 523, an iBeacon processing unit frame 524, and the like are included. The image processing unit frame 521 implements the function of the image processing unit 160 (FIG. 2). The display control unit frame 522 implements the function of the display control unit 162 (FIG. 2). The voice processing unit frame 523 realizes the function of the voice processing unit 170 (FIG. 2). The iBeacon processing unit frame 524 realizes the function of the iBeacon processing unit 172 (FIG. 2).

  The library layer 530 is a set of library software in which a program for realizing a specific function is componentized so that it can be used by another program (for example, an application of the application layer 510). Each library software included in the library layer 530 is hereinafter also referred to as “library”. A library cannot be executed alone, but is executed by being called from another program.

  In the example of FIG. 4, the library layer 530 includes a display library 533, an audio library 534, a sensor library 535, a camera library 536, an HTML (Hyper Text Markup Language) library 537, and the like. include. The display library 533 drives the right LCD 241 and the left LCD 242 (FIG. 2). The audio library 534 drives an audio IC (Integrated Circuit) built in the right earphone 32 and the left earphone 34 (FIG. 2). The sensor library 535 drives the 9-axis sensor 66 (FIG. 2), acquires the detection value by the 9-axis sensor 66, and processes the detection value into information for providing the application. The camera library 536 drives the outside scene shooting camera 61 (FIG. 2), acquires a detection value by the outside scene shooting camera 61, and generates an outside scene image from the detection value. The HTML library 537 interprets data written in a web page description language and calculates the arrangement of characters and images for screen display.

  The kernel layer 540 is a set of programs that implement the basic functions of the OS 150. The kernel layer 540 manages exchanges between software (library layer 530) and hardware (hardware layer 550), and functions as a bridge between the two.

  In the example of FIG. 4, the kernel layer 540 includes an LCD driver 541 for the right LCD 241 and the left LCD 242, an audio IC driver 542 for the audio IC, a sensor driver 543 for the 9-axis sensor 66, and an outside scene shooting camera 61. And an image sensor driver 544 for an image sensor incorporated in the image sensor.

  The hardware layer 550 is an actual hardware resource incorporated in the HMD 100. In the present embodiment, “hardware resource” means a device connected to the HMD 100 and incorporated in the HMD 100. That is, the hardware resources include devices internally connected to the motherboard of the HMD 100 (for example, a sensor device of the 9-axis sensor 66, an image sensor device of the camera 61, a sensor device of the touch pad 14), and an interface 180. And devices externally connected to the HMD 100 via the HMD 100 (for example, an external motion sensor device, an external USB device, etc.).

  In the example of FIG. 4, the hardware layer 550 includes an LCD device 551 as the right LCD 241 and the left LCD 242, an audio IC device 552, a sensor device 553 of the 9-axis sensor 66, an image sensor device 554 of the camera 61, It is included.

  The library, the driver, and the device surrounded by a broken line in FIG. 4 are in a corresponding relationship and operate in cooperation. For example, the sensor library 535, the sensor driver 543, and the sensor device 553 operate in cooperation to realize the function of the 9-axis sensor 66. In other words, the sensor library 535 of the library layer 530 and the sensor driver 543 of the kernel layer 540 are programs (software) for the application to use the sensor device 553 as a hardware resource (hardware layer 550). It can be said that there is. In addition, in order to be able to use the sensor device 553 as one hardware resource, a configuration may be adopted in which a plurality of libraries are allocated to the sensor device 553.

  On the other hand, in FIG. 4, for example, the HTML library 537 of the library layer 530 is not associated with hardware resources and does not depend on hardware resources. Thus, a program (software) that is incorporated in the HMD 100 and does not depend on hardware resources is also referred to as “software resources” in the present embodiment. As software resources, various programs included in the framework layer 520, the library layer 530, and the kernel layer 540 are assumed.

A-3. Usage of head mounted display:
FIG. 5 is an explanatory diagram showing a form of use of the HMD 100 in a museum. First, a visitor to the museum goes to the reception and pays the entrance fee (scene 1). At this time, the receptionist may perform personal authentication of the visitor. For example, the personal authentication is performed by receiving an identification card such as a license, a passport, or a health insurance card.

  In the scene 2 following the scene 1, the receptionist lends the HMD 100 described above to the visitors who have received the reception. At the time of lending, the acceptor inserts an IC card, USB memory, SIM or the like as a hard key into the HMD 100. Note that the OS 150 product key or the like may be input as a soft key instead of a hard key. When personal authentication is performed, the above-described identification number or the like may be input as a soft key. Input as a soft key is performed using the input information acquisition unit 110 such as the touch pad 14 or the cross key 16. The hard key or soft key input in this way is stored in the storage unit 120 of the HMD 100 as an HMD authentication key. The receptionist lends the HMD 100 that has finished inputting hard keys or soft keys to a visitor. The HMD 100 is lent out in a state where the museum guide application 511 (FIG. 4) is activated.

  In the scene 3 following the scene 2, the visitor uses the rented HMD 100 on the head. A visitor who is using the HMD 100 is hereinafter also referred to as a “user”. A gate (entrance) 610 is provided in front of the exhibition area 600 of the museum. In the scene 3, the user HU proceeds to the front of the gate 610. The gate 610 has a gate identification name GN as a marker for identifying the gate 610. In the scene 3, the user HU opens the gate 610 using the function of the HMD 100 including photographing of the gate identification name GN. Details of this function will be described later.

  The user HU passes through the opened gate 610 and enters the exhibition area 600. A plurality of BLE terminals 670 for iBeacon (registered trademark) are installed in the exhibition area 600. In the scene 4 moving in the exhibition area 600, the user HU receives a route (guide route) using a function of the HMD 100 including communication with the BLE terminal 670. Details of this function will also be described later.

  The user HU proceeds in front of the exhibit 680 and appreciates the exhibit 680 (scene 5). Around the exhibit 680, a marker for identifying the exhibit 680 is attached in the form of a barcode BC. At the time of viewing, the user is presented with information about the exhibit 680 in front of the user using the functions of the HMD 100 including shooting of the barcode BC. Details of this function will also be described later. The exhibit may be an exhibit.

  The user HU who has finished viewing the exhibits moves from the gate (exit) (not shown) to the outside of the display area 600 (scene 6 (not shown)) and returns the HMD 100 at the reception (scene 7 (not shown)).

A-4. Gate and central monitoring system configuration:
FIG. 6 is an explanatory diagram showing the gate and the central monitoring device. The gate 610 is of a hinged door type and includes a pair of slide shutters 612 and 614, a left storage portion 616, and a right storage portion 618. The slide shutters 612 and 614 are opened and closed by a shutter drive unit (not shown) provided in the left storage unit 616 and a shutter drive unit (not shown) provided in the right storage unit 618. In the opened state, the left slide shutter 612 is stored in the left storage unit 616, and the right slide shutter 614 is stored in the right storage unit 618. A plurality of gates 610 are provided in the museum, and the above-described gate identification names (for example, “Gate A”, “Gate B”, etc.) for identifying each gate 610 are displayed in the right storage portion 618 of each gate 610. GN is written.

  The central monitoring device 650 is connected to each gate 610 by a cable 620. The central monitoring device 650 controls opening and closing of each gate 610 and manages visitors. The central monitoring device 650 includes a CPU 652, a storage unit 654 including a ROM and a RAM, an input / output unit 656, and a wireless communication unit 658.

  The storage unit 634 includes a program storage unit 654a, an HMD authentication key storage unit 654b, a gate information storage unit 654c, a route information storage unit 654d, and an exhibit information storage unit 654e. Various computer programs are stored in advance in the program storage unit 654a. The various computer programs include the above-described museum guide application 511. The HMD authentication key storage unit 654b stores an HMD authentication key imported from the HMD 100. In the gate information storage unit 654c, information related to the gate 610 is stored in advance corresponding to the gate identification name GN. The route information storage unit 654d stores information indicating the route of the exhibition area 600 in advance. In the exhibit information storage unit 654e, detailed information about the exhibit 680 (for example, information on technique, background of the period, art history, etc.) is stored in advance corresponding to the identification code of the exhibit 680 (FIG. 5). .

  The input / output unit 656 sends an open / close command from the CPU 652 to the shutter drive unit of each gate 610 by wire. The wireless communication unit 658 performs wireless communication with the HMD 100 worn by the user HU.

  The CPU 652 executes the computer program stored in the program storage unit 654a, thereby performing various functions stored in the storage unit 654, and performing functions necessary for the scene 3 to the scene 5 in FIG. Make it happen.

A-5. Processing in scene 3:
FIG. 7 is a flowchart showing details of the entrance processing routine. The entrance processing routine is one of a plurality of processing routines included in the museum guide application 511 (FIG. 4), and is repeatedly executed by the CPU 140 of the HMD 100 at predetermined time intervals. Note that the entrance processing routine is executed only during a period from when the HMD 100 is lent out in the scene 2 until it passes through a gate described later. When the process is started, the CPU 140 activates the outside scene photographing camera 61 to photograph the outside scene (step S110). In the scene 3, the user HU proceeds to the front of the gate 610. At this time, the outside scene is automatically photographed by the outside scene photographing camera 61.

  Subsequently, the CPU 140 determines whether or not the gate identification name GN (FIG. 6) is included in the captured image captured in step S110 (step S120). This determination determines that the gate identification name GN is included when the character string of the gate identification name GN is extracted from the captured image by character pattern recognition.

  If it is determined in step S120 that the gate identification name GN is not included, the CPU 140 returns the process to step S110 and continues shooting with the outside scene shooting camera 61 so that the gate identification name GN is shot. Wait for it. On the other hand, if it is determined in step S120 that the gate identification name GN is included, the CPU 140 includes an opening command to open the gate and the gate identification name GN acquired from the photographed image and the scene 2 (FIG. 5). Then, the HMD authentication key stored in the storage unit 120 of the HMD 100 is transmitted to the central monitoring apparatus 650 via the wireless communication unit 132 (step S130). Thereafter, the CPU 140 stops the shooting of the outside scene shooting camera 61 (step S140). Note that the process from step S110 executed by the CPU 140 to YES in step S120 corresponds to the movement detection unit 164 (FIG. 2). That is, the movement detection unit 164 detects that the HMD 100 has moved before the gate as a specific place. In other words, assuming that the specific place is the display area 600, the movement detection unit 164 can also be regarded as the HMD 100 has moved to the display area 600 as the specific place.

  The CPU 652 of the central monitoring device 650 determines whether or not an open command has been received from the HMD 100 (step S210). When it is determined that the CPU 652 has received the HMD authentication key sent from the HMD 100 together with the open command, The data is stored in the HMD authentication key storage unit 654b of the storage unit 654 (step S220). Thereby, the central monitoring apparatus 650 can register a visitor in the form of an HMD authentication key. Thereafter, the CPU 652 performs a process of opening the gate 610 corresponding to the gate identification name GN sent together with the opening command from the HMD 100 (step S230). In the gate information storage unit 654c of the central monitoring device 650, as the gate information, the destination of the open signal that opens each gate 610 is stored in advance corresponding to the gate identification name GN of each gate 610. The CPU 652 opens the gate 610 by reading the destination corresponding to the gate identification name GN sent from the HMD 100 from the gate information storage unit 654c and sending an open signal to the destination.

  Next, the CPU 652 transmits an admission permission signal to the HMD 100 via the wireless communication unit 658 (step S240).

  Thereafter, the CPU 140 of the HMD 100 determines whether or not an admission permission signal has been received from the central monitoring device 650 (step S160), and if it is determined that it has been received, a message image stating “Please enter” is displayed. Is displayed (step S170). That is, the CPU 140 displays the message image on the image display unit 20.

  Subsequently, the CPU 140 locks the audio processing unit frame 523 and the camera application 513 included in the application layer 510 (step S180). Since the audio processing unit frame 523 bears all of the audio output of the HMD 100, the audio processing unit frame 523 is locked, so that all of the audio output from the HMD 100 is erased (muted). On the other hand, the shooting function using the outside scene shooting camera 61 is included in the marker recognition process by the museum guide application 511 included in the application layer 510 in addition to the camera application 516. According to the lock of the camera application 516 in step S190, shooting by the camera application 516 is prohibited, and shooting according to the marker recognition process by the museum guide application 511 is not prohibited (that is, permitted).

  After the execution of step S190, the process returns to “RETURN”, and the entrance processing routine is temporarily terminated. In the present embodiment, the shooting function provided by the camera application 516 uses the outside scene shooting camera 61, but instead, when a camera other than the outside scene shooting camera 61 is mounted. The camera application 513 may take a picture with another camera. In this case, in step S180, the camera application 513 may be prohibited so that shooting by the other camera is also prohibited.

Note that the processing in steps S170 and S180 executed by the CPU 140 corresponds to the processing control unit 166 (FIG. 2). That is, the process control unit 166
i) As an extension of the information presentation function, a message image stating “Please enter” is displayed.
ii) Locking the voice processing unit frame 523 as suppressing the voice function,
iii) As the suppression of the photographing function, photographing according to the marker recognition process is permitted, and the photographing function by the camera application 516 is prohibited.

  The user HU who has recognized the message “Please enter” in step S170 passes through the opened gate 610 and enters the exhibition area 600. Note that the locking of the audio processing unit frame 523 and the camera application 513 in step S180 is continued until the user HU, that is, the HMD 100 worn by the user HU moves outside the display area 600, and the scene 6 moves outside. The lock is released for the first time. When it is detected that the HMD 100 worn by the user HU has moved to the outside of the display area 600 as a specific place, the sound processing unit 170 is operated to output sound, for example, “unlock”. The user may be notified that the locked function has been restored by displaying a message image labeled “Done” on the drawing display unit 20. According to this configuration, it is possible to improve user convenience.

  According to the entrance processing routine configured as described above, when the user HU moves in front of the gate 610, that is, when the HMD 100 moves in front of the gate 610, the above (i) to (iii) are performed. Done. For this reason, according to the HMD 100 of the present embodiment, the information presentation function “Please enter” is executed only by moving in front of the gate 610, and the voice function is suppressed, and the camera application 516 is operated. The shooting function used is prohibited.

  Therefore, according to the HMD 100, the convenience of the user HU can be enhanced by presenting information. Further, by suppressing the voice function from the HMD 100 in the exhibition area 600, manners can be protected. Further, by prohibiting shooting by the HMD 100 in the display area 600, it is possible to prevent the exhibit 680 and the like from being voyeurized. That is, according to the HMD 100, there is an effect that a function suitable for the place in the exhibition area 600 can be easily realized without a person switching the setting of functions one by one.

  According to the process according to the route guidance process routine, it is detected that the HMD 100 worn by the user has moved to the front of the gate by photographing the gate identification name GN attached to the gate 610. Instead of this, it is also possible to detect that the HMD has moved before a specific gate from the shape of the gate of the photographed image by photographing the gate itself, assuming that the shape of each gate is different in advance. . The gate identification name GN can be replaced with any symbol, figure, mark, or the like that can be a marker for identifying the gate 610.

A-6. Processing in scene 4:
FIG. 8 is a flowchart showing details of the route guidance processing routine. The route guidance processing routine is one of a plurality of processing routines included in the museum guidance application 511 (FIG. 4), and is repeatedly executed by the CPU 140 of the HMD 100 at predetermined time intervals. In this route guidance processing routine, an iBeacon processing unit 172 is used. A BLE terminal 670 (FIG. 5) is arranged at each corner of the regular route in the exhibition area 600 or in the middle part when a straight forward part continues (hereinafter, referred to as “straight halfway part”). An iBeacon signal is output from 670. The iBeacon signal holds at least a BLE terminal identification number for identifying the BLE terminal 670 and a distance to the BLE terminal 670.

  When the processing is started, the CPU 140 of the HMD 100 first determines whether an iBeacon signal is detected from the BLE terminal 670 (step S310). The CPU 140 repeatedly executes the process of step S310 until the iBeacon signal is sensed, and if it is determined that the sensed signal is sensed in step S310, is the distance held in the sensed signal less than a predetermined value (for example, 2 m)? It is determined whether or not (step S320). If it is determined that the distance is not less than the predetermined value, the process returns to step S310.

  On the other hand, if it is determined in step S320 that the distance is less than the predetermined value, the route information corresponding to the BLE terminal identification number held in the sensed signal is acquired. In the route information storage unit 654d of the central monitoring device 650, as route information, for example, “turn left”, “turn right”, or “go straight”, corresponding to the BLE terminal identification number. In step S320, the CPU 140 performs the above acquisition by reading out the route information corresponding to the BLE terminal identification number held in the sensed signal from the route information storage unit 654d.

  Thereafter, the CPU 140 displays a route guidance message corresponding to the route information acquired in step S330 (step S340). That is, the CPU 140 causes the image display unit 20 to display the route guidance message.

  FIG. 9 is an explanatory diagram showing an example of a route guidance message visually recognized by the user as a result of step S340. FIG. 9 illustrates the visual field VR of the user. The visual field VR includes an outside scene SC in the exhibition area 600 as shown in the figure. Further, a route guidance message MS1 such as “Please turn left” is displayed superimposed on the outside scene SC.

  After execution of step S340 in FIG. 8, the process returns to “RETURN” and this route guidance processing routine is temporarily terminated. The processing of steps S310 and S320 executed by the CPU 140 corresponds to the movement detection unit 164 (FIG. 2). In other words, the movement detection unit 164 detects that the HMD 100 has moved near the corner of a normal route as a specific place or in the middle of a straight line. The processing in steps S330 and S340 executed by the CPU 140 corresponds to the processing control unit 166 (FIG. 2). That is, the process control unit 166 displays the route guidance message MS1 as an extension of the information presentation function.

  According to the route guidance processing routine configured as described above, the user wearing the HMD 100 can recognize the route guidance message MS in the visual field VR when moving in the exhibition area 600. For this reason, according to HMD100 of this embodiment, it is excellent in a user's convenience.

  According to the process according to the route guidance process routine, the movement of the HMD 100 worn by the user is detected by the iBeacon technique, but instead, when installing a Wi-Fi access point. The movement of the HMD may be detected by estimating the current position using the position information registered in. Alternatively, the movement of the HMD may be detected by visible light communication using an LED. In short, any wireless communication technology may be used as long as the movement of the HMD is detected based on a signal from an external wireless communication terminal. Furthermore, it is good also as a structure which detects the movement of HMD by the technique which calculates | requires the indoor present position using a geomagnetism, and the technique of indoor GPS. In addition, if the exhibition area 600 is outdoors, it may be configured to detect the movement of the HMD by specifying the current position using the GPS module 134. Further, instead of the configuration for detecting the movement of the HMD from one of these techniques, a plurality of these techniques may be combined to detect the movement of the HMD. You may make it use these techniques properly according to a detection place etc.

A-7. Processing in scene 5:
FIG. 10 is a flowchart showing details of the exhibit explanation routine. The exhibit description routine is one of a plurality of processing routines included in the museum guide application 511 (FIG. 4), and is repeatedly executed by the CPU 140 of the HMD 100 at predetermined time intervals. When the process is started, the CPU 140 first detects whether the user is walking by detecting the movement of the user's head with the 9-axis sensor 66 (step S410). If it is determined that the user is walking, the user exits “Return” and temporarily stops the exhibit explanation routine, assuming that the user is not in a state of appreciating the exhibit. On the other hand, if it is determined in step 410 that the user is not walking, the outside scene photographing camera 61 is activated to photograph the outside scene (step S420). As a modification, it may be configured to immediately take a picture of the outside scene without determining whether or not it is walking in step S410.

  Subsequently, the CPU 140 determines whether or not the bar code BC for exhibit identification is included in the captured image captured in step S420 (step S430). If it is determined that the image is not included, the CPU 140 returns the process to step S420, continues shooting with the outside scene shooting camera 61, and waits for the barcode BC to be shot. On the other hand, if it is determined in step S430 that the barcode BC for exhibit identification is included, the barcode BC is converted into the identification code of the exhibit 680 (step S440), and the outside scene photographing camera 61 is photographed. Is stopped (step S450). Thereafter, the CPU 140 reads exhibit information corresponding to the exhibit identification code obtained in step S440 from the exhibit information storage unit 654e (step S460), and displays the exhibit information (step S470). That is, the CPU 140 displays the exhibit information on the image display unit 20. The exhibit information is displayed at a position based on the position of the barcode BC included in the outside scene. In the present embodiment, the barcode BC is attached to the upper left of the exhibit 680 as shown in FIG. 5, and the exhibit information is displayed at the left position of the barcode BC.

  FIG. 11 is an explanatory diagram illustrating an example of exhibit information visually recognized by the user as a result of step S470. The visual field VR of the user is illustrated. The visual field VR includes an outside scene SC around the exhibit 680. In addition, the exhibit information EX is displayed superimposed on the outside scene SC. Exhibit information EX is detailed information about exhibits such as techniques, historical background, and information on art history, and is displayed on the left side of the barcode BC in this embodiment. The display position need not be limited to the position on the left side of the bar code BC, and may be a position in another direction with respect to the bar code BC. Alternatively, the position of the exhibit can be grasped by pattern recognition. Thus, it may be displayed at a predetermined relative position with respect to the exhibit.

  After execution of step S470 in FIG. 10, the process returns to “RETURN”, and this route guidance processing routine is temporarily terminated. Note that the processing of steps S420 and S430 executed by the CPU 140 corresponds to the movement detection unit 164 (FIG. 2). That is, the movement detection unit 164 detects that the HMD 100 has moved before the exhibit 680 as a specific place. The processes in steps S460 and S470 executed by the CPU 140 correspond to the process control unit 166 (FIG. 2). That is, the process control unit 166 displays the exhibit information EX as an extension of the information presentation function.

  According to the exhibit explanation routine configured as described above, the user wearing the HMD 100 can stand in front of the exhibit 680 and display the technique of the exhibit 680, information on the background of the period, art history, and the like. The product information EX can be recognized in the visual field VR. For this reason, according to HMD100 of this embodiment, it is excellent in a user's convenience.

  In addition, according to the process according to this exhibit description routine, it is detected that the HMD 100 worn by the user has moved to the front of the exhibit by photographing the barcode BC attached to the periphery of the exhibit 680. However, the bar code BC may be replaced with a simple and clear black-and-white figure or another type of code such as a QR code (registered trademark). Further, the position where the barcode BC is arranged may be any position as long as it is around or around the exhibit 680 instead of the configuration attached to the upper left of the exhibit 680. Furthermore, a code may be arranged inside the exhibit 680 in the form of a digital watermark. Furthermore, it is good also as a structure which detects having moved before the exhibit based on the picked-up image of the exhibit 680 itself.

B. Variations:
The present invention is not limited to the first embodiment and the modifications thereof, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible. is there.

・ Modification 1:
In the first embodiment and the modified example, the place where the HMD is used is an art museum, but instead, it may be used in a museum. In the museum, the forms of the scenes 3 to 5 in the above embodiment can be applied as they are. In addition to museums, museums, theaters, opera houses, movie theaters, concert venues, workshops, etc. may be used. 514 and the camera application 516 may be locked.

Modification 2
In addition to the first embodiment and the first modification, the place where the HMD is used is a sightseeing spot, a company, a department store, a factory, a school, a hospital, an expressway, a stadium, a train, a car, an airplane It can be in various places such as inside. For example, in the case of a sightseeing spot, when the movement of the HMD to the periphery of a building (for example, a shrine, a ruins, a castle, a Buddhist image, a tower, etc.) is detected by, for example, GPS, Such detailed information, that is, history information on buildings, period picture scrolls, and the like may be displayed. For example, the camera application 516 may be locked when a movement to a public space such as a train, a car, or an airplane is detected to prevent voyeurism. For example, when movement to a public space is detected, the resolution of the camera is limited. For example, a camera with 12 million pixels is usually reduced to a function of 300,000 pixels that cannot recognize a human face. Good. For example, execution of a business application program file may be permitted when movement to a company is detected. For example, in the case of a department store, iBeacon technology is used in the same manner as in the case of the scene 4, and when the movement of the HMD before a predetermined sales floor is detected, recommended information such as bargain items is displayed. May be. For example, in an aircraft, the use of a mobile phone or wireless LAN that is the basis of jamming radio waves may be prohibited, and Bluetooth and iBeacon that have less influence on electronic devices of the aircraft may be used. In addition, a specific place where the movement detection unit detects that it has moved can be a large unit such as a country. For example, when it is detected by GPS or the like that the HMD has moved to a specific country, the function determined according to the country may be changed. For example, it is possible to change the language used, change the usage unit such as length and mass, change the output limit of the wireless device, and the like.

・ Modification 3:
In the above embodiment, as the processing control unit, the audio processing unit frame 523 is locked as the audio function is suppressed. Instead, the audio output level is suppressed (that is, the volume is reduced). Also good. Moreover, it is good also as a structure which suppresses predetermined functions, such as a structure which reduces the information content of the guidance information displayed as a structure which suppresses an information presentation function. In addition, for example, a configuration that promotes the function of a predetermined process, such as a configuration that promotes the degree of audio output (that is, a volume increase), or a configuration that increases the amount of guidance information to be displayed may be adopted.

-Modification 4:
In the above-described embodiment, switching of driving on / off was performed as a change of the shooting function by the camera. Instead, switching of resolution, switching of infrared shooting on / off, and switching of video function on / off were performed. Switching, color / monochrome switching, continuous shooting ON / OFF switching, split shooting ON / OFF switching, saving format (raw data) switching, shooting image saving enable / disable, password for shooting images It is good also as switching of provision, switching of various filter photography, etc.

Modification 5:
In the above-described embodiment, various messages are displayed as a change of the information presentation function. However, the display capability may be switched. For example, switching between binocular display using the right display driving unit 22 and left display driving unit 24 and monocular display using one of them, switching of resolution, switching between 3D display and 2D display, switching between color / monochrome, It is also possible to switch the transmittance, switch the display brightness, switch the number of moving image frames, switch the display language, and the like. Also, the advertisement display may be switched on / off.

Modification 6:
In the above-described embodiment, as a change in sound function, a change in sound range, a change in bit rate (equivalent to AM broadcast, high quality, etc.), a change in Dolby on / off, and the number and types of sound sources that can be used (synthesizer, speech synthesis) It can be applied to various kinds of switching such as change, stereo / monaural change, left / right change or both change. Further, the recording function may be changed by changing the on / off state of the microphone, changing the on / off state of the recording function itself, changing the presence / absence of saving the recorded sound, changing the presence / absence of giving a password, and the like.

Modification 7:
In addition, as a function change, various sensors may be switched on / off. Further, the change in function may be a change in communication function. The change of the communication function may be switching on / off of the communication function, switching of the communication speed, switching of the communication distance, switching of the range of the connection destination network, switching of whether automatic connection is possible, or the like. In addition, on / off switching of bluetooth and iBeacom, switching of Bluetooth multilink availability, switching of RF tag reading function on / off, switching of memory access range, switching of access to external memory, The access range of a library or driver may be switched.

-Modification 8:
In the above embodiment, as a method of changing the function, an element included in the application layer 510 (for example, the camera application 516) is changed, or an element included in the framework layer 520, for example, the audio processing unit frame 523) is changed. However, instead of these, the elements included in the library layer 530 are changed, the elements included in the kernel layer 540 are changed, or the elements included in the hardware layer 550 are changed. Also good.

・ Other variations:
In the said embodiment, it illustrated about the structure of the head mounted display. However, the configuration of the head-mounted display can be arbitrarily determined without departing from the gist of the present invention. For example, each component can be added, deleted, converted, and the like.

  The allocation of components to the control unit and the image display unit in the above embodiment is merely an example, and various aspects can be employed. For example, the following aspects may be adopted. (I) A mode in which processing functions such as a CPU and a memory are mounted on the control unit, and only a display function is mounted on the image display unit. (Ii) Processing functions such as a CPU and a memory are provided in both the control unit and the image display unit. (Iii) a mode in which the control unit and the image display unit are integrated (for example, a mode in which the control unit is included in the image display unit and functions as a glasses-type wearable computer), (iv) instead of the control unit A mode in which a smartphone or a portable game machine is used, and (v) a mode in which the connection unit (code) is eliminated by adopting a configuration in which the control unit and the image display unit can perform wireless communication and wireless power feeding.

  In the above embodiment, for convenience of explanation, the control unit includes a transmission unit, and the image display unit includes a reception unit. However, each of the transmission unit and the reception unit of the above-described embodiment has a function capable of bidirectional communication, and can function as a transmission / reception unit. For example, the control unit shown in FIG. 2 is connected to the image display unit via a wired signal transmission path. However, the control unit and the image display unit may be connected by a connection via a wireless signal transmission path such as a wireless LAN, infrared communication, or Bluetooth (registered trademark).

  For example, the configurations of the control unit and the image display unit described in the above embodiment can be arbitrarily changed. Specifically, for example, the touch pad may be omitted from the control unit and the operation may be performed using only the cross key. Further, the control unit may be provided with another operation interface such as an operation stick. Moreover, it is good also as what receives an input from a keyboard or a mouse | mouth as a structure which can connect devices, such as a keyboard and a mouse | mouth, to a control part. Further, for example, in addition to an operation input using a touch pad or a cross key, an operation input using a foot switch (a switch operated by a user's foot) may be acquired. For example, after providing a line-of-sight detection unit such as an infrared sensor in the image display unit, the user's line of sight may be detected, and an operation input by a command associated with the movement of the line of sight may be acquired. For example, a user's gesture may be detected using a camera, and an operation input by a command associated with the gesture may be acquired. When detecting a gesture, a user's fingertip, a ring attached to the user's hand, a medical device to be used by the user's hand, or the like can be used as a mark for motion detection. If it is possible to acquire an operation input using a foot switch or a line of sight, the input information acquisition unit can acquire an operation input from the user even in a task in which it is difficult for the user to release his / her hand.

  FIG. 12 is an explanatory diagram showing an external configuration of the HMD in the modification. In the example of FIG. 12A, the image display unit 20x includes a right optical image display unit 26x instead of the right optical image display unit 26, and a left optical image display unit 28x instead of the left optical image display unit 28. I have. The right optical image display unit 26x and the left optical image display unit 28x are formed smaller than the optical member of the above-described embodiment, and are respectively disposed obliquely above the right eye and the left eye of the user when the HMD is worn. . 12B, the image display unit 20y includes a right optical image display unit 26y instead of the right optical image display unit 26, and a left optical image display unit 28y instead of the left optical image display unit 28. I have. The right optical image display unit 26y and the left optical image display unit 28y are formed smaller than the optical member of the above-described embodiment, and are respectively disposed obliquely below the right eye and the left eye of the user when the HMD is mounted. . Thus, it is sufficient that the optical image display unit is disposed in the vicinity of the user's eyes. The size of the optical member forming the optical image display unit is also arbitrary, and the optical image display unit covers only a part of the user's eyes, in other words, the optical image display unit completely covers the user's eyes. It can also be realized as an HMD in an uncovered form.

  For example, the head mounted display is a binocular transmissive head mounted display, but may be a monocular head mounted display. Further, it may be configured as a non-transmissive head mounted display in which the transmission of the outside scene is blocked when the user wears the head mounted display.

  For example, the functional units such as the image processing unit, the display control unit, and the audio processing unit are described as being realized by the CPU developing and executing a computer program stored in the ROM or hard disk on the RAM. However, these functional units may be configured using an ASIC (Application Specific Integrated Circuit) designed to realize the function.

  For example, in the above-described embodiment, it is assumed that the image display unit is a head mounted display that is mounted like glasses, but the image display unit is a normal flat display device (liquid crystal display device, plasma display device, organic EL display device, etc. ). Also in this case, the connection between the control unit and the image display unit may be a connection via a wired signal transmission path or a connection via a wireless signal transmission path. If it does in this way, a control part can also be utilized as a remote control of a usual flat type display device.

  Further, as the image display unit, instead of the image display unit worn like glasses, an image display unit of another shape such as an image display unit worn like a hat may be adopted. Further, the earphone may be an ear-hook type or a headband type, or may be omitted. Further, for example, it may be configured as a head-up display (HUD, Head-Up Display) mounted on a vehicle such as an automobile or an airplane. Further, for example, it may be configured as a head-mounted display built in a body protective device such as a helmet.

  For example, in the above embodiment, the display driving unit is configured using a backlight, a backlight control unit, an LCD, an LCD control unit, and a projection optical system. However, the above aspect is merely an example. The display drive unit may include a component for realizing another method together with these components or instead of these components. For example, the display driving unit may include an organic EL (Organic Electro-Luminescence) display, an organic EL control unit, and a projection optical system. For example, the display driving unit can use a DMD (digital micromirror device) or the like instead of the LCD. For example, the display driving unit includes a color light source for generating each color light of RGB and a signal light modulation unit including a relay lens, a scanning optical system including a MEMS mirror, and a drive control circuit for driving them. May be configured. As described above, even if an organic EL, DMD, or MEMS mirror is used, the “emission area in the display driving unit” is still an area where image light is actually emitted from the display driving unit. By controlling the emission area in the (display drive unit) in the same manner as in the above embodiment, the same effect as in the above embodiment can be obtained. Further, for example, the display driving unit may be configured to include one or more lasers that emit a laser having an intensity corresponding to the pixel signal to the retina of the user. In this case, the “emission area in the display drive unit” represents an area in which laser light representing an image is actually emitted from the display drive unit. By controlling the emission region of the laser beam in the laser (display drive unit) in the same manner as in the above embodiment, the same effect as in the above embodiment can be obtained.

  The present invention is not limited to the above-described embodiments, examples, and modifications, and can be realized with various configurations without departing from the spirit thereof. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in each embodiment described in the summary section of the invention are to solve some or all of the above-described problems, or In order to achieve part or all of the above-described effects, replacement or combination can be performed as appropriate. Further, if the technical feature is not described as essential in the present specification, it can be deleted as appropriate.

10. Control unit (controller)
DESCRIPTION OF SYMBOLS 12 ... Lighting part 14 ... Touch pad 16 ... Cross key 18 ... Power switch 20 ... Image display part 21 ... Right holding part 22 ... Right display drive part 23 ... Left holding part 24 ... Left display drive part 26 ... Right optical image display part 28 ... Left optical image display unit 30 ... Earphone plug 32 ... Right earphone 34 ... Left earphone 40 ... Connection unit 42 ... Right cord 44 ... Left cord 46 ... Connecting member 48 ... Body code 51 ... Transmission unit 52 ... Transmission unit 53 ... Reception Unit 54: Receiver 61 ... Camera for photographing outside scene 110 ... Input information acquisition unit 100 ... Head-mounted display device (HMD)
DESCRIPTION OF SYMBOLS 120 ... Memory | storage part 130 ... Power supply 132 ... Wireless communication part 140 ... CPU
DESCRIPTION OF SYMBOLS 160 ... Image processing part 162 ... Display control part 164 ... Movement detection part 166 ... Processing control part 170 ... Audio | voice processing part 180 ... Interface 201 ... Right backlight control part 202 ... Left backlight control part 211 ... Right LCD control part 212 ... Left LCD controller 221 ... Right backlight 222 ... Left backlight 241 ... Right LCD
242 ... Left LCD
251 ... Right projection optical system 252 ... Left projection optical system 261 ... Right light guide plate 262 ... Left light guide plate 500 ... Platform 510 ... Application layer 511 ... Museum guide application 512 ... Image processing unit application 513 ... Display control unit application 514 ... Audio processing Department application 515 ... Game application 516 ... Camera application 520 ... Framework layer 530 ... Library layer 533 ... Display library 534 ... Audio library 535 ... Sensor library 536 ... Camera library 537 ... Library 540 ... Kernel layer 542 ... Audio IC driver 543 ... Sensor driver 544 ... Image sensor driver 550 ... Hardware layer 552 ... Audio IC device 553 ... Sensor device 554 ... Image sensor device 600 ... Exhibition Area 610 ... gate 612 ... slide shutter 614 ... slide shutter 616 ... left housing portion 618 ... right housing portion 620 ... cable 634 ... storage unit 650 ... central monitoring unit 652 ... CPU
654... Storage unit 654 a. Program storage unit 654 c. Gate information storage unit 654 d. Route information storage unit 654 e. Exhibit information storage unit 656. VSync ... Vertical synchronization signal HSync ... Horizontal synchronization signal OA ... External device PC ... Personal computer BC ... Bar code SC ... Outside view GN ... Gate identification name MS ... Route guidance message EX ... Exhibition information

Claims (11)

A transmissive head-mounted display device,
An image display unit that displays an image and allows a user wearing the head-mounted display device to visually recognize the image, and can transmit an outside scene;
A movement detector for detecting that the head-mounted display device has moved to a specific location;
A process control unit that changes at least a part of a predetermined function mounted on the head-mounted display device when movement to the specific place is detected;
A head-mounted display device comprising: The head-mounted display device according to claim 1,
An outside scene photographing unit for photographing the outside scene;
The movement detector is
A head-mounted display device that detects movement to the specific location based on a photographed image obtained by the outside scene photographing unit. The head-mounted display device according to claim 2,
The movement detector is
A head-mounted display device in which movement to the specific location is detected when a marker for recognizing the specific location is included in the captured image. The head-mounted display device according to claim 1,
The movement detector is
A head-mounted display device that detects movement to the specific location based on a signal from an external wireless communication terminal. The head-mounted display device according to any one of claims 1 to 4,
The head-mounted display device, wherein the predetermined function is an information presentation function for displaying predetermined information on the image display unit. The head-mounted display device according to claim 5,
The specific location is around an exhibit or building;
The processing control unit
A head-mounted display device that displays information about the exhibit or building on the image display unit when movement to the specific location is detected. The head-mounted display device according to any one of claims 1 to 4,
The predetermined function is a photographing function including at least a marker photographing function performed for recognizing a predetermined marker,
The processing control unit
A head-mounted device that permits the marker photographing function among the photographing functions and prohibits the photographing function excluding the marker photographing function among the photographing functions when movement to the specific place is detected. Type display device. The head-mounted display device according to any one of claims 1 to 7,
The processing control unit
When the movement to the specific place is detected, the predetermined function is stopped,
A head-mounted display device that releases the stop of the predetermined function when movement of the head-mounted display device from the specific location to another location is detected. The head-mounted display device according to any one of claims 1 to 8,
The processing control unit
A head-mounted display device that notifies the user of the escape when the head-mounted display device is detected to escape from the specific location. A method for controlling the head-mounted display device having an image display unit that displays an image and allows a user wearing the head-mounted display device to visually recognize the image and transmit the outside scene,
Detecting that the head-mounted display device has moved to a specific location;
A step of changing at least a part of a predetermined function mounted on the head-mounted display device when movement to the specific place is detected;
A method for controlling a head-mounted display device. A computer program for controlling the head-mounted display device having an image display unit that displays an image and allows a user wearing the head-mounted display device to visually recognize the image and transmit an outside scene. ,
On the computer,
A function of detecting that the head-mounted display device has moved to a specific location;
A function of changing at least a part of a predetermined function mounted on the head-mounted display device when movement to the specific place is detected;
A computer program that realizes

Images