JP2011180867A - Head-mounted display - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an HMD (Head-Mounted Display) that facilitates an input to an image presented by the HMD. <P>SOLUTION: The HMD 1 includes an image presentation unit 120 for presenting an image V1 visibly to eyes of a user USR, and a pen mouse 30 serving as a movement amount detection unit for detecting the amount of the movement of a hand of the user USR. Based on the amount of the movement of the pen mouse 30, a position of the movement amount detection means in a real space is determined. A predetermined range of the surface of a table TBL, i.e., a certain object existing in the real space is defined as an operation surface CP on which input operation is performed on the image VI. The position on the operation surface CP in the real space is associated with coordinates in an image space in the image VI. When the pen mouse 30 exists on the operation surface CP, the coordinates in the image space that are associated with the position of the pen mouse 30 in the real space are received as input coordinates to the image VI. Based on the input coordinates, the operation of the image presentation unit 120 is controlled. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a head mounted display that is mounted on a user's head and presents an image to the user.

  2. Description of the Related Art Conventionally, a technique related to a see-through type head mounted display (hereinafter referred to as “HMD”) that is worn on a user's head and presents an image to the user has been proposed. For example, Patent Document 1 discloses a configuration in which a user performs input using a pen with respect to an image presented by an HMD as means for receiving input from the user.

JP 2005-339267 A

  The HMD disclosed in Patent Document 1 has a configuration in which a user inputs an image in the air using a pen. However, since the input is performed on the image in the air, it is difficult for the user to grasp the sense of distance with respect to the image to be input. That is, there is a problem that input is difficult for the user.

  An object of the present invention is to provide an HMD that allows easy input to an image presented by the HMD.

  In the present invention made in view of the above-described conventional problems, a predetermined range on the surface of an arbitrary entity existing in real space is defined as an operation surface for performing an input operation on an image. And the position in the real space in an operation surface is matched with the coordinate in the image space in an image. And the coordinate in the image space matched with the position in real space is received as an input position with respect to an image.

  In order to solve the above-described problems, a first problem-solving means that reflects one aspect of the present invention is an image that is held by a mounting unit that is mounted on the user's head and presents the image to the user's eyes so as to be visible. A presentation unit, a movement amount detection unit that is held in a user's hand and detects a movement amount of the user's hand, and a position determination unit that determines a position of the movement amount detection unit in real space based on the movement amount Defining means for defining a predetermined range on the surface of an arbitrary entity existing in the real space as an operation surface for performing an input operation on the image; and a position in the real space on the operation surface for an image in the image When the association means for associating with the coordinates in the space and the movement amount detection means exist on the operation surface, the coordinates in the image space associated with the position in the real space of the movement amount detection means are obtained. , Against the image An input accepting means for accepting as input the coordinates, based on the input coordinates, and an image control means for controlling the operation of the image presenting unit.

  According to the first problem solving means, a predetermined range on the surface of an arbitrary entity existing in the real space is defined as an operation surface for performing an input operation on an image. And the position in the real space in the operation surface is matched with the coordinate in the image space in an image. When the movement amount detection means exists on the operation surface, coordinates in the image space associated with the position of the movement amount detection means in the real space are accepted as input coordinates for the image. That is, the user can use the surface of an arbitrary entity existing in the real space as the operation surface. Since the input to the image is performed on the operation surface, the input to the image is easily performed without being influenced by the sense of distance to the image.

  According to a second problem solving means reflecting another aspect of the present invention, in addition to the first problem solving means, the image presentation unit includes a direction detection unit that detects a direction of the image presentation unit, and a user Is configured to be visible in a state where the external image and the image are superimposed, and based on the direction of the image presentation unit and the position of the movement amount detection unit in real space, the movement amount detection unit Determining means for determining whether or not the movement angle detection means is included in the range of the angle of view of the image presented by the image presentation unit. And determining means for changing the position of the operation surface such that the operation surface follows the image presented by the image presentation unit based on the direction of the image presentation unit.

  According to the second problem solving means, when the movement amount detection unit is included in the range of the angle of view of the image presented by the image presentation unit, the operation surface follows the image presented by the image presentation unit. The position of the operation surface is changed. The case where the movement amount detection unit is included in the range of the angle of view of the image is a case where the image visually recognized by the user and the operation surface exist in the same direction. In this case, the user tries to directly input, for example, a predetermined icon provided in the image by the movement amount detection unit. Since the operation surface follows the image, even if the presentation position of the image changes due to the user moving his / her head, the user can directly input the image to be viewed via the operation surface. . In addition, the user feels that an operation is performed on an image, but it is the operation surface that actually accepts the operation. Therefore, it is possible to easily input an image without being influenced by a sense of distance to the image.

  The third problem-solving means reflecting another aspect of the present invention is characterized in that, in addition to the second problem-solving means, the changing means is an angle of view of an image in which the movement amount detecting means is presented by the image presenting unit. If the determination means determines that the position is not included in the range, the image control means does not change the position of the operation surface, and the image control means When the determination means determines that the image is not included in the range, the identifier corresponding to the input coordinates is presented to the image presentation unit.

  According to the third problem solving means, when the movement amount detection unit is not included in the range of the angle of view of the image presented by the image presentation unit, the position of the operation surface is not changed. The case where the movement amount detection unit is not included in the range of the angle of view of the image means that the direction of the image visually recognized by the user is shifted from the direction of the operation surface, in other words, a direction different from the visually recognized image. This is a case where the user inputs to the operation surface existing in the screen. In this case, since the user does not directly input the image, it is not necessary to move the operation surface even if the presentation position of the image changes due to the user moving his / her head. In addition, since the identifier corresponding to the input position is presented, the user can easily grasp which position of the image is input.

  According to a fourth problem solving means reflecting another aspect of the present invention, in addition to any one of the third problem solving means, the movement amount detection means falls within the range of the angle of view of the image presented by the image presentation unit. When the determination means determines that the position is not included, it is determined whether or not there is an overlapping area where the position of the operation surface in the real space and the position of the virtual image in the real space exist, and the overlapping area exists In this case, an operation surface display means for changing the display mode of the overlapping area is further provided.

  According to the fourth problem solving means, when the movement amount detection unit is not included in the range of the angle of view of the image presented by the image presentation unit, it is determined whether or not an overlapping region exists. When the overlapping area exists, the display mode of the overlapping area is changed. As described above, the case where the movement amount detection unit is not included in the range of the angle of view of the image is a case where the user inputs to the operation surface that exists in a direction different from the visually recognized image. In the case where the overlapping region exists, for example, the position of the image in the real space is changed by moving the head of the user, and the position of the operation surface in the real space and the position of the image in the real space are at least This is a case where some overlap. In such a case, it is convenient if the user can recognize the position of the operation surface in the real space. That is, the position of the operation surface in the real space can be easily recognized by changing the display mode of the overlapping area.

  According to a fifth problem solving means reflecting another aspect of the present invention, in addition to any one of the second to fourth problem solving means, the definition means may determine whether or not the position of the operation surface is determined by a user. The operation surface is defined using at least three positions in the real space of the movement amount detection unit when an input from the user is acquired by the acquisition unit. .

  According to the fifth problem solving means, the operation surface is defined using at least three positions in the real space of the movement amount detection unit when the acquisition unit acquires an input from the user. Therefore, the user can define the operation surface only by performing input at at least three positions in the real space.

  According to a sixth problem solving means reflecting another aspect of the present invention, in addition to the fifth problem solving means, the determination means is configured by the acquisition unit related to the definition of the operation surface by the definition means from the user. After at least one position in the real space of the movement amount detection unit when the input is acquired is determined, the direction of the image presentation unit and the position of the movement amount detection unit in the real space are determined. Based on this, it is determined whether or not the movement amount detection means is included in the range of the angle of view of the image presented by the image presentation unit.

  According to the sixth problem solving means, at least one position in the real space of the movement amount detection unit when the input from the user is acquired by the acquisition unit according to the definition of the operation surface by the definition unit is determined. After that, the determination by the determination means is performed. A determination is made after receiving an input from the user, and an operation surface is defined. Therefore, the operation surface can be defined when the user intends.

  According to the present invention, it is possible to obtain an HMD that allows easy input to an image presented by the HMD.

The figure which shows the outline | summary of HMD1. (A) The top view of the image presentation part 10, (B) The front view of the image presentation part 10, (C) The left view of the image presentation part 10. The block diagram which shows the functional structure of HMD1. (A) The figure which shows the contents which RAM260 temporarily stores, (B) The figure which shows the various processing means which program ROM250 memorizes. The flowchart explaining the process which CPU230 performs. (A) The figure explaining direct mode, (B) The figure explaining indirect mode. The flowchart explaining the operation surface definition process performed in step S30 of FIG. (A) The figure explaining the image for setting the position of the image presentation part 120, (B) The figure explaining the image for setting the center position of the operation surface CP, (C) The upper left position of the operation surface CP. The figure explaining the image for setting, (D) The figure explaining the image for setting the upper right position of the operation surface CP, (E) The figure explaining the image for setting the upper left position of the virtual image VI, (F) The figure explaining the image for setting the upper right position of the virtual image VI. The flowchart explaining the operation surface display process performed in step S65 of FIG. The figure explaining the visual recognition state when overlap area SA exists in indirect mode. The flowchart explaining the operation surface change process performed in step S70 of FIG.

  Embodiments for implementing the above problem solving means reflecting the present invention will be described below in detail with reference to the drawings. The above-mentioned problem solving means is not limited to the configuration described below, and various configurations can be adopted in the same technical idea. For example, in each configuration described below, a predetermined configuration may be omitted or replaced with another configuration. Moreover, you may make it include another structure. Further, the following description is configured by connecting a head mounting unit mounted on the user's head and a control box that provides various signals to the image presentation unit included in the head mounting unit. The HMD will be described as an example, but these devices can be configured as an integrated device.

[Overview of HMD1]
The HMD 1 will be described with reference to FIG. The HMD 1 includes a head mounting unit 10, a control box 20, and a pen mouse 30. The image presentation unit 120 included in the head mounting unit 10 is connected to the control box 20 via the cable 50. The pen mouse 30 is connected to the control box 20 via the cable 51. The head mounting unit 10 is mounted on the head of the user USR. The control box 20 is attached to the waist of the user USR. The HMD 1 defines a predetermined range on the surface of an arbitrary entity existing in the real space as an operation surface CP for performing an input operation on the virtual image VI that is an image presented by the image presentation unit 120. In the present embodiment, a predetermined range on the surface of the table TBL, which is an example of an arbitrary entity, is defined as the operation surface CP. That is, the user USR performs input using the pen mouse 30 on the operation surface CP defined on the surface of the table TBL in a state where the virtual image VI is visually recognized.

  The head mounting part 10 is demonstrated using FIG. The skeleton of the head mounting unit 10 serves as a mounting unit mounted on the user's head. The skeleton of the head mounting unit 10 includes temples 104A and 104B, armor 106A and 106B, and a front frame 108. Moderns 102A and 102B that hit the ears of the user USR are attached to one end of the temples 104A and 104B. The other ends of the temples 104A and 104B are connected to the arms 106A and 106B via hinges 112A and 112B. The armatures 106A and 106B are connected to the left and right ends of the front frame 108. A nose pad 110 is provided in the center of the front frame 108 to abut the user's nose. The temples 104A and 104B can be folded with respect to the front frame 108 by hinges 112A and 112B formed on the armatures 106A and 106B. The structure of the skeleton of the head mounting unit 10 is the same as that of normal glasses, for example. The head mounting portion 10 is held on the user's head by the modern 102A, 102B and the nose pad 110 when mounted on the user USR (see FIG. 1). In FIG. 2B, illustration of the moderns 102A and 102B and the temples 104A and 104B is omitted.

  The image presentation unit 120 presents an image so as to be visible to the user's eyes. The image presenting unit 120 is held on the skeleton of the head-mounted unit 10 via the attachment unit 129 disposed near the armor 106A. The image presentation unit 120 is disposed at a position that is substantially the same height as the left eye LE of the user USR wearing the head mounting unit 10 when attached to the skeleton of the head mounting unit 10. The image presentation unit 120 emits image light based on various signals transmitted from the control box 20 via the cable 50 toward the half mirror 140. The half mirror 140 reflects a part of the image light incident from the image presentation unit 120 and guides it to the left eye LE of the user USR. The half mirror 140 also transmits a part of the external light representing the external image and guides it to the left eye LE of the user USR. That is, the user USR visually recognizes the image light superimposed on the external light. In the present embodiment, the image presentation unit 120 scans light according to the acquired image signal in a two-dimensional direction, guides the scanned image light to the left eye LE of the user, and displays the content image on the retina. It is constructed using a known retinal scanning display. However, the image presentation unit 120 can also be configured using other well-known devices such as a liquid crystal display and an organic EL (Organic Electro-Luminescence) display.

[Functional configuration of HMD1]
As shown in FIG. 3, the HMD 1 includes a head mounting unit 10, a control box 20, and a pen mouse 30. These components are driven by a battery (not shown) included in the control box 20. Hereinafter, each structure which comprises HMD1 is demonstrated.

  The control box 20 includes a flash ROM 210, a peripheral interface (hereinafter referred to as I / F) 220, a CPU 230, a pen mouse I / F 240, a program ROM 250, a RAM 260, and an input button group 270. The flash ROM 250 stores various content data presented to the user USR by the image presentation unit 120. The peripheral I / F 220 connects the HMD 1 and an external device such as a PC. For example, various contents transferred from the PC via the peripheral I / F 220 are stored in the flash ROM 210. The CPU 230 executes various programs stored in the program ROM 250 on the RAM 260. The CPU 230 also transmits and receives various signals among the flash ROM 210, the peripheral I / F 220, the pen mouse I / F 240, and the input button group 270. The pen mouse I / F 240 transmits and receives various signals to and from the pen mouse 30 via the cable 51 (see FIG. 1). The program ROM 250 stores programs for various processes executed by the CPU 230. The RAM 260 temporarily stores various programs processed by the CPU 230 and data processed by the CPU 230 in its address space. The input button group 270 is a configuration for operating the HMD 1. When the user USR operates the input button group 270, an operation signal is transmitted from the input button group 270 to the CPU 230.

  The head mounting unit 10 includes an image presentation unit 120 and an angular velocity sensor 130. The image presentation device 120 emits image light to the left eye LE of the user USR in accordance with a control signal from the CPU 230. The angular velocity sensor 130 functions as a direction detection unit that detects the direction of the image presentation unit 120. The angular velocity sensor 130 is configured by a three-dimensional acceleration sensor that detects a change in capacitance by applying, for example, MEMS technology. The direction of the image presentation unit 120 detected by the angular velocity sensor 130 is transmitted to the control box 20 as an HMD direction signal.

  The pen mouse 30 includes a three-dimensional acceleration sensor 310, a first switch 320, a second switch 330, and an I / F 340. Since the pen mouse 30 is held in the hand of the user USR, the three-dimensional acceleration sensor 310 functions as a movement amount detection unit that detects the movement amount of the user's hand. The movement amount of the user's hand detected by the three-dimensional acceleration sensor 310 is sequentially transmitted to the control box 20 via the I / F 340 as a movement amount signal. The first switch 320 is pressed by the user USR when the main process to be described later is finished. An end signal generated when the first switch 320 is pressed is transmitted to the control box 20 via the I / F 340. The second switch 330 is pressed when the user USR makes various decisions. That is, the second switch 330 functions as an acquisition unit that acquires input from the user. A determination signal generated by pressing the second switch 330 is transmitted to the control box 20 via the I / F 340. The I / F 340 transmits and receives various signals to and from the control box 20 via the cable 51 (see FIG. 1).

  FIG. 4A is a diagram for explaining the contents of data that the RAM 260 temporarily stores in its address space. The RAM 420 includes at least pen mouse position data 261, virtual image position data 262, operation surface position data 263, real space-image conversion formula data 264, input coordinate data 265, HMD direction data 266, icon coordinate data 267, and an input mode flag 268. Memorize temporarily. The pen mouse position data 261 indicates the position of the pen mouse 30 in the real space. The virtual image position data 262 indicates the position of the virtual image VI in the real space. The operation surface position data 263 indicates the position of the operation surface CP in the real space. The real space-image conversion formula data 264 indicates a conversion formula for converting a position in the real space into a coordinate in the image space of the virtual image VI. The input coordinate data 265 indicates the coordinates in the image space of the input made from the pen mouse 30 to the virtual image VI via the operation surface CP. The HMD direction data 266 indicates the direction in which the image presentation unit 120 faces in real space. The icon data 267 indicates the coordinates in the image space of the icon VIa (see FIG. 6) included in the virtual image VI for controlling the operation of the image presentation unit 120, and the processing content assigned to the icon. Note that when the CPU 230 starts reproduction of predetermined content included in the flash ROM 210, the coordinates defining the position of the icon included in the content and the processing content assigned to the icon are temporarily stored in the icon data 267. The The input mode flag 268 indicates whether the input to the virtual image VI is the direct mode or the indirect mode, as shown in FIG.

  FIG. 4B is a diagram for explaining the contents of various processing means stored in the program ROM 250. These processing means are achieved by the CPU 230 executing a program for these processing means stored in the program ROM 250 on the RAM 260. The program ROM 250 stores at least a position determination unit 251, a definition unit 252, an association unit 253, an input reception unit 254, an image control unit 255, a determination unit 256, a change unit 257, and an operation surface display unit 258. Hereinafter, these processing means will be described.

  The position determination unit 251 determines the position of the pen mouse 30 in the real space based on the movement amount of the pen mouse 30. Specifically, the CPU 230 determines the movement amount of the pen mouse 30 in the real space based on the movement amount signal transmitted from the three-dimensional acceleration sensor 310. Then, the CPU 230 adds the movement amount to the pen mouse position data 261 to determine the position of the pen mouse 30 in the real space.

  The defining unit 252 defines a predetermined range on the surface of an arbitrary entity existing in the real space as an operation surface CP for performing an input operation on the virtual image VI presented by the image presentation unit 120. In the present embodiment, as shown in FIG. 1, a predetermined range on the surface of the table TBL is defined as the operation surface CP. Specifically, the CPU 230 causes the image presentation unit 120 to present various setting images (see FIG. 8) for defining the operation surface CP. Then, the CPU 230 determines the position of the operation surface CP in the real space based on the pen mouse position 261 when the determination signal from the second switch 330 is received (refer to FIG. 7 described later for details).

  The association unit 253 associates the position in the real space on the operation surface CP with the coordinates in the image space in the virtual image VI. Specifically, the CPU 230 includes the center position, the upper left position, and the upper right position in the real space of the operation surface CP included in the operation surface position data 263, the center coordinates, the upper left coordinates, and the upper right coordinates in the image space of the virtual image VI. Is used to calculate a conversion formula from a position in the real space to a coordinate in the image space. This conversion formula is, for example, a determinant that performs a primary conversion including translation, rotation, and enlargement / reduction of coordinates. However, a conversion formula including a quadratic or higher term may be used using the above three or more coordinate points. Moreover, the coordinate point used when calculating the conversion formula of primary conversion should just be three or more different places, and is not limited to said center, an upper left position, and an upper right position. The CPU 230 temporarily stores this conversion formula in the RAM 260 as real space-image conversion formula data 264.

  When the pen mouse 30 is present at the position included in the operation surface CP, the input receiving unit 254 uses the coordinates in the image space associated with the position of the pen mouse 30 in the real space as the input coordinates for the virtual image VI. Accept. Specifically, when the pen mouse 30 exists at a position included in the operation surface CP, the CPU 230 uses the real space-image conversion formula data 264 to convert the pen mouse position data 261 into the real space of the pen mouse 30. To coordinates in the image space associated with the position of. Then, the CPU 230 temporarily stores the converted coordinates in the image space in the input coordinate data 265. The determination as to whether or not the pen mouse 30 exists at a position included in the operation surface CP is achieved by comparing the pen mouse position data 261 with the operation surface position data 263.

  The image control unit 255 controls the operation of the image presentation unit 120 based on the input coordinate data 265. Specifically, the CPU 230 compares the input coordinate data 265 with the icon data 267. When the coordinates in the image space included in the input coordinate data 265 match the coordinates in the image space of the icon VIa, the CPU 230 refers to the icon data 267 and performs the operation of the image presentation unit 120 assigned to the icon VIa. A predetermined process for control is executed. The image control unit 255 also causes the image presentation unit 120 to present an identifier corresponding to the input coordinate data 265 when the determination unit 256 determines that the pen mouse 30 is not included in the range of the angle of view of the virtual image VI. Specifically, when the input mode flag 268 is in the indirect mode, the CPU 230 controls the operation of the image presentation unit 120 so as to present the cursor CS (see FIG. 6B) at the coordinates of the input coordinate data 265. .

  Whether the determination unit 256 includes the pen mouse 30 in the range of the angle of view of the virtual image VI based on the direction of the image presentation unit 120 as the detection result of the angular velocity sensor 130 and the position of the pen mouse 30 in the real space. Judge whether or not. Specifically, CPU 230 determines the range of the angle of view of virtual image VI based on HMD direction data 266. Then, the CPU 230 determines whether or not the pen mouse 30 is included in the field angle range of the virtual image VI based on the field angle range of the virtual image VI and the pen mouse position data 261.

  When the determining unit 256 determines that the pen mouse 30 is included in the range of the angle of view of the virtual image VI, the changing unit 257 causes the operation surface CP to follow the virtual image VI based on the direction of the image presentation unit 120. The position of the operation surface CP is changed. Specifically, when the input mode flag 268 is in the direct mode, the CPU 230 determines a moving amount of the virtual image VI in the real space based on the HMD direction data 266 and adds the moving amount to the operation surface position data 263. .

  In the indirect mode where the position of the operation surface CP in the real space and the position of the virtual image VI in the real space do not coincide (details will be described later), it is convenient that the user can recognize the position of the operation surface CP in the real space. . Therefore, the operation surface display unit 258 displays the operation surface CP in the overlap region SA in the virtual image VI when there is an overlap region SA where the position of the real space of the virtual image VI and the position of the operation surface CP in the indirect mode overlap. (Refer to FIG. 10). Specifically, the CPU 230 compares the virtual image position data 262 and the operation surface position data 263 to determine whether or not the overlapping area SA exists. When the overlapping area SA exists, the CPU 230 changes the display mode of the overlapping area SA in the virtual image VI. On the other hand, in the direct mode in which the position of the operation surface CP in the real space matches the position of the virtual image VI in the real space (details will be described later), the operation surface display means 258 is not executed.

[Processing performed by HMD1]
Hereinafter, a series of processes performed by the HMD 1 will be described with reference to FIG. Note that the processing shown in FIG. 5 is triggered by the CPU 230 receiving a predetermined signal. Specifically, for example, in a state in which the virtual image VI is presented by the image presentation unit 120, an operation that occurs when the user USR performs an operation to start defining the operation surface CP on the input button group 270. The processing of FIG. 5 is started when the CPU 230 receives the signal.

  In step S <b> 10, the CPU 230 starts execution of the position determination unit 251. Specifically, the CPU 230 determines the position of the pen mouse 30 in the real space based on the movement amount signal transmitted from the three-dimensional acceleration sensor 310 of the pen mouse 30. The CPU 230 sets the position of the pen mouse 30 in the real space when the movement amount signal is first received as the origin position of the pen mouse 30. Then, the CPU 230 determines the movement amount of the pen mouse 30 in the real space based on the movement amount signal received after setting the origin position of the pen mouse 30. Then, the CPU 230 determines the position of the pen mouse 30 in the real space by adding the determined movement amount to the pen mouse position data 261. The position determining unit 251 is always executed while the series of processing shown in FIG. Therefore, the CPU 230 sequentially receives the movement amount signal from the three-dimensional acceleration sensor 310 and updates the pen mouse position data 261. After execution of the position determining unit 251 is started, the CPU 230 shifts the processing to step S20.

  In step S <b> 20, the CPU 230 starts receiving an HMD direction signal from the angular velocity sensor 130 of the head mounting unit 10. The CPU 230 sets the orientation of the image presentation unit 120 when the HMD direction signal is first received as the initial orientation of the image presentation unit 120. Then, the CPU 230 determines the moving angle of the image presentation unit 120 based on the HMD direction signal received after setting the initial orientation of the image presentation unit 120. Then, the CPU 230 determines the orientation of the image presentation unit 120 by adding the determined movement angle to the HMD direction data 266. Note that while the series of processing illustrated in FIG. 5 is being performed by the CPU 230, the CPU 230 always executes the processing of S20. Therefore, the CPU 230 sequentially receives the HMD direction signal from the angular velocity sensor 130 and updates the HMD direction data 266. After the reception of the HMD direction signal from the angular velocity sensor 130 is started, the CPU 230 shifts the processing to step S30.

  In step S30, the CPU 230 executes the definition unit 252 (details will be described later with reference to FIG. 7). By this processing, the CPU 230 defines the position in the real space of the operation surface CP that performs an input operation on the virtual image VI. Then, the CPU 230 temporarily stores the position of the operation surface CP in the real space in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S40.

  In step S <b> 40, the CPU 230 starts executing the input receiving unit 254. Specifically, the CPU 230 compares the pen mouse position data 261 with the operation surface position data 263 to determine whether or not the pen mouse 30 exists at a position included in the operation surface CP. When the pen mouse 30 exists at the position included in the operation surface CP, the CPU 230 uses the real space-image conversion formula data 264 to associate the pen mouse position data 261 with the position of the pen mouse 30 in the real space. Convert to coordinates in the specified image space. The CPU 230 temporarily stores the converted coordinates in the image space in the RAM 260 as input coordinate data 265. Note that while the series of processes shown in FIG. 5 is being performed by the CPU 230, the CPU 230 always executes the process of S40. Therefore, the CPU 230 sequentially updates the input coordinate data 265 when the pen mouse 30 exists in the operation surface CP. On the other hand, when the pen mouse 30 does not exist in the operation surface CP, the input coordinate data 265 is not updated. Therefore, once the pen mouse 30 is separated from the operation surface CP, the input coordinate data 265 when the pen mouse 30 is finally in the operation surface CP is held. Then, when the pen mouse 30 again exists in the operation surface CP, the input coordinate data 265 is updated. After starting execution of the input receiving unit 254, the CPU 230 shifts the processing to step S50.

  In step S <b> 50, the CPU 230 determines whether the input performed on the operation surface CP using the pen mouse 30 is the direct mode or the indirect mode. Specifically, CPU 230 determines whether input mode flag 268 temporarily stored in RAM 260 is the direct mode or the indirect mode. Note that the direct mode is a state in which the position of the operation surface CP in the real space matches the position of the virtual image VI in the real space, as shown in FIG. In FIG. 6A, the operation surface CP defined in a predetermined range on the surface of the table TBL is included in the range of the angle of view of the virtual image VI presented by the image presentation unit 120. In other words, the user USR visually recognizes the virtual image VI with the virtual image VI superimposed on the table TBL. Therefore, the user USR visually recognizes the virtual image VI as if the virtual image VI is presented on the surface of the table TBL. Therefore, in the direct mode, it is possible to obtain an operational feeling as if the virtual mouse VI is directly input using the pen mouse 30. The indirect mode is a state where the position of the virtual image VI in the real space does not match as shown in FIG. 6B. In FIG. 6B, the operation surface CP is defined on the surface of the table TBL existing outside the range of the angle of view of the virtual image VI. Therefore, in the indirect mode, an operational feeling as if the surface of the table TBL is an input device for the virtual image VI is obtained. When the input mode flag 268 is in the direct mode (step S50: Y), the CPU 230 shifts the processing to step S70. On the other hand, when the input mode flag 268 is in the indirect mode (step S50: N), the CPU 230 shifts the processing to step S60.

  In step S <b> 60, the CPU 230 causes the image presentation unit 120 to present the cursor CS as an identifier corresponding to the input coordinate data 265 by executing the image control unit 255. Specifically, as illustrated in FIG. 6B, the CPU 230 transmits a control signal to the image presenting unit 120 so that the cursor CS is presented at the coordinates of the virtual image VI equal to the input coordinate data 265. To do. The user USR can recognize the input coordinates performed on the virtual image VI by the pen mouse 30 by visually recognizing the cursor CS. In FIG. 6B, an arrow is used as the cursor CS. However, the cursor CS may have any shape as long as the input coordinates can be recognized. The CPU 230 refers to the input coordinate data 265 and sequentially updates the presentation position of the cursor CS. After the presentation of the cursor CS is started, the CPU 230 shifts the processing to step S65.

  In step S65, the CPU 230 executes the operation surface display unit 258 (details will be described later with reference to FIG. 9). When there is an overlapping area SA where the real space position of the virtual image VI and the real space position of the operating surface CP overlap by the execution of the operation surface display means 258, the operating surface CP in the overlapping area SA is displayed in the virtual image VI. The Thereafter, the CPU 230 shifts the processing to step S80.

  In step S70, the CPU 230 executes the changing unit 257 (details will be described later with reference to FIG. 11). By the execution of the changing unit 257, the position of the operation surface CP is changed so that the operation surface CP follows the virtual image VI. In the direct mode, it is possible to obtain an operational feeling as if an input is performed directly on the virtual image VI using the pen mouse 30. Therefore, by changing the position of the operation surface CP so that the operation surface CP follows the virtual image VI, the user USR directly touches the virtual image VI with respect to the virtual image VI without worrying about the position of the operation surface CP. You can input using. Thereafter, the CPU 230 shifts the processing to step S80.

  In step S80, the CPU 230 determines whether or not the pen mouse 30 exists at a position included in the operation surface CP. Specifically, the CPU 230 compares the pen mouse position data 261 with the operation surface position data 263. As a result of the comparison, when the pen mouse 30 is present on the operation surface CP (step S80: Y), the CPU 230 shifts the processing to step S90. On the other hand, when the pen mouse 30 does not exist on the operation surface CP (step S80: N), the CPU 230 shifts the processing to step S120.

  In step S <b> 90, the CPU 230 determines whether or not the second switch 330 of the pen mouse 30 has been pressed. Specifically, CPU 230 determines whether or not a determination signal from second switch 330 has been received. When the second switch 330 has been pressed (step S90: Y), the CPU 230 shifts the process to step S100. On the other hand, when the second switch 330 is not pressed (step S90: N), the CPU 230 shifts the processing to step 120.

  In steps S100 and S110, the CPU 230 executes the image control means 255. Specifically, in step S100, the CPU 230 compares the input coordinate data 265 with the icon data 267. When the coordinates in the image space included in the input coordinate data 265 match the coordinates in the image space of the icon VIa (step S100: Y), the CPU 230 shifts the process to step S110. On the other hand, if they do not match (step S100: N), the CPU 230 shifts the processing to step S120. In step S <b> 110, the CPU 230 executes a predetermined process for controlling the operation of the image presentation unit 120 assigned to the icon VIa that matches the position of the input coordinate data 265. This predetermined process may be any process as long as the operation of the image presentation unit 120 is controlled. As an example of the predetermined processing, there is a start of reproduction, a pause, an end of reproduction, and the like of the content presented by the image presentation unit 120. Further, as another example of the predetermined processing, control unrelated to the content, such as power-off of the image presentation unit 120 or adjustment of the presentation brightness of the virtual image VI, may be used. Thereafter, the CPU 230 shifts the processing to step S120.

  In step S120, the CPU 230 determines whether or not the first switch 320 of the pen mouse 30 has been pressed. Specifically, CPU 230 determines whether or not an end signal from first switch 320 has been received. When the first switch 320 is pressed (step S120: Y), the CPU 230 ends a series of processes. On the other hand, when the first switch 320 is not pressed (step S120: N), the CPU 230 returns the process to step S40.

[Processing content of defining means 252]
Hereinafter, processing contents when the definition unit 252 is executed by the CPU 230 in step S30 of FIG. 5 will be described with reference to FIG.

  In step S310, the CPU 230 causes the image presentation unit 120 to display the image presentation unit position setting image shown in FIG. By this step S310, the user USR is prompted to press the second switch 330 with the pen mouse 30 in contact with the image presentation unit 120. Thereafter, the CPU 230 shifts the processing to step S311.

  In step S311, the CPU 230 determines whether or not the second switch 330 of the pen mouse 30 has been pressed, as in step S90 of FIG. When the second switch 330 is pressed (step S311: Y), the CPU 230 shifts the process to step S312. On the other hand, when the second switch 330 is not pressed (step S311: N), the CPU 230 returns the process to S311 again and waits for the second switch 330 to be pressed.

  In step S311, the CPU 230 temporarily stores the pen mouse position data 261 when the second switch 330 is pressed in step S311 as the position in the real space of the image presentation unit 120 in the HMD direction data 266. Thereafter, the CPU 230 shifts the processing to step S313.

  In step S313, the CPU 230 causes the image presentation unit 120 to display the operation surface center setting image shown in FIG. By this step S313, the user USR is prompted to input using the second switch 330 in order to define the center position of the operation surface CP. Thereafter, the CPU 230 shifts the processing to step S314.

  In step S314, the CPU 230 determines whether or not the second switch 330 of the pen mouse 30 has been pressed, as in step S311. When the second switch 330 is pressed (step S314: Y), the CPU 230 shifts the process to step S315. On the other hand, when the second switch 330 is not pressed (step S314: N), the CPU 230 returns the process to S314 again and waits for the second switch 330 to be pressed.

  In step S315, the CPU 230 temporarily stores the pen mouse position data 261 when the second switch 330 is pressed in step S314 as the center position in the real space of the operation surface CP in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S316.

  In step S316, the CPU 230 executes the determination unit 256. Specifically, the CPU 230 compares the position in the real space of the image presentation unit 120 included in the HMD direction data 266 with the center position in the real space of the operation surface CP included in the operation surface position data 263. Thus, the distance between the image presentation unit 120 and the operation surface CP is determined. Since the angle of view of the virtual image VI presented by the image presentation unit 120 is determined in advance, the CPU 230 determines the distance between the image presentation unit 120 and the operation surface CP, and the image presentation unit 120 included in the HMD direction data 266. Based on the direction, the range of the angle of view of the virtual image VI at the distance of the operation surface CP is determined. In other words, the position of the virtual image VI in the real space is determined. The position of the virtual image VI in the real space and the distance between the image presentation unit 120 and the operation surface CP are temporarily stored in the virtual image position data 262 of the RAM 260. Then, the CPU 230 compares the field angle range with the pen mouse position data 261 to determine whether or not the pen mouse 30 is included in the field angle range of the virtual image VI. When the pen mouse 30 is included in the field angle range of the virtual image VI (step S316: Y), the CPU 230 shifts the process to step S324. On the other hand, when the pen mouse 30 is not included in the field angle range of the virtual image VI (step S316: N), the CPU 230 shifts the process to step S317.

  In step S317, the CPU 230 sets the input mode flag 268 of the RAM 260 to the indirect mode. Thereafter, the CPU 230 shifts the processing to step S318.

  In step S318, the CPU 230 causes the image presentation unit 120 to display an operation surface upper left setting image shown in FIG. By this step S318, the user USR is prompted to input using the second switch 330 in order to define the upper left position of the operation surface CP. Thereafter, the CPU 230 shifts the processing to step S319.

  In step S319, the CPU 230 determines whether the second switch 330 of the pen mouse 30 has been pressed, as in step S311. When the second switch 330 is pressed (step S319: Y), the CPU 230 shifts the process to step S320. On the other hand, when the second switch 330 is not pressed (step S319: N), the CPU 230 returns the process to S319 again and waits for the second switch 330 to be pressed.

  In step S320, the CPU 230 temporarily stores the pen mouse position data 261 when the second switch 330 is pressed in step S319 as the upper left position in the real space of the operation surface CP in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S321.

  In step S321, the CPU 230 causes the image presentation unit 120 to display an operation surface upper right setting image shown in FIG. By this step S321, the user USR is prompted to input using the second switch 330 in order to define the upper right position of the operation surface CP. Thereafter, the CPU 230 shifts the processing to step S322.

  In step S322, the CPU 230 determines whether or not the second switch 330 of the pen mouse 30 has been pressed, as in step S311. When the second switch 330 is pressed (step S322: Y), the CPU 230 shifts the process to step S323. On the other hand, when the second switch 330 is not pressed (step S322: N), the CPU 230 returns the process to S322 again and waits for the second switch 330 to be pressed.

  In step S323, the CPU 230 temporarily stores the pen mouse position data 261 when the second switch 330 is pressed in step S322 as the upper right position in the real space of the operation surface CP in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S331.

  Here, the processing after step S324 when the determination in step S316 is Y will be described. In step S324, the CPU 230 sets the input mode flag 268 of the RAM 260 to the direct mode. Thereafter, the CPU 230 shifts the processing to step S325.

  In step S325, CPU 230 causes image presentation unit 120 to display a virtual image upper left setting image shown in FIG. By this step S325, the user USR is prompted to input using the second switch 330 in order to define the upper left position of the virtual image VI as the upper left position of the operation surface CP. Thereafter, the CPU 230 shifts the processing to step S326.

  In step S326, the CPU 230 determines whether or not the second switch 330 of the pen mouse 30 has been pressed, as in step S311. When the second switch 330 is pressed (step S326: Y), the CPU 230 shifts the process to step S327. On the other hand, when the second switch 330 is not pressed (step S326: N), the CPU 230 returns the process to S326 again and waits for the second switch 330 to be pressed.

  In step S327, the CPU 230 temporarily stores the pen mouse position data 261 when the second switch 330 is pressed in step S326 as the upper left position in the real space of the operation surface CP in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S328.

  In step S328, CPU 230 causes image presentation unit 120 to display the virtual image upper right setting image shown in FIG. By this step S321, the user USR is prompted to input using the second switch 330 in order to define the upper right position of the virtual image VI as the upper right position of the operation surface CP. Thereafter, the CPU 230 shifts the processing to step S329.

  In step S329, the CPU 230 determines whether or not the second switch 330 of the pen mouse 30 has been pressed, as in step S311. When the second switch 330 is pressed (step S329: Y), the CPU 230 shifts the processing to step S330. On the other hand, when the second switch 330 is not pressed (step S329: N), the CPU 230 returns the process to S329 again and waits for the second switch 330 to be pressed.

  In step S330, the CPU 230 temporarily stores the pen mouse position data 261 when the second switch 330 is pressed in step S329 as the upper right position in the real space of the operation surface CP in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S331.

  In step S331, the CPU 230 determines the position of the operation surface CP in the real space based on the center position, the upper left position, and the upper right position of the operation surface CP in the real space temporarily stored in the operation surface position data 263. Specifically, the CPU 230 defines the operation surface CP as a rectangular area with the center position as the center and the upper left position and the upper right position as ends. At this time, the CPU 230 defines the operation surface CP using a plane equation. Then, the CPU 230 determines the boundary of the operation surface CP based on the center position, the upper left position, and the upper right position. The CPU 230 temporarily stores the determined plane equation of the operation surface CP and the boundary of the operation surface CP in the operation surface position data 263. Thereafter, the CPU 230 shifts the processing to step S332.

  In step S <b> 332, the CPU 230 executes the association unit 253. Specifically, the CPU 230 uses the center position, the upper left position, and the upper right position in the real space of the virtual image VI included in the operation surface data 263, and the center coordinates, the upper left coordinates, and the upper right coordinates in the image space of the virtual image VI. Thus, a conversion formula for converting the position in the real space to the coordinate in the image space is calculated. Note that the center coordinates, the upper left coordinates, and the upper right coordinates of the virtual image VI in the image space correspond to the coordinates of the pixel located at the center of the virtual image VI, the coordinates of the pixel located at the upper left, and the coordinates of the pixel located at the upper right, respectively. The CPU 230 temporarily stores the calculated conversion formula in the real space-image conversion formula data 264 of the RAM 260. Further, the CPU 230 temporarily stores the value of the HMD direction data 264 when the conversion formula is calculated in the real space-image conversion formula data 264 in a state linked to the conversion formula. Thereafter, the CPU 230 ends the execution of the definition unit 252 and returns the process to step S40 in FIG.

[Processing contents of operation surface display means 258]
Hereinafter, the contents of the operation surface change process when the operation surface display unit 258 is executed by the CPU 230 in step S65 of FIG. 5 will be described with reference to FIG.

  In step S651, the CPU 230 compares the position of the virtual image VI in the real space with the position of the operation surface CP in the real space. Specifically, based on the position of the operation surface CP included in the operation surface position data 263 in the real space and the position of the image presentation unit 120 included in the HMD direction data 266 in the real space, the CPU 230 The direction of a straight line connecting the presenting unit 120 and the boundary of the operation surface CP is defined. This straight line is defined with respect to the entire circumference of the boundary of the operation surface CP at a predetermined interval. Then, the CPU 230 determines an intersection of the straight line and the virtual image VI based on the direction of the straight line and the position of the virtual image VI included in the virtual image position data 262 in the real space. When an intersection exists, as shown in FIG. 10, the region surrounded by the intersection and the boundary of the virtual image VI is an overlapping region SA where the position of the real space of the virtual image VI and the position of the real space of the operation surface CP overlap. Become. After determining the intersection, the CPU 230 shifts the processing to step S652.

  In step S652, the CPU 230 determines whether or not the overlapping area SA exists based on the comparison result in step S651. This determination is achieved by determining whether or not there is an intersection between the straight line connecting the image presentation unit 120 and the boundary of the operation surface CP and the virtual image VI. When the overlapping area SA exists (step S652: Y), the CPU 230 shifts the process to step S653. On the other hand, when the overlapping area SA does not exist (step S652: N), the CPU 230 ends the execution of the operation surface display unit 258, and returns the process to step S80 in FIG.

  In step S653, the CPU 230 changes the color of the position of the overlapping area SA in the virtual image VI. The color of the overlapping area SA can be changed in any manner as long as the user USR can recognize the operation surface CP. For example, the CPU 230 performs a process of inverting the color of the virtual image VI in the overlapping area SA. Alternatively, the CPU 230 may present the virtual image VI that is visually recognized as if the overlapping area SA is colored by changing the color balance of the virtual image VI in the overlapping area SA. By this processing, as shown in FIG. 10, the user can recognize the overlapping area SA in the virtual image VI. Thereafter, the CPU 230 ends the execution of the operation surface display unit 258, and returns the process to step S80 in FIG.

[Processing contents of changing means 257]
Hereinafter, the contents of the operation surface changing process when the changing unit 257 is executed by the CPU 230 in step S70 of FIG. 5 will be described with reference to FIG.

  In step S701, the CPU 230 compares the orientation of the image presentation unit 120 when the position of the operation surface CP is determined with the current orientation of the image presentation unit 120. Specifically, CPU 230 compares the orientation of image presentation unit 120 included in real space-image conversion formula data 264 with HMD direction data 266. Thereafter, the CPU 230 shifts the processing to step S702.

  In step S702, based on the comparison result in step S701, the CPU 230 determines whether the orientation of the image presentation unit 120 when the position of the operation surface CP is determined matches the current orientation of the image presentation unit 120. Judging. When the directions match (step S702: Y), the CPU 230 ends the execution of the changing unit 257, and returns the process to step S80 in FIG. On the other hand, when the directions match (step S702: N), the CPU 230 shifts the processing to step S703.

  In step S703, the CPU 230 updates the position of the virtual image VI in the real space according to the comparison result in step S701. Specifically, CPU 230 determines the amount of rotation of image presentation unit 120 from the difference between the orientation of image presentation unit 120 when the position of operation surface CP is determined and the current orientation of image presentation unit 120. . Then, CPU 230 determines the amount of movement of the position of virtual image VI in the real space based on the amount of rotation and the distance between image presentation unit 120 and operation surface CP included in virtual image position data 262. The CPU 230 updates the position of the virtual image VI in the real space by adding the movement amount of the position to the virtual image position data 262. Thereafter, the CPU 230 shifts the processing to step S704.

  In step S704, the CPU 230 updates the position of the operation surface CP in the real space. Specifically, CPU 230 adds the movement amount of the position of virtual image VI determined in step S703 in the real space to operation surface position data 263. In the direct mode, the position of the virtual image VI in the real space coincides with the position of the operation surface CP. Therefore, by adding the movement amount of the position of the virtual image VI in the real space to the operation surface position data 263 of the operation surface CP, the position of the operation surface CP in the real space matches the position of the virtual image VI in the real space. To do. Thereafter, the CPU 230 shifts the processing to step S705.

  In step S <b> 705, the CPU 230 updates the conversion formula for converting the position in the real space to the coordinate in the image space, included in the real space-image conversion formula data 264. Specifically, the CPU 230 executes the association unit 253 again using the updated virtual image position data 262. By this processing, the conversion formula included in the real space-image conversion formula data 264 is updated. The CPU 230 also temporarily stores the current value of the HMD direction data 264 in the real space-image conversion formula data 264 in a state linked to this conversion formula. That is, the orientation of the image presentation unit 120 included in the space-image conversion formula data 264 is also updated. Thereafter, the CPU 230 ends the execution of the changing unit 257 and returns the process to step S80 of FIG.

<Modification>
The present invention is not limited to the embodiments described so far, and various modifications and changes can be made without departing from the spirit of the present invention. An example is described below.

  In the embodiment, the structure of the skeleton of the head mounting unit 10 is the same as that of normal glasses, for example. However, the skeleton of the head mounting unit 10 may have another configuration as long as it can be mounted on the head of the user USR while holding the image presentation unit. For example, the skeleton of the head mounting unit 10 may have a shape that can be placed on the user's head like a headband. Or the head mounting | wearing part 10 does not need to have a skeleton. In this case, the head-mounted unit can be configured by the image presentation unit 120 and an attachment unit that can attach the image presentation unit 120 to the glasses that the user USR uses on a daily basis.

  In the embodiment, the image presentation unit 120 is disposed at a position that is substantially the same height as the left eye LE of the user USR wearing the head mounting unit 10 in order to present the virtual image VI to the left eye LE of the user USR. Is done. However, the image presentation unit 120 may be disposed at a position where the virtual image VI can be presented to the right eye of the user USR. Alternatively, the image presentation unit 120 may be provided as a pair of left and right in order to present the virtual image VI to both eyes of the user USR.

  In the embodiment, the user USR performs an input on the virtual image VI using the pen mouse 30. However, it may be other than the pen mouse 30 as long as it is held by the user USR's hand and detects the amount of movement of the user USR's hand. For example, the function of the pen mouse 30 may be incorporated into a ring-type input device that fits the finger of the user USR. Or the shape like the mouse normally used with PC etc. may be used. The pen mouse 30 and the control box 20 may not be connected by the cable 51. In this case, the pen mouse I / F 240 and I / F 340 can be configured by wireless communication such as Bluetooth (registered trademark), for example. Furthermore, since it becomes impossible to supply power from the control box 20 to the pen mouse 30 via the cable 51, the pen mouse 30 is preferably provided with a separate battery.

1 HMD
DESCRIPTION OF SYMBOLS 10 Head mounting part 20 Control box 30 Pen mouse 50, 51 Cable 102A, 102B Modern 104A, 104B Temple 106A, 106B Yoroi 108 Front frame 110 Nose pad 112A, 112B Hinge 120 Image presentation part 129 Mounting part 130 Angular velocity sensor 140 Half mirror 210 Flash ROM
220 Peripheral I / F
230 CPU
240 Pen mouse I / F
250 Program ROM
260 RAM
270 Input button group 310 Three-dimensional acceleration sensor 320 First switch 330 Second switch 340 I / F
CP Operation surface CS Cursor LE Left eye SA of user USR Overlapping area TBL Table USR User VI Virtual image VIa Icon

Claims (6)

An image presentation unit that is held in a mounting unit that is mounted on the user's head and presents the image to the user's eyes so as to be visible;
A movement amount detection unit that is held in the user's hand and detects the movement amount of the user's hand;
Position determining means for determining a position of the movement amount detecting means in real space based on the movement amount;
Defining means for defining a predetermined range on the surface of an arbitrary entity existing in real space as an operation surface for performing an input operation on the image;
Association means for associating a position in the real space on the operation surface with a coordinate in the image space in the image;
Input accepting means for accepting, as input coordinates for the image, coordinates in the image space associated with a position in the real space of the travel amount detecting means when the movement amount detecting means is present on the operation surface. When,
Image control means for controlling the operation of the image presentation unit based on the input coordinates;
A head-mounted display comprising: The image presentation unit
A direction detection unit for detecting the direction of the image presentation unit;
It is configured so that the user can visually recognize the external image and the image superimposed,
Based on the direction of the image presentation unit and the position of the movement amount detection unit in real space, whether or not the movement amount detection unit is included in the range of the angle of view of the image presented by the image presentation unit. A judging means for judging;
When the determination unit determines that the movement amount detection unit is included in the range of the angle of view of the image presented by the image presentation unit, the movement amount detection unit is presented by the image presentation unit based on the direction of the image presentation unit. Change means for changing the position of the operation surface so that the operation surface follows the image;
The head mounted display according to claim 1. When the determination unit determines that the movement amount detection unit is not included in the range of the angle of view of the image presented by the image presentation unit, the changing unit does not change the position of the operation surface.
When the determination unit determines that the movement amount detection unit is not included in the range of the angle of view of the image presented by the image presentation unit, the image control unit displays an identifier corresponding to the input coordinates. To present to the department,
The head mounted display according to claim 2.
When the determination unit determines that the movement amount detection unit is not included in the range of the angle of view of the image presented by the image presentation unit,
It is determined whether or not there is an overlapping area where the position of the operation surface in the real space and the position of the virtual image in the real space overlap, and if the overlapping area exists, the display mode of the overlapping area is changed. It further has an operation surface display means,
The head mounted display according to claim 3. The definition means includes
In order to determine the position of the operation surface, an acquisition unit that acquires input from a user;
Defining the operation surface using at least three positions in the real space of the movement amount detection unit when an input from a user is acquired by the acquisition unit;
The head mounted display in any one of Claims 2-4. In the determination unit, at least one position in the real space of the movement amount detection unit when the input from the user is acquired by the acquisition unit according to the definition of the operation surface by the definition unit is determined. Later, based on the direction of the image presentation unit and the position of the movement amount detection unit in real space, whether the movement amount detection unit is included in the range of the angle of view of the image presented by the image presentation unit Determine whether or not
The head mounted display according to claim 5.