JP6567324B2 - Image display device and head mounted display - Google Patents

Description

  The present invention relates to an image display device for displaying an image.

  As one of wearable computers, an HMD (Head Mounted Display), which is a display device mounted on a user's head, has been known. The HMD is classified into a transmissive HMD that displays an image superimposed on the user's field of view while allowing the user to visually recognize the outside world, and a non-transmissive HMD that displays only an image to the user. For example, in Patent Document 1, as a transmission type HMD, a receiving unit output from an input device, an image display element that presents an image presented to a user, and an operation image based on an operation signal are displayed on the image display element. There is disclosed a head mounted display including a display processing unit.

JP2013-134532A (released on July 8, 2013)

  However, the technique of Patent Document 1 has a problem that it is necessary to prepare an input device in addition to the head-mounted display as a device that receives a user's input operation. The present invention has been made in view of the above-described problems, and an object of the present invention is to realize an image display apparatus and the like that allow an input operation to the own apparatus without requiring an input device of an external apparatus. There is.

  In order to solve the above-described problem, an image display device according to one embodiment of the present invention is superimposed on an image display element whose one side is a display surface and a surface of the image display element opposite to the display surface. A sensor panel, wherein the sensor panel detects a movement of the indicator by a user in a space opposite to the display surface side of the image display element.

  According to one embodiment of the present invention, a user can perform an operation on an image display apparatus without using an input device separately.

It is a principal figure structure of the head mounted display (HMD) which concerns on one Embodiment of this invention. (A) is a perspective view of said HMD provided with the image display apparatus, (b) is a top view of said HMD, (c) is a figure which shows the usage example of said HMD, (d) is said figure It is a figure which shows an example of the visual field of the user wearing HMD. (A) is a figure for demonstrating the detection space which the said image display apparatus detects, (b) And (c) is a figure for demonstrating the detection resolution of the said image display apparatus. (A)-(d) is a figure which shows matching with the content of the process which the said image display apparatus performs, and a predetermined | prescribed gesture. 4 is a flowchart illustrating an example of processing in the image display apparatus according to the first embodiment. It is a flowchart which shows an example of the position specific process in the said image display apparatus. (A) is a flowchart which shows an example of the process in the image display apparatus which concerns on Embodiment 2, (b) is a flowchart which shows an example of the process in the image display apparatus which concerns on Embodiment 3.

Embodiment 1
An embodiment of the present invention will be described below with reference to FIGS. First, an outline of the image display apparatus 1 according to an embodiment of the present invention will be described with reference to FIG. FIG. 2A is a perspective view of a head mounted display (hereinafter referred to as HMD) 10 provided with the image display device 1, and FIG. 2B is a top view of the HMD 10. The HMD 10 is worn on the user's head.

  The HMD 10 in the present embodiment is a transmissive HMD that displays an image superimposed on the user's field of view while allowing the user to visually recognize the outside world. For example, the HMD 10 may be a spectacle type provided with the frame 11 as shown, or may be a shape (not shown) in which the image display device 1 hangs down from the heel portion of the hat. The HMD 10 is not limited to the example described above.

  As shown in FIG. 2B, an image display element 5 is provided on a surface corresponding to the eye of the user when the user wears the HMD 10 in a portion corresponding to the lens of the glasses. In the present embodiment, the image display element 5 is described as a transmissive image display element in which one side is a display surface and presents an image to the user while overlapping the user's field of view while allowing the user to visually recognize the outside world. It is not limited. Further, the image display element 5 may be plate-shaped.

  As shown in FIG. 2A, the sensor panel 2 is provided on the image display element 5 so as to be superimposed on the image display element 5 on the surface opposite to the image display surface (display surface). For example, the sensor panel 2 is a capacitance type sensor and functions as a proximity detection type sensor. The sensor panel 2 may be a capacitive sensor as long as it can detect the movement (including the position) of the indicator (finger) by the user in the space opposite to the display surface side of the image display element 5. It is not limited. Further, the detection surface of the sensor panel 2 is the surface of the sensor panel 2 on the opposite side to the surface of the image display element 5 superimposed on the sensor panel 2. The sensor panel 2 transmits light from at least the detection surface to the display surface of the image display element 5.

  (C) of FIG. 2 is a figure which shows the usage example of HMD10. The user 100 wearing the HMD 10 can perform an input operation to the HMD 10 by pointing the finger h to the HMD 10 and moving the finger h. The finger 100 of the user 100 is detected by the sensor panel 2, and the user 100 can perform an input operation to the HMD 10 according to the movement of the finger h in a space facing the detection surface of the sensor panel 2.

  FIG. 2D is a diagram illustrating an example of the field of view of the user 100 wearing the HMD 10. The user can visually recognize the image 54 displayed on the image display element 5 while visually recognizing the actual image r such as the surrounding background via the image display element 5. The user's finger h is visually recognized by the user as an actual image r, and an input operation can be performed on the image 54. Although details of the input operation will be described later, for example, the size of the screen 53 that is an area where the image 54 is displayed can be enlarged or reduced.

  For this reason, it becomes possible to operate the image display apparatus 1 without using an input device separately. For example, the user wearing the HMD 10 can perform an input operation in the space on the back side of the image 54 while visually recognizing the image 54. Therefore, it is possible to provide a more user-friendly operational feeling than when using an input device or the like. Note that the image display device 1 provided in the HMD 10 may be one, or may be two for the right eye and for the left eye as shown in FIGS. 2 (a) and 2 (b).

(Detection space of sensor panel 2)
Next, the detection resolution of the sensor panel 2 will be described. FIG. 3A is a diagram for explaining a detection space detected by the image display device 1. The detection resolution of the sensor panel 2 is changed according to the distance z between the detection surface of the sensor panel 2 and the detection target (the indicator by the user, the user's finger h). As shown in FIG. 3A, the space perpendicular to the detection surface of the sensor panel 2 (the space on the detection surface of the sensor panel 2) depends on the distance z from the detection surface of the sensor panel 2. The detection spaces I, II, and III and the non-detection space (dead zone). The non-detection space (dead zone) and the detection space I are divided by the reference value a, and the detection space I and the detection space II are divided by the reference value b. The detection space III represents the detection surface of the sensor panel 2.

  For example, when the reference value a = 15 cm and the reference value b = 10 cm are set, the detectable space of the sensor panel is a space where the distance z from the detection surface is 15 cm or less. In this case, the detection space I is a space where the distance z from the detection surface is 10 cm or more and 15 cm or less, the detection space II is a space where the distance z from the detection surface is less than 10 cm, and the non-detection space is This is a space where the distance z from the detection surface exceeds 15 cm.

  In the present embodiment, it is not assumed that a detection target (finger) that is in contact with the detection surface of the sensor panel 2 is detected, but the detection surface of the sensor panel 2 is defined as the detection space III. It is also possible to detect a detection target that is in contact with the detection surface.

  3B to 3D are diagrams for explaining the detection resolution of the sensor panel 2. FIG. As shown in FIGS. 3B to 3D, the sensor panel 2 has a plurality of vertical electrodes (a plurality of vertical electrodes) arranged at predetermined intervals in a direction along one side of the detection surface (referred to as the vertical direction). Wire electrodes) and a plurality of horizontal electrodes (wire electrodes) arranged at a predetermined interval in a direction perpendicular to the vertical direction (referred to as the left-right direction) and arranged in a matrix so that these intersect each other It has an arrayed two-dimensional electrode structure. Although the sensor panel 2 in the present embodiment is described as having a two-dimensional electrode structure in which a plurality of wire electrodes are arranged, the structure of the sensor panel 2 is not limited to this. For example, the sensor panel 2 may have a two-dimensional electrode structure in which a plurality of electrodes are two-dimensionally arranged.

  The detection resolution of the sensor panel 2 is determined by the interval between the detection electrodes, and increases as the interval between the detection electrodes becomes smaller. The detection space differs depending on the interval between the detection electrodes. If the electrode interval that actually contributes as the detection electrode is wide, a detection space wide as much can be detected. The detection resolution is switched by thinning out the number of horizontal electrodes and vertical electrodes constituting the sensor panel 2 by electrical processing.

  For example, in the case of the detection space III, as shown in FIG. 3B, all the electrodes described above are used, and the sensor panel 2 that detects the detection target is configured in the detection space III (pattern <3>). ). In this case, the minimum detection area is the electrode spacing of the detection electrodes, and the coordinate points (indicated by black circles) to be detected are the most dense as shown in FIG.

  Further, in the case of the detection space II, as shown in FIG. 3C, the detection object is detected in the detection space II by thinning out every other vertical electrode and horizontal electrode from the state of FIG. Is configured (pattern <2>). The minimum detection area in this case is expanded four times compared to the minimum detection area in the detection space III. Further, the detection resolution is lower than that of the detection space III.

  In the case of the detection space I, as shown in FIG. 3 (d), the detection object is detected in the detection space I by thinning out every third vertical electrode and horizontal electrode from the state shown in FIG. 3 (b). A sensor panel 2 to be detected is configured (pattern <1>). In the case of pattern <1>, the minimum detection area is expanded four times and the detection resolution is lower than in the case of pattern <2>.

(Configuration of main part of image display device 1)
FIG. 1 is a schematic diagram of the HMD 10. An image display device 1 provided in the HMD 10 includes a sensor panel 2, a control unit 3, a storage unit 4, and an image display element 5. The sensor panel 2 detects a change in capacitance value that occurs in a space on the detection surface of the sensor panel 2 and transmits information indicating the changed capacitance value to the control unit 3.

  The control unit 3 executes various processes in the image display apparatus 1 and executes processes corresponding to gestures. The position specifying unit 31, the gesture determining unit 32, the determining unit 33, and the display control unit 36. It has.

  The position specifying unit 31 specifies the position (x coordinate, y coordinate, z coordinate) of the detection target facing (opposing) the detection surface of the sensor panel 2 from the detection data by the sensor panel 2. The two-dimensional coordinates set on the detection surface of the sensor panel 2 are x and y coordinates, and the direction orthogonal to the detection surface of the sensor panel 2 is the z-axis direction.

  The position specifying unit 31 switches the detection resolution of the sensor panel 2 according to the specified z coordinate (hereinafter, distance z). For example, when the position (distance z) of the detection target specified by the position specifying unit 31 is between the reference value a and the reference value b shown in FIG. The electrode configuration is set to the pattern <1> shown in FIG. 3C, and the detection space information indicating that the detectable space of the sensor panel 2 is the detection space I is transmitted to the determination unit 33.

  In addition, the position specifying unit 31 in the first embodiment transmits position information indicating the specified x-coordinate and y-coordinate of the detection target to the gesture determining unit 32. The position specifying process of the position specifying unit 31 will be described later with reference to FIG.

  The gesture determination unit 32 specifies a detection target trajectory (input operation) from the position information received from the position specifying unit 31, and determines whether or not the specified trajectory is a predetermined gesture. The predetermined gesture is a gesture defined by the processing content information 41 stored in the storage unit 4, for example, “draw a circle” or “move in the horizontal direction”. When the detection target trajectory is a predetermined gesture, the gesture determination unit 32 transmits gesture information indicating the determined gesture to the determination unit 33.

  The determination unit 33 includes an application determination unit 34 and a function determination unit 35, and determines the processing content of the image display device 1 with reference to the processing content information 41 stored in the storage unit 4. When receiving the detection space information from the position specifying unit 31, the application determining unit 34 determines an application to be activated with reference to the processing content information 41, and transmits information indicating the determined application to the function determining unit 35.

  When receiving the gesture information from the gesture determining unit 32, the function determining unit 35 refers to the processing content information 41, and in the application indicated by the information received from the application determining unit 34, a function associated with the gesture indicated by the gesture information (predetermined) Function). Then, the function determination unit 35 causes the display control unit 36 to execute the determined function.

  The display control unit (processing execution unit) 36 controls an image to be displayed on the image display element 5 in accordance with an instruction from the function determination unit 35. The image display element 5 has an LCD and an optical system, and presents an image formed by the LCD to the user via the optical system. The optical system is configured such that light emitted from the LCD is polarized and guided to the eyes of the user. The optical system includes, for example, a light guide plate made of a transparent substrate capable of guiding light by total reflection, and a reflective volume hologram diffraction grating capable of diffraction reflection. Note that the image display element 5 is not limited to the above configuration, and the LCD itself may be disposed in front of the user's eyes, or may be applied to the present invention as long as it can be used as an HMD. .

  The storage unit 4 stores information, and includes a nonvolatile storage device such as a flash memory and a ROM (Read Only Memory) and a volatile storage device such as a RAM (Random Access Memory). Examples of contents stored in the nonvolatile storage device include various programs, various operation setting values, and various data. On the other hand, examples of the contents stored in the volatile storage device include a working file and a temporary file. In the present embodiment, the storage unit 4 includes processing content information 41.

(Processing contents of image display device)
Processing content information referred to by the image display device 1 will be described. 4A to 4D are diagrams illustrating an example of the processing content information 41 referred to by the control unit 3 of the image display device 1. The processing content (predetermined function) of the control unit 3 and the gesture are illustrated in FIG. It is an associated table. The process of the control unit 3 is a process for an image displayed by the image display element 5. For example, a process for editing the image (including changing the scale of the image) may be performed, and the user opens a folder when a user operation such as double-clicking the image of the displayed folder is executed. It may be a process.

  The table referred to by the image display device 1 according to the first embodiment is the processing content information shown in FIG. 4A, and includes application functions assigned to each detection space and user gestures that trigger the execution of the functions. Are associated with each other. The HMD 10 in the first embodiment is configured to include one image display device 1.

  For example, the application “screen enlargement / reduction” and the function “screen reduction” are assigned to the detection space I, and the gesture “draw a circle” is associated with the function. For this reason, when the gesture “draw a circle” is detected in the detection space I, the size of the screen 53 shown in FIG. 2D is reduced. Even if the gesture “moving horizontally” in the detection space I is performed, the function “screen reduction” is not executed.

  Further, for example, the application “screen enlargement / reduction” and the function “screen enlargement” are assigned to the detection space II, and the gesture “moving horizontally” is associated with the function. For this reason, when the gesture “moving in the horizontal direction” is detected in the detection space II, the size of the screen 53 shown in FIG. Even if the gesture “draw a circle” is performed in the detection space II, the function “enlarge screen” is not executed. 4B will be described later in a modification of the present embodiment, FIG. 4C will be described later in the second embodiment, and FIG. 4D will be described later in the third embodiment.

(Function decision processing)
FIG. 5 is a flowchart showing an example of processing in the image display apparatus 1 according to Embodiment 1 of the present invention. As illustrated in FIG. 5, the position specifying unit 31 acquires detection data from the sensor panel 2 and specifies a position (x coordinate, y coordinate, z coordinate) of a detection target (such as a user's finger). (S1). Details of the position specifying process will be described later with reference to FIG.

  The position specifying unit 31 transmits position information indicating the x coordinate and the y coordinate of the detection target specified in S <b> 1 to the gesture determining unit 32. In addition, the position specifying unit 31 in the first embodiment transmits detection space information indicating the detectable space of the sensor panel 2 set according to the z coordinate of the detection target specified in S1 to the application determining unit 34.

  When receiving the detection space information corresponding to the position (z coordinate) of the detection target specified in S1, the application determination unit 34 indicates which detection space the detection space information indicates (whether it indicates the detection space I or II). (S2). When the detection space information indicates the detection space I or II (YES in S2), the application determining unit 34 refers to the processing content information 41 and determines to start the application assigned to the detection space ( S3). On the other hand, if NO in S2, the process returns to S1.

  In addition, when the gesture determination unit 32 receives position information indicating the position (x coordinate and y coordinate) of the detection target specified in S <b> 1, the gesture determination unit 32 specifies the detection target trajectory from the received position information. It is determined whether or not the gesture is a predetermined gesture (S4). If the locus of the detection target is a predetermined gesture (YES in S4), the gesture determination unit 32 transmits gesture information indicating the determined gesture to the function determination unit 35.

  And the function determination part 35 will determine performing the function matched with the gesture which gesture information shows in the application determined by the application determination part 34, if the gesture information which shows the gesture determined in S4 is received. Then, the display control unit 36 is caused to execute the determined function (S5). On the other hand, if NO in S4, the process returns to S1.

  For example, in the processing content information shown in FIG. 4A, when the position (z coordinate) of the detection target specified in S1 is the detection space I, the application assigned to the detection space I “screen enlargement / When “reduction” is determined (S3) and the gesture determined in S4 is “draw a circle”, the function “reduction of screen” is determined. The display control unit 36 reduces the size of the screen 53 shown in FIG. 2D in accordance with the notification from the function determining unit 35.

  According to said structure, the user can perform input operation for performing a predetermined | prescribed function with respect to the image 54 currently displayed. For example, if an input operation that imagines a predetermined function is a gesture associated with the predetermined function, the user can perform an input operation that is intuitive.

(Positioning process)
FIG. 6 is a flowchart showing an example of the position specifying process in the image display apparatus 1. First, the position specifying unit 31 checks whether or not the electrode interval of the sensor panel 2 is set to the electrode interval of the pattern <1> (S101), and the electrode interval is set to the electrode interval of the pattern <1>. If yes (YES in S101), the process proceeds to S102. On the other hand, when the electrode interval is not set to the electrode interval of the pattern <1> (NO in S101), the process proceeds to S107. In S107, the position specifying unit 31 sets (switches) the electrode interval to the electrode interval of the pattern <1> and proceeds to S102. The setting of the electrode interval in S107 is a provisional setting. The provisional setting is a temporary setting for preventing the electrode interval contributing to the detection electrode in the sensor panel 2 from becoming indefinite.

  Next, the position specifying unit 31 determines whether or not the detection level (the z coordinate of the detection target) exceeds the reference value a (S102), and when the detection level exceeds the reference value a (YES in S102), The process proceeds to S103. On the other hand, when the detection level does not exceed the reference value a (NO in S102), the operation of S102 is repeated from the beginning. Then, the position specifying unit 31 determines whether or not the detection level exceeds the reference value b (S103). If the detection level exceeds the reference value b (YES in S103), the process proceeds to S104. On the other hand, if the detection level does not exceed the reference value b (NO in S103), the process proceeds to S108.

  In S108, the position specifying unit 31 sets the electrode interval of the sensor panel 2 as the optimal electrode interval in the pattern <1>. After S108, the position specifying unit 31 specifies the position (x coordinate, y coordinate, and z coordinate) of the detection target from the detection data by the sensor panel 2 (S109), and returns to S101.

  In the position specifying process in the first embodiment, the position specifying unit 31 transmits the position information indicating the x coordinate and the y coordinate of the detection target specified in S109 to the gesture determining unit 32, and the sensor set in S108. Detection space information indicating a detectable space (detection space I) of the panel 2 is transmitted to the application determination unit 34.

  Next, the position specifying unit 31 checks whether or not the electrode interval of the pattern <2> is set (S104), and when the electrode interval is set to the electrode interval of the pattern <2> (YES in S104) ), And proceeds to S105. On the other hand, when the electrode interval is not set to the electrode interval of the pattern <2> (NO in S104), the process proceeds to S110. In step S110, the position specifying unit 31 sets (switches) the electrode interval to the electrode interval of the pattern <2>, and proceeds to S105. Note that the setting of the electrode interval in S110 is a provisional setting.

  In S105, the position specifying unit 31 sets the electrode interval of the sensor panel 2 as the optimum electrode interval in the pattern <2>. After S105, the position specifying unit 31 specifies the position (x coordinate, y coordinate, and z coordinate) of the detection target from the detection data by the sensor panel 2 (S106), and returns to S103.

  In the position specifying process in the first embodiment, the position specifying unit 31 transmits position information indicating the x coordinate and the y coordinate of the detection target specified in S106 to the gesture determining unit 32, and the sensor set in S105. Detection space information indicating a detectable space (detection space II) of the panel 2 is transmitted to the application determination unit 34. The position specifying process described above is performed every arbitrary time.

(Modification)
Although the HMD 10 provided with one image display device 1 has been described above, the HMD 10 may be configured to include two image display devices 1 for the right eye and the left eye. In this configuration, the detection spaces I and II detected by the sensor panel 2 may be divided into the right eye side and the left eye side, and different applications may be assigned to the divided detection spaces. Here, a space detected by the right-eye sensor panel 2 is defined as an operation area A, and a space detected by the left-eye sensor panel 2 is defined as an operation area B.

  The table referred to by the image display device 1 in the modification of the first embodiment is the processing content information shown in FIG. 4B. For example, different applications are assigned to the operation area A and the operation area B, respectively. . More specifically, when the gesture “draw a circle” is detected in the detection space I in the operation area A, the size of the screen 53 shown in FIG. When the gesture “moving in the horizontal direction” is detected in the space II, the size of the screen 53 shown in FIG. Further, when the gesture “draw a circle” is detected in the detection space I in the operation area B, the image 54 shown in FIG. 2D is returned to the image 54 displayed previously, and the operation area When the gesture “moving in the horizontal direction” is detected in the detection space II in B, the image 54 shown in FIG. 2D proceeds to the image 54 displayed next.

  According to said structure, the user can perform input operation with respect to each sensor panel 2 for right eyes and left eyes, and can perform a different predetermined function. The image display element 5 only needs to be provided on at least one of the right-eye sensor panel 2 and the left-eye sensor panel 2, and the image 54 shown in FIG. 2D is an arbitrary image display element. 5 is the configuration displayed.

[Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIG. 4C and FIG. 7A. For convenience of explanation, members having the same functions as those described in the embodiment are given the same reference numerals, and descriptions thereof are omitted. The HMD 10 in the second embodiment is configured to include two image display devices 1 for the right eye and the left eye. The image display device 1 according to the second embodiment includes a detection target (user's) in an operation area A that is a space detected by the right-eye sensor panel 2 and an operation area B that is a space detected by the left-eye sensor panel 2. It differs from the image display device 1 in the first embodiment in that each of the other fingers is detected.

  The table referred to by the image display device 1 in the second embodiment is the processing content information shown in FIG. 4C, and an application, a function, and a gesture are associated with each detection space. For example, in the detection space I, an application is assigned to the operation area A, and a function and a gesture are associated with each other in the operation area B.

  More specifically, when the user stands up with the finger of the right hand in the operation area A in the detection space I, the application determining unit 34 selects the application “screen enlargement / reduction” assigned to the operation area A in the detection space I. To decide. Further, when the user performs a predetermined gesture “draw a circle” in the operation area B in the detection space I with the finger of the left hand, the function determination unit 35 determines the function “reduction” associated with the gesture. Then, the display control unit 36 reduces the size of the screen 53 shown in FIG. In this embodiment, the function executed by the display control unit 36 when the detection target detected in the operation area A and the detection target detected in the operation area B are in the same detection space. It is determined.

(Another example of function decision processing)
(A) of FIG. 7 is a flowchart which shows an example of the process in the image display apparatus 1 which concerns on Embodiment 2 of this invention. As illustrated in FIG. 7A, the position specifying unit 31 acquires detection data from the right-eye sensor panel 2, and detects the position (x coordinate, y) of the detection target (such as a user's finger) in the operation area A. A position specifying process for specifying (coordinate, z coordinate) is performed (S21). And the position specific | specification part 31 transmits the detection space information which shows the space which can be detected of the sensor panel 2 set according to the z coordinate of the detection target specified in S21 to the application determination part 34. FIG. The specific process of S21 is the same as the position specifying process of FIG. 6, but may be a process of specifying only the z coordinate of the detection target.

  Next, when receiving the detection space information corresponding to the position (z coordinate) of the detection target specified in S21, the application determination unit 34 detects with reference to the processing content information 41 shown in FIG. It is determined to start the application assigned to the space (S22).

  The position specifying unit 31 acquires detection data from the left-eye sensor panel 2 and specifies a position (x coordinate, y coordinate, z coordinate) of a detection target (such as a user's finger) in the operation area B. A specific process is performed (S23). And the position specific | specification part 31 transmits the positional information which shows the x coordinate and y coordinate of the detection target specified in S23 to the gesture determination part 32. FIG. The specific process of S23 is the same as the position specifying process of FIG. 6, but may be a process of specifying the x coordinate and y coordinate of the detection target.

  And the gesture determination part 32 will specify the locus | trajectory of a detection target from the received positional information, if the positional information which shows the position of the detection object specified in S23 is received, and whether the specified locus | trajectory is a predetermined | prescribed gesture? Whether or not is determined with reference to the processing content information 41 shown in FIG. 4C (S24). If the locus of the detection target is a predetermined gesture (YES in S24), the gesture determination unit 32 transmits gesture information indicating the determined gesture to the function determination unit 35.

  And the function determination part 35 will determine performing the function matched with the gesture which gesture information shows in the application determined by the application determination part 34, if the gesture information which shows the gesture determined in S24 is received. Then, the display control unit 36 is caused to execute the determined function (S25). On the other hand, if NO in S24, the process returns to S21.

  According to the above configuration, an application to be activated is determined from the position (z coordinate) of the user's finger detected by the right-eye sensor panel 2, and the movement of the user's finger detected by the left-eye sensor panel 2 ( It is possible to determine the function to be executed from the gesture. Therefore, it is possible to perform an input operation in accordance with the user's intuition while increasing variations of functions to be executed.

[Embodiment 3]
The following will describe another embodiment of the present invention with reference to FIG. 4D and FIG. 7B. The HMD 10 in the third embodiment is configured to include one image display device 1. In the third embodiment, an application is determined by detecting a predetermined gesture, and parameters for executing a predetermined function associated with the determined application are detected and detected by the detection target and the sensor panel 2. It differs from the first embodiment in that it is changed in conjunction with the distance z to the surface.

  The table referred to by the image display device 1 in the third embodiment is the processing content information shown in (d) of FIG. In the processing content information, an application is assigned for each predetermined gesture, and a predetermined function is associated as an adjustment item for each detection space.

  For example, when the user performs a “draw circle” gesture with the finger in the detection space I or II, the application “enlargement / reduction of screen” is determined. When the detected finger position is in “detection space I”, the adjustment item “screen reduction” is determined, and the size of the screen 53 shown in FIG. Then, it is reduced following the change in the distance z from the detection target finger. When the detected finger position is in “detection space II”, the adjustment item “enlarge screen” is determined, and the size of the screen 53 shown in FIG. And is enlarged following the change in the distance z with the finger to be detected.

(Another example of function decision processing)
FIG. 7B is a flowchart illustrating an example of processing in the image display apparatus 1 according to the third embodiment of the present invention. As shown in FIG. 7B, the position specifying unit 31 acquires detection data from the sensor panel 2 and specifies the position (x coordinate, y coordinate, z coordinate) of the detection target (such as a user's finger). The position specifying process is performed (S1). Note that the position specifying unit 31 according to the third embodiment transmits detection space information indicating the detectable space of the sensor panel 2 set according to the z coordinate of the detection target specified in S <b> 1 to the function determining unit 35.

  And the gesture determination part 32 will specify the locus | trajectory of a detection target from the received position information, if the positional information which shows the position (x coordinate and y coordinate) of the detection target identified in S1, and the identified locus | trajectory is Whether or not the gesture is a predetermined gesture is determined with reference to the processing content information 41 shown in FIG. 4D (S31).

  When the detection target trajectory is a predetermined gesture (YES in S31), the gesture determination unit 32 transmits gesture information indicating the determined gesture to the application determination unit 34. When receiving the gesture information indicating the gesture determined in S31, the application determining unit 34 determines to start the application assigned to the detection space with reference to the processing content information illustrated in FIG. (S32). On the other hand, if NO in S31, the process returns to S1.

  Further, when receiving the detection space information corresponding to the position (z coordinate) of the detection target specified in S1, the function determination unit 35 determines whether or not the detection space information indicates the detection space I or II ( S33). When the detection space information indicates the detection space I or II (YES in S33), the function determination unit 35 refers to the processing content information shown in (d) of FIG. 4 and is an adjustment item assigned to the detection space. The display control unit 36 is caused to execute the determined adjustment item in conjunction with the position (z coordinate) of the detection target (S34). On the other hand, if NO in S33, the process returns to S1. According to said structure, the user can perform input operation according to intuition.

[Example of software implementation]
The control block (particularly the position specifying unit 31, the gesture determining unit 32, the application determining unit 34, the function determining unit 35, and the display control unit 36) of the image display device 1 is a logic circuit (IC chip) or the like formed in an integrated circuit (IC chip) or the like. Hardware) or software using a CPU (Central Processing Unit).

  In the latter case, the image display device 1 includes a CPU that executes instructions of a program that is software that implements each function, and a ROM (Read Only Memory) in which the program and various data are recorded so as to be readable by a computer (or CPU). Alternatively, a storage device (these are referred to as “recording media”), a RAM (Random Access Memory) that expands the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  In the description of each of the above embodiments, the correspondence between the gesture and the function of the application is set to be different depending on each detection space. However, the present invention is not limited to this, and each detection space is not limited to this. The gesture and the application function may be associated with each other.

[Summary]
An image display device (1) according to an aspect 1 of the present invention includes an image display element (5) whose one side is a display surface, and a sensor panel superimposed on a surface opposite to the display surface of the image display element ( 2), and the sensor panel detects the movement of the indicator by the user in the space (detection space I and detection space II) opposite to the display surface side of the image display element.

  According to said structure, an image display apparatus is provided with an image display element and the sensor panel superimposed on the image display element, and a sensor panel by the user in the space on the opposite side to the display surface side of an image display element The movement of the indicator is detected. Therefore, the image display apparatus can present an image to the user and detect the user's input operation in a space on the opposite side to the display surface of the image display element. Therefore, the user can perform an operation on the image display apparatus without using an input device separately.

  For example, when the image display device is applied to a head-mounted display, the user can detect a user operation in a space opposite to the display surface of the image display element while visually recognizing the image displayed on the image display element. it can. For this reason, compared with the case where the input device of an external device etc. are used, a user-friendly operation feeling can be provided.

  An image display device according to aspect 2 of the present invention, in aspect 1, the gesture determination unit (32) for determining whether or not the movement of the indicator detected by the sensor panel is a predetermined gesture; When it is determined that the movement of the indicator is the predetermined gesture, a processing execution unit (display control unit 36) that executes a predetermined function associated with the predetermined gesture may be provided. .

  According to the above configuration, the image display device executes a function associated with a predetermined gesture when the user's input operation is a predetermined gesture. Therefore, the user can execute a predetermined function on the displayed image by performing an input operation using a predetermined gesture. Therefore, for example, if a gesture that imagines a predetermined function is used as a gesture associated with the predetermined function, the user can perform an input operation in an intuitive manner.

  In the image display device according to aspect 3 of the present invention, in the aspect 2, the predetermined function may be a function of executing processing on an image displayed by the image display element. According to said structure, the user can perform input operation for processing the image displayed on an image display element.

  The image display device according to aspect 4 of the present invention is the image display apparatus according to aspect 2 or 3, wherein the processing execution unit is configured to determine the distance (z) between the detection surface of the sensor panel and the indicator (finger h) for one gesture. Different predetermined functions may be executed in accordance with (). According to said structure, an image display apparatus performs a different function according to the distance of the detection surface of a sensor panel, and the indicator which performs input operation. For this reason, the variation of the function to perform can be increased according to the distance of the detection surface of a sensor panel, and the indicator which performs input operation.

  An image display device according to an aspect 5 of the present invention includes the sensor panel according to any one of the aspects 2 to 4, and the predetermined function is associated with the predetermined gesture for each of the sensor panels. May be. According to said structure, a predetermined function will be matched with a predetermined | prescribed gesture for every sensor panel. Therefore, the user can perform an input operation on each sensor panel to execute different functions.

  The image display device according to aspect 6 of the present invention is the parameter for executing the predetermined function associated with the gesture determined by the gesture determination unit according to any one of the aspects 2 to 5. May be changed in conjunction with the distance between the indicator and the detection surface of the sensor panel.

  According to the above configuration, the image display device determines whether or not the movement of the indicator by the user is a predetermined gesture, and indicates the function associated with the gesture determined to be the predetermined gesture. Executes in conjunction with the distance between the panel and the sensing surface of the panel. For example, in a space where the distance from the detection surface of the panel is less than 10 cm, when the user's finger approaches the detection surface of the panel, the image display area is enlarged. In addition, in a space where the distance from the detection surface of the panel is 10 cm or more and 15 cm or less, when the user's finger moves away from the detection surface of the sensor panel, the image display area is reduced. For this reason, the user can adjust the displayed image in accordance with the movement of his input operation. Therefore, the user can perform an input operation in line with intuition.

  A head mounted display according to aspect 7 of the present invention may include the image display device according to any one of aspects 1 to 6 at a position that is in front of the user's eyes when worn. According to said structure, there exists an effect similar to any one aspect of the said aspects 1-6.

  The image display apparatus according to each aspect of the present invention may be realized by a computer. In this case, the image display apparatus is operated on each computer by causing the computer to operate as each unit (software element) included in the image display apparatus. The control program for the image display apparatus realized and the computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

  The present invention can be used for a head mounted display mounted on a user's head.

1 Image display device, 10 HMD (head mounted display)
2 sensor panel, 3 control unit, 4 storage unit, 5 image display element 31 position specifying unit, 32 gesture determination unit, 33 determination unit, 34 application determination unit 35 function determination unit, 36 display control unit (process execution unit), 41 Processing content information I / II detection space (space), z distance, h finger (detection target)

Claims (5)

An image display element whose one side is a display surface;
A sensor panel superimposed on a surface opposite to the display surface of the image display element,
The image display element and the sensor panel are transmissive elements that transmit light,
The image display apparatus, wherein the sensor panel detects a movement of an indicator by a user in a space opposite to the display surface side of the image display element. A gesture determination unit that determines whether or not the movement of the indicator detected by the sensor panel is a predetermined gesture;
The process execution part which performs the predetermined | prescribed function matched with the said predetermined | prescribed gesture when it determines with the motion of the said indicator being the said predetermined | prescribed gesture is provided. Image display device.   The image display apparatus according to claim 2, wherein the predetermined function is a function of executing processing on an image displayed by the image display element. An image display element whose one side is a display surface;
A plurality of sensor panels superimposed on a surface opposite to the display surface of the image display element;
Each of the plurality of sensor panels is an image display device that detects a movement of an indicator by a user in a space opposite to the display surface side of the image display element,
The image display device further includes a gesture determination unit that determines whether or not the movement of the indicator detected by each of the plurality of sensor panels is a predetermined gesture;
A process execution unit that executes a predetermined function associated with the predetermined gesture when the movement of the indicator is determined to be the predetermined gesture;
The predetermined function is, for each of the sensor panel, it images the display device you wherein associated with the above predetermined gesture.   5. A head-mounted display comprising the image display device according to claim 1 at a position in front of a user's eyes when worn.

Images