JP6805524B2 - Input device, input program, and input method - Google Patents

Description

The disclosed techniques relate to input devices, input programs, and input methods.

Conventionally, the detection result status such as gaze detection success, gaze detection failure, blinking, and the same option is selected is judged based on the viewpoint position data, and the detection result information indicating the detection result status is written according to the judgment result. There is known a device that changes the color of the selection frame to notify the user.

Japanese Unexamined Patent Publication No. 11-259226

In an environment where both hands are not used or cannot be used, it is conceivable to perform an input operation using line-of-sight detection as in the prior art. However, when performing various input operations, it is necessary to display a large number of icons on the display device. When performing an input operation using line-of-sight detection, it may not be possible to perform the input operation smoothly when a large number of icons are displayed on the display device.

As one aspect, it is an object of the present invention to smoothly perform an input operation.

In one embodiment, a plurality of virtual surfaces including icons for input operations are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transparently displayed on the three-dimensional display unit. Then, the position of the viewpoint of the operator is acquired. Then, based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the icon included in the virtual surface corresponding to the position of the viewpoint is selected.

As one aspect, it has the effect that the input operation can be performed smoothly.

It is a figure for demonstrating the line of sight of an operator on the display surface including a plurality of icons. It is explanatory drawing for demonstrating 3D display image for operation of this embodiment. It is a block diagram of the input device of 1st Embodiment. It is explanatory drawing for demonstrating an example of a virtual surface. It is explanatory drawing for demonstrating an example of a virtual surface. It is explanatory drawing for demonstrating an example of a virtual surface. It is a figure which shows an example of the binocular Head Mounted Display (HMD). It is a figure which shows an example of the naked eye 3D display. It is a figure which shows an example of a plurality of transparent displays. It is a figure which shows an example of the wearable binocular eye mark recorder. It is a figure which shows the configuration example of a 3D display part and a line-of-sight detection part. It is a figure which shows the configuration example of a 3D display part and a line-of-sight detection part. It is a figure which shows the configuration example of a 3D display part and a line-of-sight detection part. It is a figure which shows the configuration example of a 3D display part and a line-of-sight detection part. It is explanatory drawing for demonstrating the acquisition method of the position of the viewpoint of an operator. It is explanatory drawing for demonstrating the detection of the change of the shape of the mouth. It is explanatory drawing for demonstrating the processing flow of operation input part and determination processing part. It is explanatory drawing for demonstrating the processing flow of operation input part and determination processing part. It is explanatory drawing for demonstrating the processing flow of operation input part and determination processing part. It is explanatory drawing for demonstrating an example of the method of controlling the position of an input display surface. It is explanatory drawing for demonstrating an example of the method of controlling the degree of transmission of a virtual surface. It is explanatory drawing for demonstrating an example of the virtual surface which controlled the size of an icon. It is explanatory drawing for demonstrating an example of the virtual surface which controlled so that the icons do not overlap. It is explanatory drawing for demonstrating an example of each virtual surface controlled so that the icon does not overlap. It is explanatory drawing for demonstrating an example of the virtual surface which controlled the shape and color of an icon. It is a block diagram which shows the schematic structure of the computer which functions as the input device of 1st Embodiment. It is a flowchart which shows an example of the control process of 1st Embodiment. It is explanatory drawing for demonstrating an example of the virtual surface with a tab. It is explanatory drawing for demonstrating the example in the case where the tab is added to the inside of a virtual surface. It is explanatory drawing for demonstrating an example of the virtual surface which controlled the sharpness degree. It is a block diagram of the input device of 4th Embodiment. It is explanatory drawing for demonstrating an example of the method of controlling the position of a virtual surface according to the position of the head of an operator. It is explanatory drawing for demonstrating motion parallax. It is a block diagram which shows the schematic structure of the computer which functions as the input device of 4th Embodiment. It is a flowchart which shows an example of the control process of 4th Embodiment.

Hereinafter, an example of the disclosed technology will be described in detail with reference to the drawings.

<Outline of this embodiment>
First, an outline of the present embodiment will be described before the details of each embodiment are described.

There is an increasing number of situations where smooth input operations are required in various environments such as offices, outdoors, or in public places. For example, if you cannot operate the input device because you are using both hands for other work, if you cannot operate the input device under congestion, or if you cannot use both hands due to physical disability, etc., you cannot use both hands. Input operation may be required in. Therefore, there is a demand for a technology that enables smooth input operation without using both hands.

The present embodiment relates to an input operation technique in an information terminal, and more particularly to an input operation technique in an environment in which both hands are not used or cannot be used.

FIG. 1 shows a diagram for explaining the movement of the line of sight of the operator. As shown in FIG. 1, when a plurality of icons 4 are arranged on one display surface 1A, it becomes difficult to memorize the arrangement of the icons 4. Therefore, when the operator searches for the target icon 4, the operator needs to identify many icons 4, and the path 2A of the movement of the operator's line of sight becomes long. In addition, the residence time of the line of sight other than the target icon 4 becomes long.

On the other hand, as shown in the display surface 1B of FIG. 1, by reducing the number of icons per surface, it becomes easy for the operator to memorize the arrangement of the icons 4. Therefore, the number of icons to be identified is reduced, and the residence time of the line of sight other than the target icon 4 is shortened. In addition, the path 2B of the movement of the operator's line of sight becomes shorter.

Therefore, in the present embodiment, a large number of icons for input operations required for input are three-dimensionally displayed including the depth direction by using the 3D display unit that causes binocular stereoscopic vision. Then, the line-of-sight detection unit that detects the line-of-sight is used to detect the position of the operator's three-dimensional viewpoint and identify the icon selected by the operator.

For example, in the present embodiment, the three-dimensional display image 5 for operation as shown in FIG. 2 is displayed to the operator. As shown in FIG. 2, the three-dimensional display image 5 for operation has virtual display surfaces (hereinafter referred to as virtual surfaces) 5A, 5B, 5C set corresponding to a plurality of positions in the depth direction. , 5D is included. FIG. 2 shows a case where there are four virtual surfaces. The three-dimensional display image 5 for operation arranges a plurality of icons 4 on a plurality of virtual surfaces in a three-dimensional space and displays them semi-transparently. Further, the three-dimensional display image 5 includes an input display surface 5N. The input display surface 5N displays characters and the like input according to the operation of the operator.

Further, in the present embodiment, the number of icons included in one virtual surface is reduced, and the size of each icon is increased.

By arranging the semi-transparently displayed icons 4 in three dimensions as in the present embodiment, the following effects (1) to (3) can be expected.

(1) Improvement of listability In the present embodiment, by arranging the semi-transparently displayed icons in three dimensions, the operator can visually recognize all the icons at once. Further, by arranging the icons arranged in the depth direction in a staggered manner, the overlap of the apparent icons is reduced. In addition, by using the motion parallax described later, the operator can see the icons overlapping in the depth direction separately by the natural movement of moving the head a little. In this way, the listability of the display of many icons is improved, and direct selection and input operation without switching the display page is possible.

(2) Improvement of ease of search In the present embodiment, in addition to the conventional search direction of icons in the vertical and horizontal directions, a degree of freedom in the depth direction is added. That is, two search methods can be considered: a case where the depth direction is searched and then the vertical and horizontal directions are searched, and a case where the vertical and horizontal directions are searched and then the depth direction is searched. As a result, both search methods can be used properly according to the ease of search at each time, depending on the difference in the memory state, preference, search target icon, etc. of each operator.

(3) Improvement of selectivity Since the icon can be selected only by the act of "seeing" in the present embodiment, there is little hesitation or mistake in selection. For example, when selecting an icon requires multiple selection and confirmation operations, when the required selection is not completely memorized, or when an erroneous selection is made in operation, it is necessary to go back and reselect.

For example, a display page switching operation, a continuous multiple selection and confirmation operation such as a context menu or flick input, and a continuous confirmation operation after selection such as character input with the numeric keypad are performed multiple times for icon selection. It is mentioned as a case where the selection and confirmation operation of is necessary.

However, in the present embodiment, since the icon can be selected only by the act of "seeing", there are few hesitations and mistakes in selection, and there is little mistaken selection and reselection as described above.

Further, it is easier for the operator to memorize the arrangement of the icons and to search for the icons when the icons are divided into a plurality of surfaces rather than all the icons are arranged on one surface. Further, when the number of icons per surface is small, the operator can easily memorize the arrangement of the icons and search for them, so that the movement of the line of sight is reduced.

From the viewpoint of movement of the line of sight, the advantage of arranging a plurality of virtual surfaces in the depth direction will be described. For example, comparing the case where a plurality of display surfaces are arranged in three dimensions and the case where the divided display surfaces are arranged on the same plane, when the plurality of display surfaces are arranged in three dimensions, the following (1) And (2) is superior.

(1) When inputting a plurality of icons, the average line-of-sight angle and movement amount per selection can be small.
(2) The range of the line-of-sight angle to be detected can be narrowed.

Further, comparing the case where a plurality of display surfaces are arranged three-dimensionally and the case where the display is switched on the same depth surface, when the display is switched on the same depth surface, the case where the plurality of display surfaces are switched on the same depth surface is used. It is disadvantageous in the following points. That is, for the display surface that is not displayed, it is necessary for the operator to remember on which surface the icon to be input is located, or to actually switch the display to check, which increases the burden on the operator. To do.

As described above, by using the 3D display, the virtual surface including the icon is arranged and displayed in three dimensions, the position of the operator's viewpoint is detected in three dimensions, and the icon to be operated is specified. , It is possible to improve the listability and selectivity of many icons.

Further, in the present embodiment, a plurality of types of input operations input from the operator are discriminated. For example, the input operation is determined based on the change in the shape of the mouth acquired by the camera, the low-volume vocal sound pattern acquired by the pharyngeal microphone, and the like.

This allows you to quickly confirm the input of the selected icon, and for frequently used input operations (for example, kana / kanji conversion, Backspace (BS), cursor movement, etc.), direct operations that do not require icon selection. To enable. Therefore, smoother and more diverse input operations are realized.

Hereinafter, embodiments will be described in detail. In each of the following embodiments, an input device that selects an icon according to the position of the viewpoint of the operator will be described as an example.

<First Embodiment>
As shown in FIG. 3, the input device 50 according to the first embodiment includes a 3D display unit 52, a line-of-sight detection unit 54, an operation input unit 56, and a control unit 58.

The 3D display unit 52 arranges a plurality of virtual surfaces side by side in the depth direction when viewed from the operator, and transmits the plurality of virtual surfaces to display them in three dimensions. Each of the virtual surfaces contains an icon 4 for input operations. The 3D display unit 52 displays a plurality of virtual surfaces in three dimensions by displaying a three-dimensional display image generated by the display processing unit 60 described later. For example, the 3D display unit 52 displays a plurality of virtual surfaces (5A, 5B, 5C, 5D) as shown in FIG. 2 in three dimensions. The virtual surface of the bottom layer may be displayed transparently or may not be transparently displayed. For example, the virtual surface 5D of the lowermost layer shown in FIG. 2 may be displayed transparently or may not be transparently displayed.

An example of an icon for input operation included in the virtual surface is shown in FIGS. 4A and 4B. In the plurality of virtual surfaces 5A, 5B, 5C, 5D, and 5E shown in FIG. 4A, the hiragana and the icon 4 are mainly associated with each other. Further, when the icon "switch" included in the virtual surface is selected, the virtual surfaces are switched to the plurality of virtual surfaces 5A, 5B, 5C, 5D, and 5E shown in FIG. 4B. In the plurality of virtual surfaces 5A, 5B, 5C, 5D, and 5E, letters and symbols are associated with the icon 4. Similarly, when the icon "switch" included in the virtual surfaces 5A, 5B, 5C, 5D, and 5E is selected, the hiragana is switched to katakana.

Further, another example of the icon 4 for input operation is shown in FIG. In the plurality of virtual surfaces 5A, 5B, 5C, and 5D shown in FIG. 5, the Roman characters and the icon 4 are mainly associated with each other. By selecting the icon 4 shown in FIG. 5, Roman character input is performed.

The 3D display unit 52 is realized by, for example, a binocular Head Mounted Display (HMD) display type display or a naked eye stereoscopic display type display.

FIG. 6 shows an example of the binocular HMD display type display 52A. The binocular HMD display type display includes a non-transmissive type HMD and a transmissive type HMD. As for the non-transparent HMD, when the operator 6 wears the non-transparent HMD, the outside cannot be seen. On the other hand, with respect to the transmissive HMD, even if the operator 6 wears the transmissive HMD, the outside can be seen.

The non-transparent HMD is used for character input and operation input for persons with physical disabilities in the limbs and for VR games (for example, both hands are used as a controller for game operation).

The transmissive HMD is used for character input and operation input in an Augmented Reality (AR) work support system for workers such as maintenance, inspection, or assembly (for example, both hands are used for tools, etc.). Used. In addition, the transparent HMD is used for character input and operation input for social network communication for visual content (in the case of an HMD with a camera) and AR information display when watching sports at a stadium.

Examples of the autostereoscopic display include a lenticular lens type 3D display, a hologram type stereoscopic image display device, and a method using a plurality of transparent displays.

FIG. 7 shows an example of the lenticular lens type 3D display 52B. As shown in FIG. 7, the lenticular lens type 3D display 52B is provided with a lens 52D on the display 52C, and is configured to cause binocular stereoscopic vision of the operator.

Further, FIG. 8 shows an example of a method using a plurality of transparent displays 52E. The plurality of transparent displays 52E are configured to cause binocular stereoscopic vision of the operator.

The autostereoscopic display is used for character input and operation input for persons with physical disabilities in the limbs and for TV games (for example, both hands are used as a controller for game operation). In addition, the autostereoscopic display can be used for interactive operations such as digital signage and tourist information display boards, as well as for operations such as displaying and inputting information on displayed objects at specific facilities (for example, entertainment). Used.

The line-of-sight detection unit 54 acquires information representing the line-of-sight of the operator. For example, the line-of-sight detection unit 54 acquires an image of both eyes of the operator as information representing the line of sight of the operator.

When a binocular HMD display type display is used as the 3D display unit 52, the line-of-sight detection unit 54 is provided in the 3D display unit 52.

Further, when a naked-eye stereoscopic display is used as the 3D display unit 52, there are a method of being fixed to the 3D display unit 52 and a method of using a wearable binocular eye mark recorder.

For example, when the method in which the line-of-sight detection unit 54 is fixed to the 3D display unit 52 is used, information on the line-of-sight of the operator is acquired by using a movable zoom camera. When the input device 50 is provided with a panoramic shooting camera, the panoramic shooting camera may be a high-resolution camera, and the panoramic shooting camera and the line-of-sight detection camera may be combined.

FIG. 9 shows an example of a wearable binocular eye mark recorder 54A. The binocular eye mark recorder 54A is provided with a sensor 54B for the right eye and a sensor 54C for the left eye, and an captured image of the corneal reflex of both eyes of the operator is acquired.

10 to 13 show a configuration example of the 3D display unit 52 and the line-of-sight detection unit 54.

10 and 11 are examples in which the transmissive HMD 52A is used as the 3D display unit 52.

As shown in FIG. 10, a virtual surface is displayed by the transparent HMD 52A, and a work comment or the like is input by the operator 6. Since the input device 50 of the present embodiment supports the operator, it is possible to input a comment while working with both hands.

Further, as shown in FIG. 11, the virtual surface is displayed by the transmissive HMD 52A, and the value of the meter 23 and the like are input by the operator 6. The configuration example shown in FIG. 11 is used as a terminal for maintenance and inspection work such as factory facilities and airframe maintenance.

12 and 13 are examples in the case where the autostereoscopic display 52B is used as the 3D display unit 52.

As shown in FIG. 12, the autostereoscopic display 52B as a guidance display board functions as a 3D display unit 52, and the operator 6 visually observes the guidance display board to acquire additional information or input a question. You can do things. The line of sight of the operator is acquired by the camera 54D.

Further, as shown in FIG. 13, the autostereoscopic display 52B as a guidance display board functions as a 3D display unit 52, and can be used as a digital signage, a facility guidance terminal, or the like. The line of sight of the operator is acquired by the camera 54D.

In the present embodiment, a case where the 3D display unit 52 is realized by the autostereoscopic display 52B will be described as an example. Further, in the present embodiment, a case where the line-of-sight detection unit 54 is realized by the camera 54D and fixed to the 3D display unit 52 will be described as an example.

The operation input unit 56 acquires information representing the manifestation of intention of the operator. For example, the operation input unit 56 acquires an image representing the operator or a sound emitted by the operator. The operation input unit 56 is an example of an intention acquisition unit of the disclosed technology.

The operation input unit 56 is realized by a camera for capturing the face of the operator, a pharyngeal microphone, an exhalation switch, or the like.

When the operation input unit 56 is realized by the camera for face photographing, the operation input unit 56 acquires an image representing the face of the operator.

When the operation input unit 56 is realized by the pharyngeal microphone, the operation input unit 56 acquires the sound emitted by the operator. The pharyngeal microphone captures the simple sound emitted by the operator.

By using a pharyngeal microphone and a simple sound, operation input at a low volume can be realized without making a clear utterance such as when the mouth is closed. For example, in a place where the environmental noise is louder than the noise environment of an office, the operation can be performed without disturbing the surroundings.

When the operation input unit 56 is realized by the exhalation switch, the operation input unit 56 acquires the exhalation emitted by the operator.

The operation input unit 56 may be realized by a plurality of button switches. For example, it may be realized by a wearable input device such as a normal button operated by a hand, a normal button operated by a foot, or a glove type that acquires a finger pulling motion.

In the present embodiment, a case where the operation input unit 56 is realized by a camera for facial photography will be described as an example.

The control unit 58 displays the 3D display unit 52 based on the image representing the operator's face acquired by the line-of-sight detection unit 54 and the information representing the operator's manifestation of intention acquired by the operation input unit 56. Control. As shown in FIG. 3, the control unit 58 includes a display processing unit 60, a viewpoint acquisition unit 62, a determination processing unit 64, a display data processing unit 66, and an input processing unit 68. The display data processing unit 66 is an example of a selection unit of the disclosed technology. The determination processing unit 64 is an example of the determination unit of the disclosed technology.

The display processing unit 60 generates a three-dimensional display image for operation in response to the control signals output by the display data processing unit 66 and the input processing unit 68, which will be described later. Then, the display processing unit 60 outputs the generated three-dimensional display image to the 3D display unit 52.

In the three-dimensional display image for operation, a plurality of virtual surfaces are arranged side by side in the depth direction when viewed from the operator. In addition, a plurality of virtual surfaces are transparently displayed.

The viewpoint acquisition unit 62 acquires the position of the operator's viewpoint in the three-dimensional space based on the image representing both eyes of the operator acquired by the line-of-sight detection unit 54.

FIG. 14 shows a diagram for explaining a method of acquiring the position of the viewpoint of the operator. As shown in FIG. 14, the unit vectors on the x-axis, y-axis, and z-axis of the orthogonal triaxial coordinate system with the center position of both eyes of the operator as the origin are i, j, and k, respectively, and the line-of-sight direction of the right eye. Assuming that the unit vector indicating is u = (l, m, n), the straight line indicating the line-of-sight direction of the right eye is shown by the following equation (1). Note that (x0, y0, z0) is the position of the operator's viewpoint, and 2E is the distance between the operator's eyes.

Since i, j, and k in the above equation (1) are linearly independent,

Will be. Let l, m, and n be the direction cosine of a straight line indicating the line-of-sight direction. Similarly, if the unit vector indicating the line-of-sight direction of the left eye is v = (p, q, r), the straight line indicating the line-of-sight direction of the left eye is

Will be. From the above equations (2) and (3), the following equation (4) is established.

Then, assuming that the coefficient matrix is H and the matrix of the constant term on the right side is C, the equation (5) shown below is obtained.

In addition, T in the above equation (5) represents a transposed matrix.

Therefore, the viewpoint acquisition unit 62 calculates the position (x0, y0, z0) of the viewpoint of the operator by calculating the above equation (6).

The coordinate system is not limited to the one with the center position of both eyes as the origin, but when it is based on a fixed display device or a point, or when the center of both eyes at the start of operation is used to change the angle of the head position. It may be reflected. In the present embodiment, the position of the viewpoint calculated according to the above equation (6) is converted into the position of the viewpoint of the world coordinate system, and the position of the viewpoint of the world coordinate system is used in the process described later.

The determination processing unit 64 determines the type of operation of the operator based on the information regarding the manifestation of intention of the operator acquired by the operation input unit 56. Examples of the determination content of the operation type include the following (1) to (5).

(1) Confirm input of the selected icon (2) Kana-Kanji conversion (3) Delete one character (Back Space)
(4) Cursor movement (up, down, left, right)
(5) Context menu display

When the operation input unit 56 is realized by the camera for facial photography, for example, the determination processing unit 64 changes the shape of the mouth as shown in FIG. 15 based on the image acquired by the operation input unit 56. To detect. Then, the determination processing unit 64 determines the type of operation of the operator based on the type of operation associated in advance according to the shape of the mouth.

For example, the confirmation of the input of the selected icon is associated with the shape of "A", the conversion of the input is associated with the shape of "I", and the deletion of one character (Back Space) is associated with the tongue out. There is. The camera for facial photography may be used in combination with the camera for detecting the line of sight, or another camera may be prepared.

FIG. 16 shows an outline of the processing flow when the operation input unit 56A is realized by a camera for facial photography. As shown in FIG. 16, an image showing the operator's face is acquired by the face photographing camera of the operation input unit 56A, and each process of face detection, mouth part extraction, and mouth shape identification is performed by the determination processing unit 64A. Is done.

When the operation input unit 56 is realized by the pharyngeal microphone, the determination processing unit 64 determines the operation of the operator based on the simple sound acquired by the operation input unit 56.

For example, as shown in (1) to (5) below, the determination processing unit 64 is based on the information in which the simple sound and the type of operation are associated in advance and the simple sound emitted by the operator. Determine the type of operation of the operator.

Operation judgment processing (judgment assignment example)
(1) Confirmation of input of selected icon: Bass and short sound "n""
(2) Delete one character (Back Space): High and short sounds "Mu"
(3) Conversion: Bass and medium-long sound "Vun""
(4) Cursor movement left ←: Bass and long sound "Voo"
(5) Cursor movement right →: High-pitched and long-pitched sound "Moo"

FIG. 17 shows an outline of the processing flow when the operation input unit 56 is realized by the pharyngeal microphone. As shown in FIG. 17, the sound emitted by the operator is acquired by the pharyngeal microphone of the operation input unit 56B, and the sound input determination for determining the input sound is performed by the determination processing unit 64B, and the determined sound is associated with the determined sound. Each operation sound identification process for determining the type of operation performed is performed.

When the operation input unit 56 is realized by the exhalation switch, the determination processing unit 64 determines the operation of the operator based on the exhaled air acquired by the operation input unit 56.

For example, the determination processing unit 64 determines the operation of the operator according to the exhalation and the operation issued by the operator, which are assigned as shown in (1) to (5) below.

Operation judgment processing (judgment assignment example)
(1) Confirmation of input of selected icon: Vomiting (short) "F"
(2) Delete one character (Back Space): Suck (long) "Sue"
(3) Conversion: Suck (short) "Su"
(4) Move cursor left ←: Vomiting (long) "Fu"
(5) Move the cursor to the right →: Vomiting (twice in a row) "Fufu"

FIG. 18 shows an outline of the processing flow when the operation input unit 56 is realized by the exhalation switch. As shown in FIG. 18, the exhaled air emitted by the operator is converted into a voltage value and acquired by the piezoelectric element constituting the exhaled air switch of the operation input unit 56C. Then, the determination processing unit 64C performs a pressurization pattern identification process for determining the type of operation associated with the exhaled breath indicated by the pressurization pattern based on the acquired voltage value.

In the present embodiment, the determination processing unit 64 detects the shape of the operator's mouth based on the image acquired by the operation input unit 56A, as shown in FIG. Then, the determination processing unit 64A determines the type of operation of the operator based on the shape of the operator's mouth, as shown in FIG.

The display data processing unit 66 is a three-dimensional display generated by the display processing unit 60 based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62 and the type of operation determined by the determination processing unit 64. Control the image.

For example, the display data processing unit 66 selects the icon 4 corresponding to the position of the operator's viewpoint based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62. Then, the display data processing unit 66 outputs a control signal to the display processing unit 60 so as to generate a three-dimensional display image for operation indicating that the selected icon 4 is in the selected state.

Further, whether or not the display data processing unit 66 confirms the selection of the icon 4 based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62 and the type of operation determined by the determination processing unit 64. Is determined. For example, the display data processing unit 66 confirms the selection of the icon 4 when the operation type is “confirmed”.

Then, the display data processing unit 66 outputs a control signal to the display processing unit 60 so as to generate a three-dimensional display image in which the icon 4 is in a fixed state. Further, the display data processing unit 66 outputs key information (for example, the character "A" or the like) corresponding to the icon 4 whose selection has been confirmed to the input processing unit 68.

On the other hand, when the operation type is other than "confirmation" (for example, "input conversion", "single character deletion", etc.), the display data processing unit 66 provides information according to the operation type. Output to the input processing unit 68.

Further, the display data processing unit 66 generates the display processing unit 60 so as to generate a three-dimensional display image in which the degree of transparency of the virtual surface is changed according to the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62. Output the control signal to.

Specifically, the display data processing unit 66 determines the degree of transparency of another virtual surface different from the virtual surface corresponding to the position of the viewpoint based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62. Control to be higher.

For example, the display data processing unit 66 controls so that the transparency of the virtual surface in front of the corresponding virtual surface is made higher at the position x0 in the depth direction of the position of the viewpoint of the operator. Then, the display data processing unit 66 controls such that when the position of the viewpoint of the operator returns to the front, the transparency of the virtual surface in front is restored. Note that 100% transparency of the virtual surface is controlled so that only the frame of the virtual surface is not 100% transparent because the transparently displayed layer cannot be "seen" again.

Further, for example, the display data processing unit 66 controls the background surface (for example, a window) of the icon 4 on the virtual surface in front of the virtual surface at the viewpoint position so that the transparency is 100%. By changing the transparency of the icon 4 main body and the background surface individually, it becomes easy to gaze at the display surface on the back side and to select the icon 4 on the front side.

Further, for example, the display data processing unit 66 determines the transparency of the display (for example, character engraving) in front of the corresponding virtual surface at the position x0 in the depth direction of the viewpoint position or in the icon 4 of the other virtual surface. Control to make it higher.

Further, the display data processing unit 66 generates the display processing unit 60 so as to generate a three-dimensional display image in which the degree of transparency of the virtual surface is changed according to the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62. Output the control signal to.

Further, the display data processing unit 66 is based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62, and the position of the operator's viewpoint in the depth direction x0 corresponds to the position of the corresponding virtual surface. Control the position of the input display surface. For example, as shown in FIG. 19A, the display data processing unit 66 controls to move the position of the input display surface 5N for character input in the depth direction to the depth position of the virtual surface at the viewpoint position. As shown in FIG. 19A, when the position of the viewpoint of the operator corresponds to the icon 4 included in the virtual surface 5A, the display data processing unit 66 moves the position of the viewpoint to the depth position of the virtual surface 5A corresponding to the position of the viewpoint. Control so that the input display surface 5N is displayed. Further, when the position of the viewpoint of the operator moves from the virtual surface 5A to the icon 4 of the virtual surface 5B, the display data processing unit 66 controls so that the input display surface 5N is displayed at the depth position of the virtual surface 5B. .. By aligning the virtual surface of the icon 4 in the selected state with the position of the input display surface in the depth direction, unnecessary movement of the line of sight in the depth direction is prevented.

FIG. 19B shows an explanatory diagram for explaining an example of processing by the display data processing unit 66.

First, as shown in scene 7A of FIG. 19B, the position of the viewpoint of the operator is acquired by the viewpoint acquisition unit 62. In scene 7A, it is detected that the operator's viewpoint is located on the foremost virtual surface 5A as seen from the operator.

Then, as shown in scene 7A, the icon 4 corresponding to "a" is selected from the viewpoint of the operator, and the character "a" of the icon 4 is input by the confirmation operation of the input of the operator.

In this case, as shown in scene 7A of FIG. 19B, after a three-dimensional display image indicating that the icon 4 corresponding to “a” is in the selected state is displayed to the operator, it corresponds to “a”. A three-dimensional display image for operation indicating that the icon 4 is in the confirmed state is displayed to the operator. Further, as described in FIG. 19A, the input display surface 5N is displayed at the same depth position as the virtual surface 5A.

Next, in scene 7B of FIG. 19B, when the viewpoint acquisition unit 62 detects that the position of the operator's viewpoint has moved from the front to the virtual surface 5B of the second layer, the display data processing unit 66 has the first layer. It is controlled to increase the degree of transmission of the virtual surface 5A of the eye. Further, as described in FIG. 19A, the display data processing unit 66 adjusts the display position of the input display surface 5N to the position of the viewpoint of the operator, and sets the virtual surface of the first layer to the virtual surface of the second layer. It is controlled to move to the depth position of 5B.

In the example of scene 7B, when the position of the viewpoint of the operator moves to the virtual surface 5B of the second layer, the display data processing unit 66 sets the degree of transparency of the background of the virtual surface 5A of the first layer to 100%, 1 The degree of transparency of the characters on the icon 4 of the virtual surface 5A of the layer is controlled to be increased.

As shown in scene 7B, a three-dimensional display image for operation indicating that the icon 4 corresponding to "k" is in the selected state is displayed to the operator. In FIG. 19A, the diagonal line shown in the icon indicates the selected state, and the dot shown in the icon indicates the confirmed state.

Next, as shown in scene 7C, the position of the operator's viewpoint is on the second layer without performing the operation of confirming the input of the operator with respect to the icon 4 corresponding to the "k" selected in scene 7B. Move on the virtual surface 5B and icon 4 "o" is selected. Further, the display position of the input display surface 5N is controlled so as to be displayed at the depth position of the virtual surface 5B. Then, a three-dimensional display image for operation indicating that "o" is in the selected state is displayed. Then, the character "o" of the icon 4 is reflected on the input display surface 5N by the confirmation operation of the input of the operator.

Further, the display data processing unit 66 reduces the degree of transparency of the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62. You may control it.

Further, in the display data processing unit 66, the size of each of the icons 4 included in the plurality of virtual surfaces becomes larger as the position of the virtual surface is located in the depth direction when viewed from the operator. You may control the display of the virtual surface. Further, in the display data processing unit 66, each of the icons 4 included in the plurality of virtual surfaces is spaced from the adjacent icons 4 so that the position of the virtual surface is located in the depth direction when viewed from the operator. The display of the virtual surface may be controlled so as to be wide.

FIG. 20 shows an example of a virtual surface controlled so that the size of the icon 4 becomes larger and the distance from the icon 4 arranged next to the icon 4 becomes wider as the icon 4 is located in the depth direction. As shown in FIG. 20, the icon 4 of the virtual surface 5A of the first layer, the icon 4 of the virtual surface 5B of the second layer, and the icon 4 of the virtual surface 5C of the third layer are large according to the layer of the virtual surface. Is controlled.

By controlling the virtual surface as shown in FIG. 20, it becomes easy to detect the icon 4 at the viewpoint in the depth direction. As the position of the operator's viewpoint goes in the depth direction, the size and spacing of the icons 4 are widened, so that the moving angle of the line of sight between the icons 4 at a distant position becomes large. That is, the amount of movement of the line of sight on the virtual surface in the foreground can be made equal to the amount of movement of the line of sight on the virtual surface located far in the depth direction, and the position of the viewpoint can be easily detected. ..

Further, the display data processing unit 66 controls the display of the virtual surface so that at least a part of the icon 4 is arranged so as not to overlap the icon 4 included in another virtual surface different from the virtual surface including the icon 4. You may.

FIG. 21 shows an example of virtual surfaces arranged so that the icons 4 do not overlap. FIG. 21 is an example of the apparent arrangement of the icons 4 when a plurality of virtual surfaces are viewed from the front. In FIG. 21, the notation in parentheses of the icon 4 corresponds to each virtual surface (5A to 5E), for example, the icon 4 (5A) represents the icon 4 included in the virtual surface 5A. As shown in FIG. 21, the icons 4 on each virtual surface are arranged so as not to overlap so much in the line of sight of the operator.

FIG. 22 shows an example of virtual surfaces of each layer arranged so that the icons 4 do not overlap. As shown in FIG. 22, the display size of each virtual surface is different, the icon 4 of the virtual surface in front of the operator is displayed small, and the icon 4 of the virtual surface in the back is displayed large when viewed from the operator. Is controlled.

Further, the icons 4 may be classified and arranged on a plurality of virtual surfaces according to the type of the icons 4. For example, the virtual surface including the hiragana icon 4 is displayed on the first layer, the virtual surface including the alphabetic icon 4 is displayed on the second layer, and the virtual surface including the numbers is displayed on the third layer. You may do so.

Further, each shape or color of the icon 4 may be determined according to the type of information input by selecting the icon 4.

Further, each shape or color of the icon 4 may be determined according to the position of the icon 4 in the virtual surface.

FIG. 23 shows an example in which each shape and color of the icon 4 is determined according to the type of information input by selecting the icon 4, and is determined according to the position of the icon 4 in the virtual surface. Is shown.

Due to human visual perception characteristics, shapes and colors are processed independently. Also, shapes and colors are recognized faster than letters. By utilizing this characteristic and using the icon 4 having a different shape and color for each character, the association memory of the arrangement between the operator's character and the icon 4 is promoted, and the icon search time for the selection operation can be shortened. become.

Further, it is known that even if the same color and shape are used on different display screens, the influence on identification is small. By utilizing this characteristic, even if the same color and shape as other display surfaces are used on the display surfaces different in the depth direction, the influence on the icon identification can be small.

Therefore, the identification of the icon 4 by the operator is improved.

The input processing unit 68 acquires the key information corresponding to the icon 4 whose selection has been confirmed by the display data processing unit 66. Then, the input processing unit 68 outputs a control signal to the display processing unit 60 so that the key corresponding to the icon 4 whose selection is confirmed is displayed on the input display surface.

Further, the input processing unit 68 outputs a control signal to the display processing unit 60 so as to reflect the state corresponding to the type of operation output by the display data processing unit 66 on the input display surface.

The control unit 58 of the input device 50 can be realized by, for example, the computer 80 shown in FIG. The computer 80 includes a CPU 81, a memory 82 as a temporary storage area, and a non-volatile storage unit 83. Further, the computer 80 includes an input / output I / F 84, a read / write (R / W) unit 85 that controls reading and writing of data to the recording medium 89, and a network I / F 86 connected to a network such as the Internet. To be equipped. The CPU 81, the memory 82, the storage unit 83, the input / output I / F 84, the R / W unit 85, and the network I / F 86 are connected to each other via the bus 87.

The storage unit 83 can be realized by a Hard Disk Drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 83 as a storage medium stores an input program 90 for causing the computer 80 to function as a control unit 58 of the input device 50. The input program 90 includes a display processing process 92, a viewpoint acquisition processing process 93, a determination processing process 94, a display data processing process 95, and an input processing process 96.

The CPU 81 reads the input program 90 from the storage unit 83, expands it in the memory 82, and sequentially executes the processes of each of the input programs 90.

The CPU 81 operates as the display processing unit 60 shown in FIG. 3 by executing the display processing process 92. Further, the CPU 81 operates as the viewpoint acquisition unit 62 shown in FIG. 3 by executing the viewpoint acquisition processing process 93. Further, the CPU 81 operates as the determination processing unit 64 shown in FIG. 3 by executing the determination processing process 94. Further, the CPU 81 operates as the display data processing unit 66 shown in FIG. 3 by executing the display data processing process 95. Further, the CPU 81 operates as the input processing unit 68 shown in FIG. 3 by executing the input processing process 96.

In this way, the computer 80 that executes the input program 90 functions as the input device 50.

The function realized by the input program 90 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an Application Specific Integrated Circuit (ASIC) or the like.

Next, the operation of the input device 50 according to the present embodiment will be described with reference to FIG. 25.

FIG. 25 shows an example of a flow of control processing executed by the control unit 58 of the input device 50.

The 3D display unit 52 of the input device 50 displays a three-dimensional display image for operation, the line-of-sight detection unit 54 sequentially acquires images of both eyes of the operator, and the operation input unit 56 sequentially acquires the face image of the operator. When this is done, the control processing routine shown in FIG. 25 is executed.

<Control processing routine>

In step S10, the viewpoint acquisition unit 62 acquires an image representing both eyes of the operator acquired by the line-of-sight detection unit 54.

In step S12, the viewpoint acquisition unit 62 detects the operator's line-of-sight vector according to the above equations (1) to (3) based on the image representing both eyes of the operator acquired in step S10.

In step S14, the viewpoint acquisition unit 62 calculates the position of the operator's viewpoint according to the above equations (4) to (6) based on the line-of-sight vector detected in step S12.

In step S16, the viewpoint acquisition unit 62 calculates the position of the viewpoint in the world coordinate system based on the position of the operator's viewpoint calculated in step S14.

In step S18, the display data processing unit 66 determines whether or not the operable icon 4 is displayed at a position corresponding to the position of the viewpoint based on the position of the viewpoint of the operator calculated in step S16. judge. If the operable icon 4 is displayed at the position corresponding to the position of the viewpoint, the process proceeds to step S20. On the other hand, if the operable icon 4 is not displayed at the position corresponding to the position of the viewpoint, the process proceeds to step S22.

In step S20, the display data processing unit 66 selects the icon 4 corresponding to the position of the operator's viewpoint calculated in step S16. Then, the display data processing unit 66 outputs a control signal to the display processing unit 60 so as to generate a three-dimensional display image for operation indicating that the selected icon 4 is in the selected state. The display processing unit 60 generates a three-dimensional display image for operation according to the output control signal. Then, the display processing unit 60 outputs the generated three-dimensional display image for operation to the 3D display unit 52. Then, the 3D display unit 52 displays the three-dimensional display image for operation generated by the display processing unit 60 to the operator.

In step S22, the display data processing unit 66 changes the degree of transparency of the virtual surface based on the position of the viewpoint of the operator obtained in step S16. Further, the display data processing unit 66 performs display processing so as to generate a three-dimensional display image for operation in which the degree of transparency of the virtual surface is changed according to the position of the viewpoint of the operator obtained in step S16. A control signal is output to unit 60. The display processing unit 60 generates a three-dimensional display image for operation according to the output control signal. Then, the display processing unit 60 outputs the generated three-dimensional display image for operation to the 3D display unit 52. The 3D display unit 52 displays the three-dimensional display image for operation generated by the display processing unit 60 to the operator.

In step S24, the determination processing unit 64 acquires the image of the operator acquired by the operation input unit 56.

In step S26, the determination processing unit 64 determines the operation of the operator based on the image acquired in step S24. If the type of operation of the operator is determined, the process proceeds to step S28. On the other hand, if it is not determined by any of the operation types, it is determined that there is no operation by the operator, and the process returns to step S10.

In step S28, the determination processing unit 64 determines whether or not the type of operation is an input confirmation operation based on the determination result in step S26. If the type of operation is an operation for confirming the input, the process proceeds to step S30. On the other hand, if the type of operation is not the operation of confirming the input, the process proceeds to step S34.

In step S30, the display data processing unit 66 determines whether or not there is the selected icon selected in step S20. If there is an icon in the selected state, the process proceeds to step S32. On the other hand, if there is no selected icon 4, the process returns to step S10.

In step S32, the display data processing unit 66 generates a three-dimensional display image in which the display of the icon 4 in the selected state corresponds to the position of the viewpoint of the operator acquired in step S16. The control signal is output to the display processing unit 60. Further, the display data processing unit 66 outputs the key information corresponding to the icon 4 whose selection is confirmed to the input processing unit 68.

In step S34, the input processing unit 68 acquires the key information output in step S32. Then, the input processing unit 68 outputs a control signal to the display processing unit 60 so that the key corresponding to the icon 4 whose selection is confirmed is displayed on the input display. Further, the input processing unit 68 outputs a control signal to the display processing unit 60 so as to reflect the state corresponding to the type of operation output in step S28 on the input display surface.

In step S36, the display processing unit 60 generates a three-dimensional display image for operation in response to the control signal output in step S34. Then, the display processing unit 60 outputs the generated display image to the 3D display unit 52.

The 3D display unit 52 displays the generated three-dimensional display image for operation to the operator.

As described above, in the input device 50 of the present embodiment, a plurality of virtual surfaces including the icon 4 for input operation are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transmitted through the plurality of virtual surfaces. It is displayed on the three-dimensional display unit. Then, the input device 50 selects the icon 4 corresponding to the position of the viewpoint based on the position of the viewpoint of the operator. As a result, the operator can smoothly perform the input operation.

Further, since the input device 50 of the present embodiment arranges a large number of icons 4 corresponding to various input operations so as to improve the listability and selectivity, the operator smoothly selects and operates the icons 4. can do.

<Second Embodiment>
Next, the input device according to the second embodiment will be described. In the second embodiment, the virtual surface has a tabbed area. Since the configuration of the input device according to the second embodiment is the same as the configuration of the input device 50 according to the first embodiment shown in FIG. 3, different parts will be described using the same reference numerals.

In the second embodiment, a tab area is provided for each virtual surface, and the operator directs the line of sight to the tab area, which facilitates the operator's gaze in the depth direction of the line of sight. The tab area of each virtual surface is provided in a portion that does not overlap with other virtual surfaces.

FIG. 26 shows an example of the virtual surface of the second embodiment. As shown in FIG. 26, each of the virtual surfaces has a tab area 34 at a position where the display does not overlap with another virtual surface different from the virtual surface.

The display data processing unit 66 of the second embodiment is different from the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62. Controls the degree of transparency of the virtual surface. For example, the display data processing unit 66 controls so that when the tab area is watched by the operator, the degree of transmission of the virtual surface in front of the virtual surface having the tab area 34 is increased.

The display data processing unit 66 may control the degree of transparency of the tab area 34 so as not to change even when the degree of transparency of the virtual surface is changed. This makes it easy to return the viewpoint to the virtual surface in front of the virtual surface to which the viewpoint is aligned.

The display data processing unit 66 reduces the degree of transparency of the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62. It may be controlled to.

Then, the display data processing unit 66 generates a three-dimensional display image for operation in which the degree of transparency of the virtual surface is changed according to the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62. A control signal is output to the display processing unit 60.

An example of the processing of the display data processing unit 66 of the second embodiment will be described with reference to FIG. 26.

First, as shown in scene 30 of FIG. 26, the position of the operator's viewpoint is acquired by the viewpoint acquisition unit 62. In scene 30, it is detected that the operator's viewpoint is located on the foremost virtual surface 5A as seen from the operator.

Then, as shown in the scene 30, the icon 4 corresponding to "a" is selected from the viewpoint of the operator, and the character "a" of the selected icon 4 is input by the confirmation operation of the input of the operator.

In this case, as shown in the scene 30 of FIG. 26, a three-dimensional display image for operation indicating that the icon 4 corresponding to “a” is in the selected state is displayed to the operator. After that, a three-dimensional display image for operation indicating that the icon 4 corresponding to "a" is in the confirmed state is displayed to the operator. Further, the input display surface 5N is displayed at the same depth position as the virtual surface 5A.

Next, in scene 31 of FIG. 26, when the viewpoint acquisition unit 62 detects that the position of the operator's viewpoint has moved from the front to the virtual surface 5B of the second layer, the display data processing unit 66 moves to the first layer. It is controlled to increase the degree of transmission of the virtual surface 5A of the eye. Further, the display data processing unit 66 moves the display position of the input display surface 5N from the virtual surface 5A of the first layer to the depth position of the virtual surface 5B of the second layer according to the position of the viewpoint of the operator. Control.

In the example of scene 31, when the position of the viewpoint of the operator moves to the virtual surface 5B of the second layer, the display data processing unit 66 sets the degree of transparency of the background of the virtual surface 5A of the first layer to 100%, 1 The degree of transparency of the characters on the icon 4 of the virtual surface 5A of the layer is controlled to be increased. The degree of transparency of the tab area 34 is not changed.

As shown in the scene 31, a three-dimensional display image for operation indicating that the icon 4 of "k" is in the selected state is displayed to the operator.

Next, as shown in scene 32, the position of the operator's viewpoint is on the virtual surface 5B of the second layer without performing the operation of confirming the input of the operator for the icon 4 "k" selected in the scene 31. Move with, and the "o" icon 4 is selected. Then, it becomes a three-dimensional display image for operation indicating that "o" is in the selected state. Then, the character "o" of the selected icon 4 is reflected on the input display surface 5N by the confirmation operation of the input of the operator.

Next, as shown in scene 33, the position of the operator's viewpoint moves from the virtual surface 5B of the second layer to the tab area 34 of the virtual surface 5D of the fourth layer.

Then, as shown in the scene 35, the transparency of the virtual surface 5B of the second layer and the virtual surface 5C of the third layer is controlled to be high as in the virtual surface 5A of the first layer. Further, the input display surface 5N is also moved to the depth position of the virtual surface 5D of the fourth layer.

As shown in FIG. 27, the tab area 34 may be arranged at the center of the virtual surface. By arranging the tab area 34 in the central portion, the vertical and horizontal movement of the line of sight when viewing the tab area 34 is reduced as compared with the case where the tab area 34 is arranged outside the background surface of the icon 4, and the movement in the depth direction is reduced. It enables quick gaze movement.

Next, the operation of the input device according to the second embodiment will be described as being different from the first embodiment.

In the first embodiment, in the control processing routine of FIG. 25, after the icon corresponding to the position of the viewpoint of the operator is selected in step S20, the degree of transparency of the virtual surface is controlled in step S22. Will be. However, the second embodiment is different from the processing flow of the first embodiment in that the degree of transmission of the virtual surface different from the virtual surface including the tab area corresponding to the position of the viewpoint of the operator is controlled.

As described above, in the input device according to the second embodiment, each of the virtual surfaces has a tab area at a position where the display does not overlap with other virtual surfaces different from the virtual surface. Then, the input device controls the degree of transmission of another virtual surface different from the virtual surface having the tab area corresponding to the position of the viewpoint based on the position of the viewpoint of the operator. As a result, the input operation can be performed smoothly.

Further, since the operator only needs to direct the line of sight to the tab area, it becomes easy to gaze in the depth direction. Therefore, the input operation can be performed smoothly.

<Third Embodiment>
Next, the input device according to the third embodiment will be described. In the third embodiment, the degree of sharpness of the display of the virtual surface is controlled. Since the configuration of the input device according to the third embodiment is the same as the configuration of the input device 50 according to the first or second embodiment shown in FIG. 3, different parts will be described using the same reference numerals.

The display data processing unit 66 of the third embodiment controls the sharpness of the display of the virtual surface based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit 62.

For example, the display data processing unit 66 increases the sharpness of the display of the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62. To control. Further, the display data processing unit 66 displays a virtual surface different from the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position based on the position of the operator's viewpoint acquired by the viewpoint acquisition unit 62. It is controlled so that the sharpness of is low.

In the third embodiment, the sharpness of the virtual surface other than the virtual surface corresponding to the position x0 in the depth direction of the position of the viewpoint of the operator is lowered, and the virtual surface corresponding to the viewpoint is easily gazed. As a method of changing the sharpness, for example, unsharpness filtering is used.

FIG. 28 shows a diagram for explaining a change in the display sharpness of the virtual surface. The example of FIG. 28 is an example in which the sharpness of each display of the other virtual surface is changed according to the distance between the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position and each of the other virtual surfaces. Is. Further, the example of FIG. 28 is an example in which a Gaussian filter is used as a filter for blurring the display of the virtual surface.

As shown in scene 37 of FIG. 28, since the position of the operator's viewpoint corresponds to the virtual surface 5A of the first layer, the virtual surfaces 5B to 5D of the second, third, and fourth layers are clear. The degree is controlled to be low. Further, as shown in the lower part of the scene 37 in FIG. 28, the sharpness is controlled to be lower as the virtual surface is farther from the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position.

Next, in scene 38 of FIG. 28, since the position of the operator's viewpoint corresponds to the virtual surface 5B of the second layer, the virtual surfaces 5A, 5C, and 5D of the first layer, the third layer, and the fourth layer It is controlled so that the sharpness of is low. Similarly, as shown in the lower part of the scene 38 in FIG. 28, the sharpness is controlled to be lower as the position of the viewpoint is farther from the corresponding virtual surface.

In scene 39 of FIG. 28, the position of the operator's viewpoint corresponds to the tab area 34 of the virtual surface 5D of the fourth layer, and the sharpness of the virtual surfaces 5A to 5C of the first layer, the second layer, and the third layer corresponds to each other. It is controlled to be low. Similarly, as shown in the lower part of scene 39 in FIG. 28, the sharpness is controlled to be lower as the virtual surface is farther from the corresponding virtual surface.

As described above, in the input device according to the third embodiment, the degree of sharpness of the display of the virtual surface is controlled based on the position of the viewpoint of the operator. As a result, the input operation can be performed smoothly.

Further, the sharpness of the virtual surface other than the virtual surface corresponding to the viewpoint is lowered, and the virtual surface corresponding to the viewpoint can be easily gazed, so that the operator can smoothly perform the input operation.

<Fourth Embodiment>
Next, the input device according to the fourth embodiment will be described. In the fourth embodiment, the display of the virtual surface is controlled according to the position of the head of the operator. In the input device according to the fourth embodiment, the same parts as those of the input devices according to the first to third embodiments are designated by the same reference numerals, and detailed description thereof will be omitted.

As shown in FIG. 29, the input device 450 according to the fourth embodiment includes a 3D display unit 52, a line-of-sight detection unit 54, an operation input unit 56, a head position angle detection unit 457, and a control unit 458. I have. The head position angle detection unit 457 is an example of the head state acquisition unit of the disclosed technology.

The head position angle detection unit 457 acquires the position and angle of the operator's head. The head position angle detection unit 457 is realized by various methods. FIG. 30 shows an example of a method for realizing the head position angle detection unit 457.

As shown in FIG. 30, for example, the position of the operator's head is obtained by wirelessly communicating between the communication unit 42A of the HMD worn by the operator and the plurality of communication units 42B fixed at predetermined locations. And the angle can be obtained.

Further, as shown in FIG. 30, the operator's head is based on a method based on a marker (not shown) attached to a predetermined position of the HMD worn by the operator and an image captured by the fixed side camera 43. The position and angle of the part can be acquired. Specifically, the fixed-side camera 43 can image an HMD marker (not shown), and can acquire the position and angle of the operator's head according to the position of the marker.

Further, as shown in FIG. 30, the position and angle of the operator's head can be acquired by a method based on the HMD camera 44 worn by the operator and the fixed side marker 46. Specifically, the camera 44 of the HMD can image the fixed side marker 46, and can acquire the position and angle of the operator's head according to the position of the fixed side marker 46.

Further, as shown in FIG. 30, the position and angle of the operator's head can be acquired by a method based on the detection of feature points of the HMD camera 44 worn by the operator and surrounding fixed objects. Specifically, the camera 44 of the HMD can detect the feature points of the surrounding fixed objects and acquire the position and angle of the operator's head according to the position of the feature points.

Further, as shown in FIG. 30, for example, the position and angle of the operator's head can be acquired by a method based on a predetermined sensor 45 provided in the HMD worn by the operator. For example, the sensor 45 is an acceleration sensor, a gyro sensor, a geomagnetic sensor, or the like, and the position and angle of the operator's head are acquired based on the values obtained by the sensor 45.

As shown in FIG. 29, the control unit 458 includes a display processing unit 460, a viewpoint acquisition unit 62, a determination processing unit 64, a line-of-sight position fluctuation calculation unit 467, a display data processing unit 466, and an input processing unit 68. It has.

The line-of-sight position variation calculation unit 467 calculates the variation in the position of the operator's eyes based on the position and angle of the operator's head acquired by the head position angle detection unit 457.

The display data processing unit 466 controls the display positions of the plurality of virtual surfaces based on the fluctuation of the eye position of the operator calculated by the line-of-sight position fluctuation calculation unit 467.

FIG. 31 shows an explanatory diagram for explaining a method of controlling the position of the virtual surface. For example, as shown in FIG. 31, when the operator moves the head, the position of the eyes changes and the appearance of the object changes.

Therefore, the display data processing unit 466 determines the positions of the plurality of virtual surfaces so that the appearance of the plurality of virtual surfaces changes based on the fluctuation of the position of the operator's eyes calculated by the line-of-sight position fluctuation calculation unit 467. To control. Then, the display data processing unit 466 outputs a control signal to the display processing unit 60 so as to generate a three-dimensional display image for operation in which the appearance of the plurality of virtual surfaces is changed.

The display processing unit 60 generates a three-dimensional display image for operation in response to the control signal output from the display data processing unit 66.

The control unit 458 of the input device 450 can be realized by, for example, the computer 480 shown in FIG. The computer 480 includes a CPU 81, a memory 82 as a temporary storage area, and a non-volatile storage unit 483. Further, the computer 480 includes an input / output I / F 84, a read / write (R / W) unit 85 that controls reading and writing of data to the recording medium 89, and a network I / F 86 connected to a network such as the Internet. To be equipped. The CPU 81, the memory 82, the storage unit 483, the input / output I / F 84, the R / W unit 85, and the network I / F 86 are connected to each other via the bus 87.

The storage unit 483 can be realized by a Hard Disk Drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 483 as a storage medium stores an input program 490 for making the computer 480 function as a control unit 458 of the input device 450. The input program 490 includes a display processing process 492, a viewpoint acquisition processing process 93, a determination processing process 94, a display data processing process 495, an input processing process 96, and a line-of-sight position variation calculation process 497.

The CPU 81 reads the input program 490 from the storage unit 83, expands it into the memory 82, and sequentially executes the processes of each of the input programs 490.

The CPU 81 operates as the display processing unit 460 shown in FIG. 29 by executing the display processing process 492. Further, the CPU 81 operates as the viewpoint acquisition unit 62 shown in FIG. 29 by executing the viewpoint acquisition processing process 93. Further, the CPU 81 operates as the determination processing unit 64 shown in FIG. 29 by executing the determination processing process 94. Further, the CPU 81 operates as the display data processing unit 466 shown in FIG. 29 by executing the display data processing process 495. Further, the CPU 81 operates as the input processing unit 68 shown in FIG. 29 by executing the input processing process 96. Further, the CPU 81 operates as the input processing unit 68 shown in FIG. 29 by executing the line-of-sight position fluctuation calculation process 497.

In this way, the computer 80 that has executed the input program 490 functions as the input device 450.

The function realized by the input program 490 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an Application Specific Integrated Circuit (ASIC) or the like.

Next, the operation of the input device 450 according to the present embodiment will be described with reference to FIG. 33.

FIG. 33 shows an example of a flow of control processing executed by the control unit 458 of the input device 450.

The 3D display unit 52 of the input device 450 displays a three-dimensional display image, the line-of-sight detection unit 54 sequentially acquires images of both eyes of the operator, and the operation input unit 56 sequentially acquires the face image of the operator. Then, when the position and angle of the operator's head are sequentially detected by the head position / angle detection unit 457, the control processing routine shown in FIG. 33 is executed.

<Control processing routine>

In step S410, the line-of-sight position variation calculation unit 467 acquires the position and angle of the operator's head acquired by the head position angle detection unit 457.

In step S412, the line-of-sight position change calculation unit 467 calculates the change in the position of the operator's eyes based on the position and angle of the operator's head acquired in step S410.

In step S414, the display data processing unit 466 determines the positions of the plurality of virtual surfaces so that the appearance of the plurality of virtual surfaces changes based on the fluctuation of the position of the operator's eyes calculated in step S412. Control. Then, the display data processing unit 466 outputs a control signal to the display processing unit 460 so as to generate a three-dimensional display image for operation in which the appearance of the plurality of virtual surfaces is changed. Then, the display processing unit 460 generates a three-dimensional display image for operation in response to the control signal output from the display data processing unit 466. Then, the display processing unit 460 outputs the generated three-dimensional display image to the 3D display unit 52. The 3D display unit 52 displays the three-dimensional display image generated by the display processing unit 460 to the operator.

As described above, in the input device according to the fourth embodiment, the variation in the position of the operator's eyes is calculated based on the position and angle of the operator's head. As a result, the appearance of the plurality of virtual surfaces changes, and the operator can smoothly perform the input operation.

Next, a modified example of the embodiment will be described.

In the second embodiment, a case where the degree of transparency of the virtual surface is controlled according to the corresponding virtual surface at the position x0 in the depth direction of the viewpoint based on the position of the viewpoint of the operator has been described as an example. Is not limited to this. For example, the degree of transparency of the virtual surface may be controlled according to the virtual surface having the tab area corresponding to the position (y0, z0) of the viewpoint of the operator. In addition, the degree of transparency of the virtual surface is controlled according to the corresponding virtual surface at the position x0 in the depth direction of the viewpoint position, and the tab area corresponding to the viewpoint position (y0, z0) on the virtual surface is created. The degree of transmission of the virtual surface may be controlled according to the virtual surface having the virtual surface. In this case, priority is given to either the virtual surface corresponding to the position x0 in the depth direction of the viewpoint position or the virtual surface having the tab area corresponding to the viewpoint position (y0, z0) of the virtual surface. The degree of transparency may be controlled.

Further, in the above embodiment, an input device for receiving character input from an operator has been described as an example, but the present invention is not limited to this. For example, it may be an input device related to a complicated operation input (for example, an operation system of an aircraft, an operation of a device provided in a control tower, etc.).

Further, in the above embodiment, an input device for receiving an operation input from an operator has been described as an example, but the present invention is not limited to this. Since the operation input is not essential, for example, the input device may only perform the process of selecting the icon included in the virtual surface corresponding to the position of the viewpoint of the operator.

Further, in the above embodiment, the case where the number of virtual surfaces is 3 to 5 layers has been described as an example, but the present invention is not limited to this, and the virtual surface may be any number of layers.

In the above embodiment, an example will be described in which an icon corresponding to the position of the viewpoint of the operator is selected and the degree of transmission (or degree of sharpness) of the virtual surface in front of the virtual surface including the icon is controlled. However, the processing flow is not limited to this. For example, a virtual surface corresponding to the position of the operator's viewpoint is specified, the degree of transparency of the virtual surface is controlled, and an icon on the specified virtual surface is selected according to the subsequent movement of the viewpoint. It may be controlled to.

The following additional notes will be further disclosed with respect to the above embodiments.
(Appendix 1)
A control unit that arranges a plurality of virtual surfaces including icons for input operations side by side in the depth direction when viewed from the operator, and allows the plurality of virtual surfaces to pass through and be displayed on a three-dimensional display unit.
A viewpoint acquisition unit that acquires the position of the operator's viewpoint, and
A selection unit that selects the icon included in the virtual surface corresponding to the position of the viewpoint based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit.
Input device equipped with.

(Appendix 2)
The input device according to Appendix 1, wherein the control unit controls the degree of transmission of each of the virtual surfaces based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit.

(Appendix 3)
The control unit controls the degree of transmission of the virtual surface corresponding to the position of the viewpoint based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit. Input device.

(Appendix 4)
Based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the control unit controls so that the degree of transmission of the virtual surface in front of the virtual surface corresponding to the position of the viewpoint is higher. The input device according to Appendix 2 or Appendix 3.

(Appendix 5)
The input device according to any one of Supplementary note 1 to Appendix 4, wherein each of the virtual surfaces has a tab area at a position where the display does not overlap with another virtual surface different from the virtual surface.

(Appendix 6)
The intention acquisition unit that acquires information on the manifestation of intention of the operator,
Further including a determination unit for determining the type of operation of the operator based on the information regarding the manifestation of intention of the operator acquired by the intention acquisition unit.
The selection unit determines whether or not to confirm the selection of the icon based on the selected icon and the type of operation determined by the determination unit. Any one of Supplementary notes 1 to 5. The input device according to item 1.

(Appendix 7)
The control unit controls the degree of sharpness of each display on the virtual surface based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit. Input device.

(Appendix 8)
Based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the control unit has a higher degree of sharpness of the display of the virtual surface corresponding to the position of the viewpoint than the degree of sharpness of other virtual surfaces. The input device according to Appendix 7, which is controlled so as to be.

(Appendix 9)
Based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the control unit reduces the sharpness of the display of another virtual surface different from the virtual surface corresponding to the position of the viewpoint. The input device according to Appendix 7 or Appendix 8.

(Appendix 10)
Based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the control unit displays information input from the operator at a position corresponding to the virtual surface corresponding to the position of the viewpoint. The input device according to any one of Supplementary note 1 to Supplementary note 9, which controls the three-dimensional display unit so as to perform the operation.

(Appendix 11)
Further including a head state acquisition unit for acquiring the position and angle of the operator's head,
The input device according to any one of Supplementary note 1 to Supplementary note 10, wherein the control unit controls a display position of a plurality of the virtual surfaces based on the position and the angle detected by the head state acquisition unit. ..

(Appendix 12)
Each of the icons included in the plurality of virtual surfaces has a larger size of the icon as the position of the virtual surface is located in the depth direction when viewed from the operator. Any one of Addendums 1 to 11. The input device described in.

(Appendix 13)
Each of the icons included in the plurality of virtual surfaces has a wider distance from the icons arranged next to each other so that the position of the virtual surface is located in the depth direction when viewed from the operator. The input device according to any one of the above items.

(Appendix 14)
Each of the icons included in the plurality of virtual surfaces is arranged so that at least a part of the icons does not overlap with the icons contained in another virtual surface different from the virtual surface including the icon. ~ The input device according to any one of Appendix 13.

(Appendix 15)
The input device according to any one of Supplementary note 1 to Supplementary note 14, wherein the icons are classified and arranged on a plurality of the virtual surfaces according to the type of the icon.

(Appendix 16)
The input device according to any one of Supplementary note 1 to Supplementary note 15, wherein at least one of the shapes and colors of the icons is defined according to the type of information input by selecting the icon.

(Appendix 17)
The input device according to Appendix 1 to Appendix 16, wherein at least one of the shapes and colors of each of the icons is determined according to the position of the icon in the virtual surface.

(Appendix 18)
Based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the control unit controls so that the degree of transmission of the virtual surface deeper than the virtual surface corresponding to the position of the viewpoint is higher. The input device according to any one of Supplementary note 1 to Supplementary note 17.

(Appendix 19)
A plurality of virtual surfaces including icons for input operations are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transparently displayed on the three-dimensional display unit.
Acquire the position of the viewpoint of the operator and
Based on the acquired position of the viewpoint of the operator, the icon corresponding to the position of the viewpoint is selected.
An input program that causes a computer to perform processing.

(Appendix 20)
The input program according to Appendix 19, which causes a computer to execute a process of controlling the degree of transmission of each of the virtual surfaces based on the acquired position of the viewpoint of the operator.

(Appendix 21)
The input program according to Appendix 20, wherein a computer is made to execute a process of controlling the degree of transparency of the virtual surface corresponding to the position of the viewpoint based on the acquired position of the viewpoint of the operator.

(Appendix 22)
Addendum 20 or annex to cause the computer to execute a process of controlling the degree of transparency of the virtual surface in front of the virtual surface corresponding to the position of the viewpoint based on the acquired position of the viewpoint of the operator. 21. The input program.

(Appendix 23)
The input program according to any one of Supplementary note 19 to Supplementary note 22, wherein each of the virtual surfaces has a tab area at a position where the display does not overlap with another virtual surface different from the virtual surface.

(Appendix 24)
Obtain information on the manifestation of intention of the operator
Based on the information regarding the manifestation of intention of the operator acquired by the acquisition unit, the type of operation of the operator is determined.
The item described in any one of Supplementary note 19 to Supplementary note 23, wherein a computer is made to execute a process of determining whether or not to confirm the selection of the icon based on the selected icon and the determined type of the operation. Input program.

(Appendix 25)
The input program according to any one of Appendix 19 to Appendix 24, which causes a computer to execute a process of controlling the sharpness of the display of the virtual surface based on the acquired position of the viewpoint of the operator.

(Appendix 26)
Based on the acquired position of the viewpoint of the operator, a computer controls the degree of sharpness of each display of the virtual surface corresponding to the position of the viewpoint so as to be higher than the degree of sharpness of other virtual surfaces. The input program according to any one of Appendix 19 to Appendix 25.

(Appendix 27)
Addendum: Based on the acquired position of the viewpoint of the operator, the computer is made to execute a process of controlling the display of another virtual surface different from the virtual surface corresponding to the position of the viewpoint to be less clear. The input program according to 26.

(Appendix 28)
Based on the acquired position of the viewpoint of the operator, the three-dimensional display unit is controlled so as to display the information input from the operator at the position corresponding to the virtual surface corresponding to the position of the viewpoint. The input program according to Appendix 26 or Appendix 27, which causes a computer to execute the processing to be performed.

(Appendix 29)
Appendix 19 Causes a computer to execute a process of controlling a plurality of virtual surface display positions based on the position and the angle detected by the head state acquisition unit that acquires the position and angle of the operator's head. -The input program according to any one of Appendix 28.

(Appendix 30)
Each of the icons included in the plurality of virtual surfaces has a larger size of the icon as the position of the virtual surface is located in the depth direction when viewed from the operator. Any one of Appendix 19 to Appendix 29. The input program described in.

(Appendix 31)
Each of the icons included in the plurality of virtual surfaces has a wider distance from the icons arranged next to each other so that the position of the virtual surface is located in the depth direction when viewed from the operator. The input program according to any one of the above.

(Appendix 32)
Each of the icons included in the plurality of virtual surfaces is arranged so that at least a part of the icons does not overlap with the icons contained in another virtual surface different from the virtual surface including the icon. -The input program according to any one of Appendix 31.

(Appendix 33)
The input program according to any one of Supplementary note 19 to Supplementary note 32, wherein the icons are classified and arranged on a plurality of the virtual surfaces according to the type of the icon.

(Appendix 34)
The input program according to any one of Supplementary note 19 to Supplementary note 33, wherein at least one of the shape and color of each of the icons is defined according to the type of information input by selecting the icon.

(Appendix 35)
The input program according to any one of Supplementary note 19 to Supplementary note 34, wherein at least one of the shape and color of each of the icons is defined according to the position of the icon in the virtual surface.

(Appendix 36)
Addendum 19 to Addendum 19 to make the computer execute a process of controlling the degree of transparency of the virtual surface deeper than the virtual surface corresponding to the position of the viewpoint based on the acquired position of the viewpoint of the operator. The input program according to any one of 35.

(Appendix 37)
A plurality of virtual surfaces including icons for input operations are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transparently displayed on the three-dimensional display unit.
Acquire the position of the viewpoint of the operator and
Based on the acquired position of the viewpoint of the operator, the icon corresponding to the position of the viewpoint is selected.
An input method that causes a computer to perform processing.

(Appendix 38)
The input method according to Appendix 37, wherein a computer is made to execute a process of controlling the degree of transmission of each of the virtual surfaces based on the acquired position of the viewpoint of the operator.

(Appendix 39)
The input method according to Appendix 38, wherein the computer executes a process of controlling the degree of transparency of the virtual surface corresponding to the position of the viewpoint based on the acquired position of the viewpoint of the operator.

(Appendix 40)
Addendum 38 or annex to cause the computer to execute a process of controlling the degree of transparency of the virtual surface in front of the virtual surface corresponding to the position of the viewpoint based on the acquired position of the viewpoint of the operator. 39. The input method.

(Appendix 41)
The input method according to any one of Appendix 37 to Appendix 40, wherein each of the virtual surfaces has a tab area at a position where the display does not overlap with another virtual surface different from the virtual surface.

(Appendix 42)
Obtain information on the manifestation of intention of the operator
Based on the information regarding the manifestation of intention of the operator acquired by the acquisition unit, the type of operation of the operator is determined.
The item described in any one of Supplementary note 37 to Appendix 41, which causes a computer to execute a process of determining whether or not to confirm the selection of the icon based on the selected icon and the determined type of the operation. Input method.

(Appendix 43)
The input method according to any one of Supplementary note 37 to Supplementary note 42, wherein a computer is made to execute a process of controlling the sharpness of each display of the virtual surface based on the acquired position of the viewpoint of the operator.

(Appendix 44)
Based on the acquired position of the viewpoint of the operator, the computer executes a process of controlling the display sharpness of the virtual surface corresponding to the position of the viewpoint to be higher than the sharpness of other virtual surfaces. The input method according to Appendix 43.

(Appendix 45)
Addendum: Based on the acquired position of the viewpoint of the operator, the computer is made to execute a process of controlling the display of another virtual surface different from the virtual surface corresponding to the position of the viewpoint to be less clear. 43 or the input method according to Appendix 44.

(Appendix 46)
Based on the acquired position of the viewpoint of the operator, the three-dimensional display unit is controlled so as to display the information input from the operator at the position corresponding to the virtual surface corresponding to the position of the viewpoint. The input method according to any one of Supplementary note 37 to Supplementary note 45, which causes a computer to execute the process to be performed.

(Appendix 47)
Appendix 37 Causes a computer to execute a process of controlling a plurality of virtual surface display positions based on the position and the angle detected by the head state acquisition unit that acquires the position and angle of the operator's head. -The input method according to any one of Appendix 46.

(Appendix 48)
Each of the icons included in the plurality of virtual surfaces has a larger size of the icon as the position of the virtual surface is located in the depth direction when viewed from the operator. Any one of Appendix 37 to Appendix 47. Input method described in.

(Appendix 49)
Each of the icons included in the plurality of virtual surfaces has a wider distance from the icons arranged next to each other so that the position of the virtual surface is located in the depth direction when viewed from the operator. The input method according to any one of the above items.

(Appendix 50)
Each of the icons included in the plurality of virtual surfaces is arranged so that at least a part of the icons does not overlap with the icons contained in another virtual surface different from the virtual surface including the icon. -The input method according to any one of Appendix 49.

(Appendix 51)
The input method according to any one of Supplementary note 37 to Supplementary note 50, wherein the icons are classified and arranged on a plurality of the virtual surfaces according to the type of the icon.

(Appendix 52)
The input method according to any one of Supplementary note 37 to Supplementary note 51, wherein at least one of the shapes and colors of each of the icons is defined according to the type of information input by selecting the icon.

(Appendix 53)
The input method according to any one of Supplementary note 37 to Supplementary note 52, wherein at least one of the shape and color of each of the icons is determined according to the position of the icon in the virtual surface.

(Appendix 54)
Addendum 37 to Addendum 37 to make the computer execute a process of controlling the degree of transparency of the virtual surface deeper than the virtual surface corresponding to the position of the viewpoint based on the acquired position of the viewpoint of the operator. The input method according to any one of 53.

(Appendix 55)
A plurality of virtual surfaces including icons for input operations are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transparently displayed on the three-dimensional display unit.
Acquire the position of the viewpoint of the operator and
Based on the acquired position of the viewpoint of the operator, the icon corresponding to the position of the viewpoint is selected.
A storage medium that stores an input program that causes a computer to execute processing.

4 Icon 5 3D display image 5A, 5B, 5C, 5D, 5E Virtual surface 5N Input display surface 6 Operator 34 Tab area 50,450 Input device 52 3D display 54 Line-of-sight detection unit 56 Operation input unit 58,458 Control unit 60,460 Display processing unit 62 Viewpoint acquisition unit 64 Judgment processing unit 66,466 Display data processing unit 68 Input processing unit 80,480 Computer 81 CPU
82 Memory 83,483 Storage unit 89 Recording medium 90,490 Input program 92,492 Display processing process 93 Viewpoint acquisition processing process 94 Judgment processing process 95,495 Display data processing process 96 Input processing process 497 Line-of-sight position fluctuation calculation process 457 Head Position / angle detection unit 467 Line-of-sight position fluctuation calculation unit

Claims (10)

There are a plurality of virtual surfaces including icons for input operations, and each of the icons included in the plurality of virtual surfaces is such that the position of the virtual surface is located in the depth direction when viewed from the operator. A control unit that arranges the plurality of virtual surfaces having a large size side by side in the depth direction when viewed from the operator, and allows the plurality of virtual surfaces to pass through and display on a three-dimensional display unit.
A viewpoint acquisition unit that acquires the position of the operator's viewpoint, and
A selection unit that selects the icon included in the virtual surface corresponding to the position of the viewpoint based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit.
Equipped with a,
The control unit is a depth position of the virtual surface corresponding to the position of the viewpoint based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, and is selected by the selection unit. The three-dimensional display unit is controlled so as to display the information input from the operator at a position different from the icon.
Input device. The first aspect of claim 1, wherein the control unit controls the degree of transmission of the virtual surface corresponding to the position of the viewpoint based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit. Input device. The input device according to claim 1 or 2, wherein each of the virtual surfaces has a tab area at a position where the display does not overlap with another virtual surface different from the virtual surface. The intention acquisition unit that acquires information on the manifestation of intention of the operator,
Further including a determination unit for determining the type of operation of the operator based on the information regarding the manifestation of intention of the operator acquired by the intention acquisition unit.
The selection unit determines whether or not to confirm the selection of the icon based on the selected icon and the type of the operation determined by the determination unit. The input device according to any one item. Based on the position of the viewpoint of the operator acquired by the viewpoint acquisition unit, the control unit has a higher degree of sharpness of the display of the virtual surface corresponding to the position of the viewpoint than the degree of sharpness of other virtual surfaces. The input device according to any one of claims 1 to 4, wherein the input device is controlled so as to be. Further including a head state acquisition unit for acquiring the position and angle of the operator's head,
The one according to any one of claims 1 to 5 , wherein the control unit controls a display position of a plurality of the virtual surfaces based on the position and the angle detected by the head state acquisition unit. Input device. Claim that each of the icons included in the plurality of virtual surfaces is arranged so that at least a part of the icons does not overlap with the icons contained in another virtual surface different from the virtual surface including the icon. The input device according to any one of claims 1 to 6 . Each of the icons included in the plurality of virtual surfaces has a wider distance from the icons arranged next to each other as the position of the virtual surface is located in the depth direction when viewed from the operator.
The input device according to any one of claims 1 to 7 . There are a plurality of virtual surfaces including icons for input operations, and each of the icons included in the plurality of virtual surfaces is such that the position of the virtual surface is located in the depth direction when viewed from the operator. The plurality of virtual surfaces having a large size are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transparently displayed on the three-dimensional display unit.
Acquire the position of the viewpoint of the operator and
Based on the acquired position of the viewpoint of the operator, the icon corresponding to the position of the viewpoint is selected .
Based on the acquired position of the viewpoint of the operator, the operator inputs the depth position of the virtual surface corresponding to the position of the viewpoint and a position different from the selected icon. Control the three-dimensional display unit to display information,
An input program that causes a computer to perform processing. There are a plurality of virtual surfaces including icons for input operations, and each of the icons included in the plurality of virtual surfaces is such that the position of the virtual surface is located in the depth direction when viewed from the operator. The plurality of virtual surfaces having a large size are arranged side by side in the depth direction when viewed from the operator, and the plurality of virtual surfaces are transparently displayed on the three-dimensional display unit.
Acquire the position of the viewpoint of the operator and
Based on the acquired position of the viewpoint of the operator, the icon corresponding to the position of the viewpoint is selected .
Based on the acquired position of the viewpoint of the operator, the operator inputs the depth position of the virtual surface corresponding to the position of the viewpoint and a position different from the selected icon. Control the three-dimensional display unit to display information,
An input method that causes a computer to perform processing.