JP2005332231A - Pointing method, pointing device and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize intuitive and accurate perception by an operator on which depth position a pointer is located and where the pointer is pointing, in pointing using a pointer in a three-dimensional GUI. <P>SOLUTION: A pointing method is provided for pointing by making a pointer, which is indicated on a display unit capable of expressing a three-dimensional space, move and rotate in a two-dimensional plane, and pointing an arbitrary position in the three-dimensional space expressed on the display unit by moving the pointer in the normal line direction (hereafter referred to as depth direction) of the two-dimensional plane. The pointing method includes a step 1 for displaying the pointer on the display unit by moving or rotating the pointer in the two-dimensional space, and a step 2 for displaying the pointer on the display unit by moving the pointer in the depth direction. The step 2 further includes a step 3 for displaying the pointer on the display unit by changing the depth position of one end of the pointer, while maintaining the depth position of a portion of the pointer. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a pointing method, a pointing device, and a pointing program, and more particularly to a technique that is effective when applied to a method and a device for pointing by operating a pointer displayed on a display device capable of three-dimensional display. .

  Conventionally, GUI (Graphical User Interface) has more display, function, and operation than ever before due to remarkable improvement in computing performance of PC (Personal Computer), speeding up of processing speed, enhancement of graphic function, etc. When the operator performs a desired operation, it is often complicated to prevent efficient operation.

  In order to improve such a state, in recent years, it has been proposed to present and manipulate information using a three-dimensional space. This is often called a three-dimensional GUI or the like, and is a mechanism in which an object is arranged three-dimensionally in a three-dimensional space and the object is operated using a predetermined input device. In addition to the 3D GUI, a similar mechanism may be used in, for example, CAD (Computer Aided (or Assisted) Design) or CG (Computer Graphics) for designing in a 3D space. From the point of view of manipulating and pointing to a three-dimensional object, let's proceed with a three-dimensional GUI as an example. When the three-dimensional GUI is used, it is possible to three-dimensionally arrange objects that have been arranged two-dimensionally or superimposed on each other so that the work space can be handled efficiently. In addition, since the real world surrounding us is a three-dimensional space, the GUI can be handled more intuitively than a two-dimensional GUI by making the GUI three-dimensional.

  Here, one of the problems that are required in the process of operating the three-dimensional GUI is a method of pointing objects at various depth positions. In the conventional technology, a new function necessary for moving in the depth direction is added to the operation of the mouse, keyboard, joystick, etc. used in the two-dimensional GUI, so that the pointer in the three-dimensional space is added. The movement is realized. Further, the position of the pointer in the depth direction is expressed by, for example, enlarging or reducing the pointer, and pointing to an object in the back of the space (for example, see Non-Patent Document 1). In other words, a method has been proposed in which a pointer is displayed larger when the pointer is in front in the three-dimensional space, and is displayed smaller when it is in the back.

  However, many of these methods allow the pointer displayed in the three-dimensional space to move freely in the depth direction in the space, so no matter how much the pointer is displayed on the display device, The operator often cannot perceive where the user is pointing at the depth position.

  In order to solve this problem, for example, there is a method in which a reference representing an xyz axis or the like whose depth position does not change is displayed near the pointer to make it easier to perceive the depth position (see, for example, Patent Document 1). However, the appearance of the reference on the display surface may obstruct the operator's view and reduce the work efficiency. Also, the reference does not appear when pointing in the real world. Therefore, the method of displaying the reference is not an optimal method from the viewpoint of operating the GUI with the same feeling as in the real world.

From the above, in order to operate the 3D GUI in the same way as in the real world, in the pointing operation using the pointer in the 3D GUI, the depth position of the pointer and where it is pointing There is a need for an efficient pointing method that allows an operator to quickly and accurately perceive.
Keita Watanabe, Toshiaki Yasumura, "RUI: Realizable User Interface," Proceedings of Human Interface Symposium 2003, pp.541-544, 2003 JP-A-8-248938

  As described in the background art, the problem to be solved by the present invention is that in the conventional pointing operation using the pointer in the three-dimensional GUI, the pointer is located at which depth position and where the pointer is pointed. It is difficult for the operator to perceive intuitively and accurately.

  An object of the present invention is to allow an operator to intuitively and accurately perceive which depth position the pointer is pointing to and where in the pointing using the pointer in the three-dimensional GUI. It is an object to provide a pointing method and a pointing device.

  The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  The outline of the invention disclosed in the present application will be described as follows.

  (1) A pointer displayed on a display device capable of expressing a three-dimensional space is moved or rotated in a two-dimensional plane and moved in the normal direction (hereinafter referred to as a depth direction) of the two-dimensional plane. A pointing method for pointing an arbitrary position in a three-dimensional space represented on the display device, wherein the pointer is moved or rotated in the two-dimensional plane and displayed on the display device; and And moving the pointer in the depth direction to display on the display device, wherein the step 2 changes the depth position of one end of the pointer while maintaining the depth position of a part of the pointer. A pointing method having step 3 of displaying on the display device.

  (2) A pointer displayed on a display device capable of expressing a three-dimensional space is moved or rotated in a two-dimensional plane and moved in a normal direction (hereinafter referred to as a depth direction) of the two-dimensional plane. An input information acquisition unit for pointing an arbitrary position in a three-dimensional space expressed on the display device, wherein the input information acquisition means acquires operation information of the pointer input from the input device, and the input information acquisition Based on the operation information acquired by the means, the pointer movement / rotation angle in the two-dimensional plane is calculated, and the pointer position / deformation calculation for calculating the pointer movement in the depth direction is calculated. Based on the movement amount or the rotation angle of the pointer in the two-dimensional plane calculated by the means and the pointer position / deformation amount calculation means. Based on the means for translating or rotating the pointer and the calculated amount of movement of the pointer in the depth direction, the pointer at which the depth position of one end of the pointer is changed while maintaining the depth position of a part of the pointer A pointing device including a pointer generating unit including a unit for generating the pointer, and a display control unit configured to display the pointer generated by the pointer generating unit on the display device.

  In the pointing method of the present invention, as described in the means (1), when the pointer is moved in the depth direction, the position of only a part of the pointer is displayed so as to change. Therefore, the operator can intuitively and accurately perceive the depth position of the pointer and the pointing position. At this time, in the means of (1), the step 2 includes, for example, a step 4 in which the display device is displayed by being translated in the depth direction while maintaining the shape of the pointer in addition to the step 3. May be.

  In step (1), for example, the step 3 may be a method of rotating the pointer by a predetermined angle around an axis parallel to the two-dimensional plane and displaying the pointer on the display device. . In such a method, for example, the tip of the pointer, that is, the part where the object is pointed may be rotated in a direction toward the back as viewed from the operator. In this way, the pointer viewed by the operator becomes narrower at the tip of the pointer. Therefore, the operator can intuitively understand that the position pointed by the pointer is pointing deeper than the depth position where the pointer was located before the operation.

  Further, at this time, a plurality of axes for rotating the pointer may be set, and the display device may be displayed by rotating about one of the plurality of axes or about two or more axes. . In this case, for example, two axes of the first axis and the second axis are set, and the pointer is rotated around either the first axis or the second axis for display. By doing so, the pointer can be displayed in four ways of tilting. Therefore, for example, it is possible to point an object on the top, bottom, left and right of the pointer without moving the pointer displayed at a certain position.

  In addition to these methods, in step 3, for example, only a part of the pointer may be deformed into a shape moved in the depth direction and displayed on the display device. In this case, for example, the pointer may be deformed in a staircase shape such that the moving portion is translated in the depth direction, or the moving portion may be curved in the depth direction. Furthermore, the shape may be such that only the moving part is inclined in the depth direction.

  In the step 3, the pointer may be rotated or deformed stepwise based on operation information input from the input device and displayed on the display device. By doing so, the shape of the pointer viewed from the operator changes stepwise, and it becomes easy to perceive which depth position is pointing.

  In the pointing method described in the means of (1), for example, a reference pointer that does not change the position in the depth direction may be displayed on the display device together with the pointer. In this way, the operator can intuitively and accurately perceive the depth position of the pointer.

  In the pointing method described in the means (1), when a general two-dimensional display device such as a CRT display or a liquid crystal display is used as the display device, a DFD (Depth Fusion Display) is used. The present invention can be applied to any case where a display device (display) capable of displaying such a three-dimensional stereoscopic image is used, but is particularly preferable when applied to a case where a three-dimensional display device such as the DFD is used. . When displaying the pointer in the DFD, the pointer projects from the viewpoint of the operator onto the plurality of display surfaces that overlap in the depth direction when viewed from the operator, and overlaps when viewed from the viewpoint. What is necessary is just to change and display the ratio of the brightness | luminance of the point on a display surface. In this way, the pointer viewed from the operator can be seen near (front) when the brightness of the display surface in front of the operator is high, and far away from the high brightness of the display surface at the back. It looks like (back).

  Further, the pointing method described in the means (1) can be realized by using, for example, the pointing device described in the means (2). At this time, the pointer generating means may include means for enlarging or reducing the pointer while maintaining the shape of the pointer, in addition to the means described above.

  Further, in the pointing generation means of the pointing device, the means for generating a pointer in which the depth position of one end of the pointer is changed while maintaining the depth position of a part of the pointer is the two-dimensional It is only necessary to rotate a predetermined angle around an axis parallel to the plane. In addition, for example, the pointer may be rotated by a predetermined angle around a first axis or a second axis parallel to the two-dimensional plane. Still further, only a part of the pointer may be deformed so as to translate in the depth direction.

  Further, at this time, the means for generating a pointer in which the depth position of one end of the pointer is changed while maintaining the depth position of a part of the pointer is based on the operation information input from the input device. You may rotate or deform in steps.

  The display control unit may display a reference pointer that does not change the position in the depth direction on the display device together with the pointer.

  Further, when the display device is the DFD, the display control means displays, for example, a pointer projected from the viewpoint of the operator on a plurality of display surfaces overlapping in the depth direction when viewed from the operator, and The luminance ratio of the points on the display surfaces that overlap when viewed from the viewpoint may be changed.

  In addition to the above-described means, the pointing device may further include an amount of movement or a rotation angle of the pointer in the two-dimensional plane calculated by the pointer position / deformation amount calculation means, and a position of the pointer in the depth direction. Pointing determination means for determining whether or not there is an object pointed to by the pointer when the pointer is moved based on a movement amount, and a pointing state when there is the pointing object You may provide the object production | generation means which produces | generates the object changed into. In this way, it is possible to intuitively and accurately perceive the object pointed by the pointer.

  In addition, the pointing device described in (2) may be, for example, one means (component) incorporated in a computer such as the PC or an information processing system. In this case, a program for causing the computer to execute the pointing method as described in (1) above may be created and installed in the computer.

Hereinafter, the present invention will be described in detail together with embodiments (examples) with reference to the drawings.
In all the drawings for explaining the embodiments, parts having the same function are given the same reference numerals and their repeated explanation is omitted.

  In the pointing method of the present invention, when the pointer is moved in the depth direction and displayed, the pointer is rotated, or only a part of the pointer is moved, so that the operator can point the depth position of the pointer. To intuitively and accurately perceive the current position.

FIG. 1 is a schematic diagram showing a configuration example of a system for realizing the pointing method of the present invention.
In FIG. 1, 1 is a pointing device, 101 is input information acquisition means, 102 is pointer position / deformation amount calculation means, 103 is pointer generation means, 104 is display control means, 105 is pointing determination means, 106 is object generation means, Reference numeral 107 denotes processing control means, 108 denotes storage means, 2 denotes an input device, and 3 denotes a display device.

  In the pointing method of the present invention, for example, a pointer in a three-dimensional space represented on a display device connected to a system control device such as the PC is used to input a pointer using an input device connected to the system control device. This is a pointing method that is preferably applied when operating in a three-dimensional manner and pointing at an arbitrary position in the three-dimensional space.

  At this time, for example, as shown in FIG. 1, the system control apparatus 1 includes input information acquisition means 101 for acquiring input information input from the input apparatus 2 and input information acquired by the input information acquisition means 101. Is a pointer position / deformation amount calculation means 102 for calculating a pointer movement direction and movement amount, a rotation direction, a rotation amount, and the like based on the input information, and the pointer position / deformation amount. A pointer generation unit 103 that generates a pointer based on a calculation result of the calculation unit 102 and a display control unit 104 that displays the pointer generated by the pointer generation unit 103 on the display device 3 are provided. In addition to the above-described units, the system control apparatus 1 includes, for example, an object that is pointed to a pointer that is generated based on the calculation result of the pointer position / deformation amount calculation unit 102 as illustrated in FIG. For example, a pointing determination unit 105 that determines whether or not there is an object, and an object generation unit 106 that changes the color of the object when the object is pointed to, for example.

  The system control device 1 is a device that activates and operates software according to input information from the input device 2 and controls other devices, such as the PC. As shown in FIG. 1, in addition to the above means, for example, a processing control means 107 and a storage means 108 are provided. When the information acquired by the input information acquisition unit 101 is different from the information related to the control of the pointer, the input information acquisition unit 101 passes the acquired information to the processing control unit 107 and adds the acquired information to the acquired information. The system controller 1 is caused to execute the corresponding process.

  The display device 3 may be a display device that can express a three-dimensional space. For example, an existing two-dimensional display device such as a CRT display, a liquid crystal display, or an HMD (Head Mount Display) may be used, such as a DFD. A display device capable of displaying a three-dimensional stereoscopic image may be used. That is, any display device may be used as long as the operator (observer) can perceive the displayed pointer or object three-dimensionally.

  The input device 2 may be, for example, a pen tablet or a joystick (joypad) in addition to an input device generally connected to the PC such as a keyboard or a mouse. Furthermore, the input device 2 may be configured such that, for example, a touch panel or a tablet is integrated with the display device 3 and information can be input by touching the display screen of the display device 3 with a pen or a fingertip.

  2 to 5 are schematic diagrams for explaining the pointing method according to the first embodiment of the present invention. FIG. 2 is a diagram for explaining the principle. FIG. 3 is a diagram showing a pointing method for an object behind the pointer. FIG. 4 is a diagram for explaining a pointing method for an object in front of the pointer, and FIG. 5 is a flowchart for explaining a processing procedure in the system control apparatus. Also, in FIG. 3, among the nine diagrams below the long horizontal arrow, the three diagrams shown at the left end of the page are a perspective view of the pointer and the object before pointing from the operator's viewpoint. The figure seen from above and the figure seen from the right side. The three figures shown in the center of the page are the figure when the pointer and the object are viewed from the operator's point of view when the pointer is operated. The figure is a view from the right side, and the three figures shown at the right end of the page are the pointer and object viewed from the operator's point of view when the object is pointed, the view from the upper right, and the right side. It is the figure seen from the surface. In addition, in FIG. 4, among the four diagrams under the long horizontal arrow in the drawing, the two drawings shown on the left side of the drawing are the view of the pointer and the object before pointing from the viewpoint of the operator, and the right side. The two views shown on the right side of the drawing are a view of the pointer and the object when the object is pointed and the view from the viewpoint of the operator, and a view from the right side.

  In the pointing method of the first embodiment, as shown in FIG. 2, for example, a keyboard 201 and a mouse 202 are used as the input device 2. For example, as shown in FIG. 2, when the wheel 202A of the mouse 202 is rotated while pressing a predetermined key such as a control key (Ctrl key) 201A of the keyboard 201, the rotation direction of the wheel 202A is rotated. The pointer 4 displayed on the display device 3 rotates and tilts according to the number of rotations. At this time, it is assumed that xyz three-dimensional coordinates as shown in FIG. 2 are taken on the display device 3, and an operator (observer) is looking at the display device 3 from the −z direction. At this time, the pointer 4 rotates, for example, around a rotation axis parallel to the y-axis, for example, according to the rotation direction of the wheel 202A, or in the back direction (+ z direction) as viewed from the operator. In either direction (−z direction).

  In the pointing method according to the first embodiment, even if the pointer 4 and the object 5 overlap with each other as viewed from the operator's viewpoint on the display device 3, for example, as shown on the left side of FIG. Are different, the object 5 is not pointed. In such a state, for example, as shown at the left end of FIG. 3, when the wheel 202 </ b> A of the mouse 202 is rotated toward the back as viewed from the measurer while pressing the control key 201 </ b> A of the keyboard 201. As shown in the center of FIG. 3, the pointer 4 tilts in the direction of the object 5 (+ z direction). At this time, the operator sees that the tip of the pointer 4, in other words, the width of the arrow portion is narrower than the width of the arrowhead side. Therefore, the operator can perceive from the shape of the pointer 4 that the pointer 4 is pointing deeper than the position before the operation. Here, when the wheel 202A of the mouse 202 is further rotated toward the back as viewed from the measurer while pressing the control key 201A of the keyboard 201, as shown at the right end of FIG. The pointer 4 further tilts in the direction of the object 5 (+ z direction). At this time, it appears to the operator that the width of the tip of the pointer 4 is further narrowed. At this time, as shown in FIG. 3, when the tip of the pointer 4 comes into contact with the object 5, the object 5 is in a state pointed to by the pointer 4. Therefore, for example, the color of the object 5 is changed to indicate that it is in a pointing state. In this way, the operator can intuitively and accurately perceive the position in the depth direction of the pointer from the shape of the pointer 4 and the position in the depth direction where the pointer 4 is pointing. In addition, by changing the color of the object 5 when it is pointed by the pointer 4, it is possible to intuitively and accurately perceive whether or not the pointer 4 overlapping the object 5 is pointing to the object 5. .

  Further, FIG. 3 illustrates the operation when the object 5 is in the back (far) than the pointer 4 and the pointer 4 is tilted in the back (+ z direction) when viewed from the operator. In the pointing method of Example 1, the pointer 4 can be tilted forward (-z direction). Here, for example, as shown on the left side of FIG. 4, a case is considered where the pointer 4 is displayed behind (distant from) the object 5 as viewed from the operator. At this time, contrary to the operation method shown in FIG. 3, for example, when the wheel 202A of the mouse 202 is rotated forward while pressing the control key 201A of the keyboard 201, as shown on the right side of FIG. The pointer 4 is tilted forward (-z direction). At this time, since the width of the arrow portion of the pointer 4 looks wider than the width of the arrowhead side, the operator can perceive that the pointer 4 is tilted forward. If the color of the object 5 is changed when the tip of the pointer 4 tilted forward reaches a certain position, the operator can perceive that the object 5 is pointed.

  At this time, for example, the pointer 4 may be displayed so as to incline continuously in accordance with the total rotation angle of the wheel 202A of the mouse 202, or in synchronization with the rotation step of the wheel 202A. The pointer 4 may be displayed so as to be inclined by the angle θ each time the step is rotated.

  In FIG. 3, the arrow-shaped pointer 4 is used for pointing the object 5. However, any shape can be used as long as the operator can perceive the tilt and the object 5 can be pointed. There may be. Further, the object 5 is also a folder type object in FIG. 3, but it may have any shape as long as it can be pointed by the pointer 4.

  Hereinafter, the pointing method of the first embodiment will be described using the system control apparatus 1 having the configuration shown in FIG. At this time, the system control device 1 first causes the display device 3 to display the object 5 and the pointer 4 using the display control means 104 as shown in FIG. 5 (step 601). At this time, a plurality of the objects 5 may be displayed, and may be displayed at any position within the three-dimensional space represented by the display device 3.

  Next, information input by the operator using the keyboard 201 and the mouse 202 is acquired by the input information acquisition unit 101 (step 602). At this time, the input information acquired by the input information acquiring unit 101 acquires not only information related to the operation of the pointer 4 but also input information that causes the processing control unit 107 to perform processing. Suppose that information about operations is acquired. When the input information acquisition unit 101 acquires input information related to the operation of the pointer 4, it passes the input information to the pointer position / deformation amount calculation unit 102.

  Upon receipt of the input information, the pointer position / deformation amount calculation means 102 first calculates the movement direction, movement amount, etc. of the pointer based on the input information as shown in FIG. 5 (step 603). . In step 603, for example, the movement direction and movement amount of the pointer 4 in the xy plane shown in FIG.

  If the movement direction, movement amount, etc. of the pointer 4 are calculated in the step 603, the pointer 4 displayed on the three-dimensional display device 3 is moved and displayed based on the calculation result (step 604). In step 604, for example, the pointer generation unit 103 generates the destination pointer 4 based on the movement direction, movement amount, etc. of the pointer 4 in the xy plane, and then uses the display control unit 104. The generated pointer 4 is displayed on the three-dimensional display device 3. If the input information does not include information for moving or rotating the pointer 4 in the xy plane, the operation in step 604 is omitted and the processing in the next step 605 is performed.

  When the processing of step 603 is completed, the pointer position / deformation amount calculation means 103 next calculates the rotation direction and rotation angle of the pointer 4 based on the input information (step 605). In step 605, the rotation direction is determined from the rotation direction of the wheel 202A of the mouse 202, for example. The rotation angle is calculated from the rotation angle of the wheel 202A of the mouse 202, for example.

  If the rotation direction and rotation angle of the pointer 4 are calculated in the step 605, the pointer 4 displayed on the three-dimensional display device 3 is rotated and displayed based on the calculation result (step 606). In the step 606, for example, after the pointer 4 is tilted by the pointer generation means 103 based on the rotation angle of the pointer 4, the display control means 104 is used as shown in FIG. 3 and FIG. The tilted pointer 4 is displayed on the display device 3. If the input information does not include information for rotating the pointer 4 in the z-axis direction, the operation in step 606 is omitted and the processing in the next step 607 is performed.

  When the pointer position / deformation amount calculation unit 102 performs the processing of step 603 and step 606, it passes the calculation result to the pointer generation unit 103 and also to the pointing determination unit 105. At this time, the pointing determination means 105 determines whether there is an object 5 to which the pointer 4 after the operation is pointing from the received calculation result (step 607). At this time, if there is no pointing object 5, the process returns to step 602 and waits until the next input information is acquired.

  If there is the pointing object 5, the object generating means 106 generates an object with the color of the pointing object changed, and displays it on the display device 3 (step 608). After displaying the object 5 whose color has been changed, the process returns to step 602 and waits until the next input information is acquired.

  Further, after the pointer 4 is tilted and displayed in the above-described procedure, information related to the operation of the pointer 4 is acquired from the input device. For example, when the pointer 4 is moved in the xy plane, the pointer 4 is tilted. The pointer 4 may be moved after being restored, or may be moved while the pointer 4 is tilted.

  Although not shown, when input information other than information related to the operation of the pointer 4 is acquired in the step 602, the input information acquisition unit 101 passes the acquired input information to the processing control unit 107. At this time, examples of input information other than information related to the operation of the pointer 4 include activation of software associated with the object 5 pointed by the pointer 4 and input information of numerical values or character strings. In this case, the processing control means 107 performs processing such as activation of software associated with the object 5 based on the input information, and displays the processing result on the display device 3 using the display control means 104. Let

  As described above, according to the pointing method of the first embodiment, when the pointer 4 is tilted in the depth direction and displayed on the display device 3, the operator who viewed the pointer 4 can see the depth of the pointer 4. It is possible to intuitively and accurately perceive the position and the depth position at which the pointer 4 is pointing.

  Further, in the first embodiment, the pointer 4 has an arrow shape and is rotated by providing the rotation shaft at the end of the arrowhead side. You may let them.

  Further, in the pointing method of the first embodiment, the example in which the pointer 4 is tilted in the depth direction by combining the control key 201A of the keyboard 201 and the wheel 202A of the mouse 202 has been described. It may be a combination of a key and a wheel 202A, or may be combined with a cursor key instead of the wheel 202A. In addition, for example, it may be inclined when a predetermined operation is performed with a pen tablet, a touch panel, a joystick, or the like.

  6 to 8 are schematic diagrams for explaining an application example of the first embodiment. FIG. 6 is a diagram illustrating an example of displaying a reference, and FIG. 7 is a diagram illustrating an example of translation in the depth direction. FIG. 8 is a diagram showing an example of rotation in the xy plane.

  In the pointing method according to the first embodiment, for example, the pointing position in the depth direction of the pointer 4 can be perceived by looking at the shape of the pointer 4, but for example, when the pointer 4 tilts as shown in FIG. The reference device 7 may be displayed on the display device 3 as an index for allowing the operator to perceive how much it is tilted. The reference 7 may be fixed, or may move as the pointer 4 moves, and may be fixed on the spot when the pointer 4 tilts. Further, the reference 7 may be arranged at any position in the three-dimensional space represented by the display device 3, and the operator can set the arrangement position. In FIG. 6, a display object representing the xyz axis is used as the reference 7. However, the display object is not limited as long as it is an index that allows the operator to perceive the degree of inclination of the pointer 4.

  In the pointing method of the first embodiment, as shown in FIGS. 3 and 4, the method of tilting the pointer 4 and pointing to objects having different depth positions has been described. At this time, the pointer 4 is tilted. In addition, as shown in FIG. 7, an operation of translating in the depth direction while maintaining the shape of the pointer 4 may be added. In this case, for example, when the z-axis direction of the three-dimensional space represented by the display device 3, that is, the depth is large (wide) and the object 5 cannot be pointed by simply tilting the pointer 4, FIG. As shown in FIG. 4, the object 5 can be pointed by moving the pointer 4 in the depth direction. In FIG. 7, an example in which translation is performed in the back (+ z direction) as viewed from the operator is shown, but translation can be performed in front (−z direction). When adding such translational movement, for example, as shown in FIG. 7, the wheel 202A of the mouse 202 is rotated while pressing a key different from the control key 201A of the keyboard 201, for example, the shift key 201B. The pointer 4 may be moved backward (+ z direction) or forward (−z direction).

  In the pointing method of the first embodiment, for example, as shown in FIG. 8, the pointer 4 can be rotated in the xy plane. In this case, for example, an operation different from an operation of tilting the pointer 4 in the depth direction or an operation of moving the pointer 4 in a translational direction, for example, as shown in FIG. The pointer 4 may be rotated in the xy plane by rotating the wheel 202A.

  9 and 10 are schematic diagrams for explaining the pointing method according to the second embodiment of the present invention, FIG. 8 is a diagram for explaining the principle, and FIG. 9 is a diagram for explaining the pointing method. . In addition, in FIG. 10, among the four views under the long horizontal arrow in the drawing, the two drawings shown on the left side of the drawing are the perspective view of the pointer and the object before pointing from the operator's viewpoint. The two views shown on the right side of the drawing are a view when the object is pointed and the pointer and the object as seen from the operator's viewpoint, and a view as seen from diagonally below.

  In the pointing method of the second embodiment, for example, a keyboard 201 is used as the input device 2 as shown in FIG. For example, as shown in FIG. 9, when one of the cursor keys 201D is pressed while a predetermined key such as a control key (Ctrl key) 201A of the keyboard 201 is pressed, the pressed cursor key 201D is pressed. Accordingly, the pointer 4 displayed on the display device 3 rotates and tilts. At this time, it is assumed that xyz three-dimensional coordinates as shown in FIG. 2 are taken on the display device 3, and an operator (observer) is looking at the display device 3 from the −z direction. At this time, as shown in FIG. 9, for example, the pointer 4 rotates around a rotation axis parallel to the x axis or a rotation axis parallel to the y axis. When the movement key is pressed, the pointer 4 tilts around a rotation axis parallel to the x axis so that the left end when viewed from the operator is in the back direction (+ z direction). Similarly, when the rightward movement key is pressed, the pointer 4 is tilted around the rotation axis parallel to the x axis, and the right end when viewed from the operator is inclined in the back direction (+ z direction), and the upward direction. When the movement key is pressed, the pointer 4 is rotated around the rotation axis parallel to the y axis, the upper end when viewed from the operator is inclined in the back direction (+ z direction), and the downward movement key is pressed. In this case, it is assumed that the pointer 4 is tilted around the rotation axis parallel to the y axis and the lower end as viewed from the operator is in the back direction (+ z direction).

  The pointing method according to the second embodiment is based on the ObjA, ObjB, ObjC, ObjD around the pointer 4 on the display device 3 as shown on the left side of FIG. This is suitable for pointing when four objects 5 are displayed. In this state, for example, when the rightward movement key of the cursor key 201D is pressed while pressing the control key 201A of the keyboard 201 as shown in the upper part of the drawing in FIG. 10, the right side of FIG. As shown, the right end of the pointer 4 tilts to face the direction ObjB. At this time, although not shown, if the pointer 4 is a three-dimensional pointer having a thickness in the depth direction, the side surface of the pointer 4 can be seen on the left side of the pointer 4. Therefore, the operator can perceive from the shape of the pointer 4 that the pointer 4 is tilted so that the right end is at the back. At this time, as shown in FIG. 9, if the color of the ObjB where the tip of the pointer 4 approaches is changed, it can be easily perceived that the ObjB is in a pointing state. At this time, the pointer 4 may be displayed so as to be continuously inclined as in the first embodiment, or may be displayed so as to be inclined by an angle Δθ each time the cursor key 201D is pressed.

  In FIG. 10, the diamond-shaped pointer 4 is used for pointing the object. However, any shape can be used as long as the operator can perceive the tilt and can point the object 5. Also good. The object 5 is also a button-type object in FIG. 10, but may have any shape as long as it can be pointed by the pointer 4.

  Hereinafter, the pointing method of the second embodiment will be described using the system control apparatus 1 having the configuration shown in FIG. 1, but the basic processing procedure is the procedure described in the first embodiment, that is, FIG. This is the processing procedure shown. Therefore, detailed description of steps for performing the same processing as that described in the first embodiment is omitted. First, the system control device 1 first causes the display device 3 to display the object 5 and the pointer 4 using the display control means 104 (step 601).

  Next, information input by the operator using the keyboard 201 and the mouse 202 is acquired by the input information acquisition unit 101 (step 602). At this time, if the input information acquired by the input information acquisition unit 101 is information related to the operation of the pointer 4, the input information acquisition unit 101 sends the acquired input information to the pointer position / deformation amount calculation unit 102. hand over.

  Upon receipt of the input information, the pointer position / deformation amount calculation means 102 first calculates the movement direction, amount of movement, etc. of the pointer based on the input information as shown in FIG. 4 (step 603). Based on the calculation result, the pointer 4 displayed on the display device 3 is moved and displayed (step 604). If the input information does not include information for moving or rotating the pointer 4 in the xy plane, the operation in step 604 is omitted and the processing in the next step 605 is performed.

  When the processing of step 603 is completed, the pointer position / deformation amount calculation means 102 next calculates the rotation direction and rotation angle of the pointer 4 based on the input information (step 605). In the case of the pointing method of the second embodiment, when calculating the rotation direction of the pointer 4, which axis of the plurality of axes is rotated around, for example, an axis parallel to the x axis and parallel to the y axis. Also determine which of the two axes is to be rotated. In addition, the rotation angle is calculated from, for example, the number of times the pressed key of the cursor key 202B is pressed or the time during which the key is pressed.

  If the rotation axis, rotation direction, and rotation angle of the pointer 4 are calculated in the step 605, the pointer 4 displayed on the display device 3 is rotated and displayed based on the calculation result (step 606). In step 606, for example, after the pointer 4 is tilted by the pointer generation means 103 based on the rotation angle of the pointer 4, the tilted pointer 4 is displayed on the display device 3 using the display control means 104. Let If the input information does not include information for rotating the pointer 4 in the z-axis direction, the operation in step 606 is omitted and the processing in the next step 607 is performed.

  When the pointer position / deformation amount calculation unit 102 performs the processing of step 603 and step 606, it passes the calculation result to the pointer generation unit 103 and also to the pointing determination unit 105. At this time, the pointing determination means 105 determines whether or not there is an object 5 to which the pointer 4 after the operation is pointing based on the received calculation result (step 607). If there is, the object generation means 106 generates an object in which the color of the object being pointed is changed and displays it on the display device 3 (step 608).

  Further, after the pointer 4 is tilted and displayed in the above-described procedure, information related to the operation of the pointer 4 is acquired from the input device. For example, when the pointer 4 is moved in the xy plane, the inclination of the pointer 4 is changed. The pointer 4 may be moved after being restored, or may be moved while the pointer 4 is tilted.

  Although not shown, when input information other than information related to the operation of the pointer 4 is acquired in the step 602, the input information acquisition unit 101 passes the acquired input information to the processing control unit 107. At this time, examples of input information other than information related to the operation of the pointer 4 include activation of software associated with the object 5 pointed by the pointer 4 and input information of numerical values or character strings. In this case, the processing control means 107 performs processing such as activation of software associated with the object 5 based on the input information, and displays the processing result on the display device 3 using the display control means 104. Let

  As described above, according to the pointing method of the second embodiment, the operator who viewed the pointer 4 moves the pointer 4 by tilting the pointer 4 in the depth direction and displaying the pointer on the 3D display device. The depth position and the depth position to which the pointer 4 is pointing can be intuitively and accurately perceived.

  Further, in the pointing method of the second embodiment, for example, a rhombus pointer 4 as shown in FIG. 9 is displayed and rotated around an axis parallel to the x axis or an axis parallel to the y axis. Although an example is shown in which any one of the four vertices is inclined in the back (+ z direction) when viewed from the operator, the shape of the pointer 4 may be, for example, a hexagon or an octagon. Further, three or more axes for rotating the pointer 4 may be set. For example, when the pointer is hexagonal, three axes are set, and when the pointer is octagonal, four axes are set. One of the vertices can be inclined to the back. The shape of the pointer 4 is not limited to polygons such as rhombuses, hexagons, and octagons as described above, and the operator can perceive the position and inclination of pointing with the pointer 4, and Any shape can be used as long as the object 5 can be pointed to.

  In the pointing method of the second embodiment, the example in which the pointer 4 is tilted in the depth direction by combining the control key 201A and the cursor key 201D of the keyboard 201 has been described. It may be a combination.

  Further, in the pointing method of the second embodiment, as in the first embodiment, the method in which the pointer 4 can be moved in the xy plane has been described. However, the present invention is not limited to this, and there is a point in the xy plane. The pointer 4 may be fixed so that only four buttons around the pointer 4 are pointed. In that case, the processing of step 603 and step 604 is unnecessary in the flow shown in FIG.

  FIG. 11 is a schematic diagram for explaining an application example of the second embodiment and illustrates an application example of the input device.

  In the pointing method of the second embodiment, the example in which the pointer 4 is tilted in the depth direction by combining the control key 201A and the cursor key 201D of the keyboard 201 is shown, but the present invention is not limited to this. As described above, a joypad 203 including a joystick 203A that can be moved in four directions may be used as an input device. In this case, for example, among the four corners of the pointer 4, the end in the direction in which the joystick 203A is tilted may be inclined to the back (+ z direction). The pointer 4 may be tilted by tilting the joystick 203A while pressing a button 203B provided on the joypad 203. An input device such as a game pad having a cross key or the like can be used instead of the joystick 203A.

  Further, in the pointing method according to the second embodiment, not only the keyboard 201 and the joypad 203 but also a tilting can be performed when a predetermined operation is performed with a pen tablet, a touch panel, or the like.

  Although not described in detail, the pointing method according to the second embodiment not only rotates and tilts the pointer 4 in the depth direction but also translates the pointer 4 in the depth direction as shown in FIG. You may be able to perform.

  12 to 15 are schematic diagrams for explaining the pointing method according to the third embodiment of the present invention. FIG. 12 is a diagram for explaining the principle. FIG. 13 is a diagram for explaining the pointing method. 14 (a) and 14 (b) are diagrams for explaining another example of a pointer deformation method, and FIG. 15 is a flowchart for explaining a processing procedure in the system control apparatus. In addition, in FIG. 13, among the nine diagrams under the long horizontal arrow in the drawing, the three diagrams shown at the left end of the drawing are a view of the pointer and the object before pointing from the viewpoint of the operator. The figure seen from above and the figure seen from the right side. The three figures shown in the center of the page are the figure when the pointer and the object are viewed from the operator's point of view when the pointer is operated. The figure is a view from the right side, and the three figures shown at the right end of the page are the pointer and object viewed from the operator's point of view when the object is pointed, the view from the upper right, and the right side. It is the figure seen from the surface.

  In the pointing method of the third embodiment, for example, a keyboard 201 and a mouse 202 are used as the input device 2 as shown in FIG. For example, as shown in FIG. 12, when the wheel 202A of the mouse 202 is rotated while pressing a predetermined key 201A such as a control key (Ctrl key) of the keyboard 201, the rotation direction of the wheel 202A is rotated. Depending on the number of rotations, the pointer 4 displayed on the display device 3 is deformed into a shape in which only a part is translated in the depth direction. At this time, it is assumed that xyz three-dimensional coordinates as shown in FIG. 12 are taken on the display device 3, and an operator (observer) is looking at the display device 3 from the −z direction. At this time, the deformed portion of the pointer 4 is, for example, either the back direction (+ z direction) or the near direction (−z direction) as viewed from the operator in accordance with the rotation direction of the wheel 202A. To move to.

  In the pointing method according to the third embodiment, even if the pointer 4 and the object 5 overlap with each other as viewed from the operator's viewpoint on the display device 3, for example, as shown at the left end of FIG. Are different, the object 5 is not pointed. In this state, for example, when the control key 201A of the keyboard 201 is pressed and the wheel 202A of the mouse 202 is rotated toward the back as viewed from the measurer, the pointer 4 is moved to the center of FIG. As shown in (2), the arrow portion changes into a stepped shape that translates in the direction of the object 5 (+ z direction). At this time, the operator sees that the tip of the pointer 4, in other words, the width of the arrow portion is narrower than the width of the arrowhead side. Therefore, the operator can perceive from the shape of the pointer 4 that the pointer 4 is pointing deeper than the position before the operation. Here, when the wheel 202A of the mouse 202 is further rotated toward the back as viewed from the measurer while pressing the control key 201A of the keyboard 201, as shown at the right end of FIG. The arrow portion of the pointer 4 has a shape that is further translated in the direction of the object 5. At this time, it appears to the operator that the width of the tip of the pointer 4 is further narrowed. At this time, as shown in FIG. 13, when the tip of the pointer 4 comes into contact with the object 5, the object 5 is in a state pointed to by the pointer 4. Therefore, for example, the color of the object 5 is changed to indicate that it is in a pointing state. In this way, the operator can intuitively and accurately perceive the position in the depth direction of the pointer from the shape of the pointer 4 and the position in the depth direction where the pointer 4 is pointing. In addition, by changing the color of the object 5 when it is pointed by the pointer 4, it is possible to intuitively and accurately perceive whether or not the pointer 4 overlapping the object 5 is pointing to the object 5. . Although detailed description is omitted, in the case of the pointing method of the third embodiment, as described in the first embodiment, for example, when the wheel 202A is rotated in the reverse direction while pressing the control key 201A, as shown in FIG. The pointer 4 is deformed into a stepped shape in which the arrow portion is translated forward (-z direction).

  At this time, for example, the pointer 4 may be displayed so as to continuously move in accordance with the total rotation angle of the wheel 202A of the mouse 202, or in synchronization with the rotation step of the wheel 202A. The pointer 4 may be displayed so as to move by Δz in the depth direction every time it is rotated by one step.

  In FIG. 13, the arrow-shaped pointer 4 is used for pointing the object 5. However, any shape can be used as long as the operator can perceive the tilt and can point the object 5. There may be. The object 5 is also a folder type object in FIG. 13, but may have any shape as long as it can be pointed by the pointer 4.

  Further, in FIG. 13, the pointer 4 is deformed so as to have a staircase shape when viewed from the side, but not limited to this, for example, as shown in FIG. The arrow portion may be deformed so that it is curved, or as shown in FIG. 14B, only the arrow portion of the pointer 4 is tilted backward (+ z direction) or near (−z direction). You may deform | transform into such a shape.

  Hereinafter, the pointing method of the first embodiment will be described using the system control apparatus 1 having the configuration shown in FIG. At this time, the system control device 1 first displays the object 5 and the pointer 4 on the three-dimensional display device 3 using the display control means 104 as shown in FIG. 15 (step 601). At this time, a plurality of the objects 5 may be displayed, and may be displayed at any position within the three-dimensional space represented by the display device 3.

  Next, information input by the operator using the keyboard 201 and the mouse 202 is acquired by the input information acquisition unit 101 (step 602). At this time, the input information acquired by the input information acquiring unit 101 acquires not only information related to the operation of the pointer 4 but also input information that causes the processing control unit 107 to perform processing. Suppose that information about operations is acquired. When the input information acquisition unit 101 acquires input information related to the operation of the pointer 4, it passes the input information to the pointer position / deformation amount calculation unit 102.

  Upon receipt of the input information, the pointer position / deformation amount calculation means 101 first calculates the direction and amount of movement of the pointer based on the input information as shown in FIG. 14 (step 603). . In step 603, for example, the movement direction and movement amount of the pointer 4 in the xy plane shown in FIG.

  If the movement direction, movement amount, etc. of the pointer 4 are calculated in the step 603, the pointer 4 displayed on the three-dimensional display device 3 is moved and displayed based on the calculation result (step 604). In step 604, for example, the pointer generation unit 103 generates the destination pointer 4 based on the movement direction, movement amount, etc. of the pointer 4 in the xy plane, and then uses the display control unit 104. The generated pointer 4 is displayed on the display device 3. If the input information does not include information for moving or rotating the pointer 4 in the xy plane, the operation in step 604 is omitted and the processing in the next step 605 is performed.

  When the processing of step 603 is completed, the pointer position / deformation amount calculation unit 103 next calculates the deformation amount of the pointer 4, that is, the movement direction and movement amount of the pointer 4 ahead based on the input information. (Step 609). In step 609, the moving direction is determined from, for example, the rotating direction of the wheel 202A of the mouse 202. The amount of movement is calculated from the rotation angle of the wheel 202A of the mouse 202, for example.

  If the deformation amount of the pointer 4 is calculated in the step 609, the pointer 4 displayed on the display device 3 is deformed and displayed based on the calculation result (step 610). In step 610, for example, the pointer generation unit 103 generates a pointer in which only the tip portion of the pointer 4 is moved in the depth direction based on the movement direction and movement amount of the pointer 4, and then displays the display control unit 104. Utilizing this, the generated pointer 4 is displayed on the display device 3. If the input information does not include information for moving the pointer 4 in the z-axis direction, the operation in step 610 is omitted, and the next step 607 is performed.

  When the pointer position / deformation amount calculation unit 102 performs the processing of step 603 and step 606, it passes the calculation result to the pointer generation unit 103 and also to the pointing determination unit 105. At this time, the pointing determination means 105 determines whether there is an object 5 to which the pointer 4 after the operation is pointing from the received calculation result (step 607). If there is no object 5 pointing at this time, the process returns to step 602 and waits until the next input information is acquired.

  If there is the pointing object 5, the object generating means 106 generates an object with the color of the pointing object changed, and displays it on the display device 3 (step 608). After displaying the object 5 whose color has been changed, the process returns to step 602 and waits until the next input information is acquired.

  Further, after displaying a part of the pointer 4 deformed in the depth direction in the procedure as described above, and obtaining information on the operation of the pointer 4 from the input device, for example, when moving in the xy plane, The pointer 4 may be moved after returning to its original shape, or the pointer 4 may be moved while being deformed.

  Although not shown, when input information other than information related to the operation of the pointer 4 is acquired in the step 602, the input information acquisition unit 101 passes the acquired input information to the processing control unit 107. At this time, examples of input information other than information related to the operation of the pointer 4 include activation of software associated with the object 5 pointed by the pointer 4 and input information of numerical values or character strings. In this case, the processing control means 107 performs processing such as activation of software associated with the object 5 based on the input information, and displays the processing result on the display device 3 using the display control means 104. Let

  As described above, according to the pointing method of the third embodiment, only the part of the pointer 4 is deformed in the depth direction and displayed on the display device 3 so that the operator who has viewed the pointer 4 can The depth position of the pointer 4 and the depth position to which the pointer 4 is pointing can be intuitively and accurately perceived.

  In the pointing method of the third embodiment, the example in which the pointer 4 is tilted in the depth direction by combining the control key 201A of the keyboard 201 and the wheel 202A of the mouse 202 has been described. It may be a combination of a key and a wheel 202A, or may be combined with a cursor key instead of the wheel 202A.

  FIG. 16 is a schematic diagram for explaining an application example of the third embodiment, and is a diagram illustrating an application example of the input device.

  In the pointing method according to the third embodiment, the example in which the pointer 4 is deformed in the depth direction by combining the control key 201A of the keyboard 201 and the wheel 202A of the mouse 202 has been described. For example, as shown in FIG. 16, a pen tablet including a tablet 204A and an input pen 204B may be used as an input device. When the pen tablet is used, for example, when the input pen 204B is slid on the tablet 204A, the pointer according to the movement amount in the X-axis direction and the movement amount in the Y-axis direction of the input pen 204B 4 is moved in the xy plane on the three-dimensional space represented by the display device 3. The pen tablet generally senses the movement of the input pen 204B by an electromagnetic induction method, and can detect pressure (writing pressure) in addition to the position and amount of movement of the input pen 204B. Therefore, for example, the deformation amount of the pointer 4 can be controlled in accordance with the magnitude of the pressure when the input pen 204B is pressed against the tablet 204A. At this time, for example, when the input pen 204B is pressed with the tablet 204A while pressing the control key 201A of the keyboard 201, the arrow portion of the pointer 4 is deformed to the back, and a key different from the control key 201A is used. If the input pen 204B is pressed with the tablet 204A while pressing the key, if the arrow part of the pointer 4 is deformed to the front, both the object at the back of the pointer 4 and the object at the front can be pointed. it can. When using the pen tablet, for example, instead of the magnitude of the pressure (writing pressure) of the input pen 204B, the amount of deformation of the pointer according to the number of times the tablet 204A is pressed with the input pen 204B, for example. May be determined.

  Further, in the pointing method of the third embodiment, when the pen tablet is used, for example, the tablet 204A is integrated with the display device 3 without being limited to the external pen tablet as shown in FIG. May be provided. If the tablet 204A is integrated with the display device 3, for example, the pointer 4 can be operated by touching the input pen 204B on the display screen of the display device 3, so that the operator Can operate the pointer 4 with an operation feeling close to real space.

  In addition, the pointing method of the third embodiment is also suitable for pointing using an input device that can be integrated with the display device 3, such as a touch panel, instead of the pen tablet. In the case of the touch panel, for example, the pointer 4 can be operated by touching the screen of the display device 3 with an operator's finger instead of the input pen 204B. Therefore, the touch panel is more than a pen tablet using the input pen 204B. The pointer 4 can be operated with an operation feeling closer to real space.

  Further, the input device 2 is not limited to the combination of the keyboard 201, the mouse 202, and the pen tablet, and for example, a joy pad or a game pad can be used.

  Although the detailed description is omitted, the pointing method according to the third embodiment not only deforms only a part of the pointer 4 in the depth direction but also maintains the shape as shown in FIG. An operation for translational movement in the depth direction may be performed.

  FIG. 17 is a schematic diagram for explaining the pointing method according to the fourth embodiment of the present invention, and is a schematic diagram for explaining the principle of a pointer display method when a DFD is used as a display device.

  The pointing method described in the first to third embodiments may be any display device as long as the display device can express a three-dimensional space. It is preferable to use a three-dimensional display device (display). For example, as shown in FIG. 17, the DFD is a display device including a plurality of display surfaces that overlap in the depth direction when viewed from the viewpoint of the operator (observer) (for example, Japanese Patent No. 3022558 and (See Japanese Patent No. 3460671). In the DFD, for example, as shown in FIG. 17, the pointer 4 is displayed as two-dimensional images 4A and 4B projected onto the display surfaces 3A and 3B from the viewpoint of the operator. Then, a stereoscopic visual effect is obtained by changing the luminance ratio of the points (pixels) on the display surfaces 3A and 3B that overlap when viewed from the viewpoint of the operator. At this time, for example, as shown in FIG. 17, the brightness of the two-dimensional pointer image 4A displayed on the display surface 3A in front of the operator increases from the top to the bottom of the paper. It is assumed that the two-dimensional pointer image 4B that is displayed and displayed on the display surface 3B in the back is displayed so that the luminance increases from the bottom to the top of the page. Further, regarding the overlapping points from the viewpoint of the operator, it is assumed that the brightness of the back display surface 3B is higher on the upper side of the paper and the brightness of the front display surface 3A is higher on the lower side of the paper. When the display is performed using the display control means 104, the operator sees the pointer 4 whose upper side is inclined to the back as shown in FIG. Therefore, the operator can perform an object pointing operation while perceiving the stereoscopic image of the pointer 4.

  Further, when the DFD is used, instead of controlling the luminance of the points (pixels) on the display surfaces 3A and 3B, for example, the transmittance of each point can be controlled. In this case, for example, as shown in FIG. 17, the front display surface 3A has a lower transmittance from the top to the bottom, and the back display surface 3B has a lower transmittance from the bottom to the top. If it does in this way, the paper surface lower side of 3 A of this front side will not permeate | transmit the color of the point (pixel) of the back display surface 3B. Therefore, the operator can see the pointer 4 whose upper side is inclined to the back.

  In addition, in the case of a two-dimensional display device such as a general CRT display or a liquid crystal display, a process for two-dimensionally converting the three-dimensional space to be expressed is necessary, but a three-dimensional display device such as the DFD is required. In this case, it is only necessary to set the luminance ratio of points (pixels) on each display surface according to the position in the depth direction of the three-dimensional space, so that the load on the pointing device (system control device 1) is reduced. it can. Also, in the case of a two-dimensional display device such as a general CRT display or a liquid crystal display, the three-dimensional space to be displayed is converted two-dimensionally. Although it may be difficult to do so, a pointing operation can be performed with a sense closer to real space by using a three-dimensional display device such as the DFD. From these facts, by using a 3D display device such as the DFD, the operator can perform the 3D operation with better accuracy and speed than the case of performing a pointing operation using a general 2D display. Dimension depth can be pointed.

  In the fourth embodiment, a DFD type display device capable of three-dimensional stereoscopic display is taken as an example of the display device 3. However, if the operator can perceive a three-dimensional solid, another display device may be used. There may be.

  The present invention has been specifically described above based on the above-described embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention. is there.

  For example, the pointing method as described in the first to third embodiments is described as a program to be executed by the computer, installed in the computer, and executed. The pointing method of the invention can be realized. At this time, for example, the program can be provided by being recorded on a magnetic, electrical, or optical recording medium, or can be provided via a network such as the Internet.

It is a schematic diagram which shows the structural example of the system for implement | achieving the pointing method of this invention. It is a schematic diagram for demonstrating the pointing method of Example 1 by this invention, and is a figure for demonstrating a principle. It is a schematic diagram for demonstrating the pointing method of Example 1 by this invention, and is a figure for demonstrating the pointing method of the object behind a pointer. It is a schematic diagram for demonstrating the pointing method of Example 1 by this invention, and is a figure for demonstrating the pointing method of the object in front of a pointer. It is a schematic diagram for demonstrating the pointing method of Example 1 by this invention, and is a flowchart for demonstrating the process sequence in the said system control apparatus. It is a schematic diagram for demonstrating the application example of the present Example 1, and is a figure which shows the example which displays a reference. It is a schematic diagram for demonstrating the application example of the present Example 1, and is a figure which shows the example which translates in the depth direction. It is a schematic diagram for demonstrating the application example of the present Example 1, and is a figure which shows the example rotated within xy plane. It is a schematic diagram for demonstrating the pointing method of Example 2 by this invention, and is a figure for demonstrating a principle. It is a schematic diagram for demonstrating the pointing method of Example 2 by this invention, and is a figure for demonstrating a pointing method. It is a schematic diagram for demonstrating the application example of the present Example 2, and is a figure which shows the application example of an input device. It is a schematic diagram for demonstrating the pointing method of Example 3 by this invention, and is a figure for demonstrating a principle. It is a schematic diagram for demonstrating the pointing method of Example 3 by this invention, and is a figure for demonstrating a pointing method. FIGS. 14A and 14B are schematic diagrams for explaining a pointing method according to the third embodiment of the present invention, and FIGS. 14A and 14B are diagrams for explaining modifications of the shape of the pointer. It is a schematic diagram for demonstrating the pointing method of Example 3 by this invention, and is a flowchart for demonstrating the process sequence in the said system control apparatus. It is a schematic diagram for demonstrating the application example of the present Example 3, and is a figure which shows the application example of an input device. It is a schematic diagram for demonstrating the pointing method of Example 4 by this invention, and is a schematic diagram explaining the principle of the display method of the pointer at the time of using DFD as a display apparatus.

Explanation of symbols

1. System control device (pointing device)
DESCRIPTION OF SYMBOLS 101 ... Input information acquisition means 102 ... Pointer position / deformation amount calculation means 103 ... Pointer generation means 104 ... Display control means 105 ... Pointing determination means 106 ... Object generation means 107 ... Processing control means 108 ... Storage means 2 ... Input device 201 ... Keyboard 202 ... Mouse 203 ... Joypad 204A ... Tablet 3 ... Display device 3A, 3B ... Display surface 4, 4A, 4B ... Pointer 5 ... Object

Claims (18)

A pointer displayed on a display device capable of expressing a three-dimensional space is moved or rotated in a two-dimensional plane, and is moved in a normal direction (hereinafter referred to as a depth direction) of the two-dimensional plane. A pointing method for pointing an arbitrary position in a three-dimensional space represented on a display device,
Moving or rotating the pointer in the two-dimensional plane to display on the display device; and moving the pointer in the depth direction to display on the display device.
The step 2 includes a step 3 in which the depth position of one end of the pointer is changed and displayed on the display device while maintaining the depth position of a part of the pointer.   2. The pointing method according to claim 1, wherein the step 2 includes a step 4 of translating in the depth direction while the shape of the pointer is maintained and displaying on the display device in addition to the step 3. 3.   3. The pointing according to claim 1, wherein the step 3 displays the pointer on the display device by rotating the pointer by a predetermined angle around an axis parallel to the two-dimensional plane. 4. Method.   4. The pointing according to claim 3, wherein in the step 3, the pointer is displayed on the display device by rotating around one axis or two or more axes of the plurality of axes. 5. Method.   3. The pointing method according to claim 1, wherein in the step 3, only a part of the pointer is deformed into a shape moved in the depth direction and displayed on the display device. 4.   6. The method according to claim 3, wherein in the step 3, the pointer is rotated or deformed stepwise based on operation information input from the input device and displayed on the display device. The pointing method according to item.   The pointing method according to any one of claims 1 to 6, wherein a reference pointer that does not change a position in the depth direction is displayed on the display device together with the pointer.   The pointer projects from a viewpoint of the operator onto a plurality of display surfaces overlapping in the depth direction when viewed from the operator, and indicates a ratio of luminance of the points on the display surfaces that overlap when viewed from the viewpoint. The pointing method according to claim 1, wherein the pointing method is changed and displayed. A pointer displayed on a display device capable of expressing a three-dimensional space is moved or rotated in a two-dimensional plane, and is moved in a normal direction (hereinafter referred to as a depth direction) of the two-dimensional plane. A pointing device for pointing an arbitrary position in a three-dimensional space represented on a display device,
Input information acquisition means for acquiring operation information of the pointer input from an input device;
Based on the operation information acquired by the input information acquisition means, the pointer position for calculating the movement amount or rotation angle of the pointer in the two-dimensional plane and calculating the movement amount of the pointer in the depth direction / Deformation amount calculating means;
Means for translating or rotating the pointer based on the movement amount or rotation angle of the pointer in the two-dimensional plane calculated by the pointer position / deformation amount calculation means, and movement of the pointer in the calculated depth direction Pointer generating means comprising means for generating a pointer with the depth position of one end of the pointer changed based on the amount while maintaining the depth position of a part of the pointer;
A pointing device comprising: display control means for displaying the pointer generated by the pointer generating means on the display device.   The pointing device according to claim 9, wherein the pointer generation unit includes a unit that expands or reduces the pointer while maintaining the shape of the pointer in addition to the units.   The means for generating a pointer in which the depth position of one end of the pointer is changed while maintaining a depth position of a part of the pointer is a predetermined angle around an axis parallel to the two-dimensional plane. The pointing device according to claim 9 or 10, wherein the pointing device is rotated only by the rotation amount.   Means for generating a pointer in which the depth position of one end of the pointer is changed while maintaining the depth position of a part of the pointer is configured to cause the pointer to be one of a plurality of axes parallel to the two-dimensional plane. The pointing device according to claim 11, wherein the pointing device is rotated about a single shaft or two or more shafts.   The means for generating a pointer in which the depth position of one end of the pointer is changed while maintaining the depth position of a part of the pointer is deformed into a shape in which only a part of the pointer is moved in the depth direction. The pointing device according to claim 9 or 10.   Means for generating a pointer in which the depth position of one end of the pointer is changed while maintaining a depth position of a part of the pointer rotates the pointer stepwise based on operation information input from an input device The pointing device according to claim 11, wherein the pointing device is deformed.   The pointing device according to claim 9, wherein the display control unit causes the display device to display a reference pointer that does not change a position in the depth direction together with the pointer. .   The display control means displays pointers projected from the viewpoint of the operator on a plurality of display surfaces that overlap in the depth direction when viewed from the operator, and on each display surface that overlaps when viewed from the viewpoint. The pointing device according to any one of claims 9 to 15, wherein the display is performed by changing a luminance ratio of points.   In addition to each means, the pointer is based on the movement amount or rotation angle of the pointer in the two-dimensional plane calculated by the pointer position / deformation amount calculation means, and the movement amount of the pointer in the depth direction. A pointing determination means for determining whether or not there is an object pointed by the pointer when the object is moved, and generating an object that is changed to a pointing state when the pointing object is present The pointing device according to claim 9, further comprising: an object generation unit that performs the operation. A pointing program for causing a computer to execute the pointing method according to claim 1.

Images