JP2007140186A - Three-dimensional pointing method, three-dimensional pointing device, and three-dimensional pointing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable an operator to quickly and accurately perceive a pointing position and to make the operator perform pointing by using a pointer having such a natural shape that his or her work is not hindered, with respect to pointing work using the pointer in three-dimensional GUI. <P>SOLUTION: A three-dimensional pointing method of pointing one arbitrary point of a three-dimensional display space expressed on an external display device includes; a step 1 of calculating two-dimensional plane coordinates and a depth position of a point to be pointed in the three-dimensional display space, on the basis of input information from an external input device; a step 2 of generating a pointer having a first portion indicating the two-dimensional plane coordinates of the point to be pointed, a second portion indicating the depth position of the point to be pointed, and a third portion of which the position in a two-dimensional plane is varied in accordance with the depth position; and a step 3 of displaying the generated pointer on the external display device. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a three-dimensional pointing method, a three-dimensional pointing device, and a three-dimensional pointing program, and in particular, applied to pointing in a display device capable of presenting a three-dimensional stereoscopic image such as a DFD (Depth-Fused 3-D) display device. It relates to effective technology.

  Conventionally, GUI (Graphical User Interface) has more display, function, and operation than ever before due to remarkable improvement in computing performance of PC (Personal Computer), speeding up of processing speed, enhancement of graphic function, etc. It is getting complicated. For this reason, when an operator performs a desired operation, they may become a toe and prevent efficient operation.

  In order to improve such a state, in recent years, it has been proposed to present and manipulate information using a three-dimensional display space (see, for example, Non-Patent Document 1). This is often called a three-dimensional GUI, and is a mechanism in which an object is arranged three-dimensionally in a three-dimensional display space and the object is operated using a predetermined input device. In addition to the 3D GUI, a similar mechanism may be used in, for example, CAD (Computer Aided (or Assisted) Design) and CG (Computer Graphics) for designing in a 3D display space. However, from the viewpoint of manipulating and pointing to an object arranged three-dimensionally, the discussion will be made with a three-dimensional GUI as an example.

  Using the 3D GUI, it is possible to arrange the objects that have been arranged two-dimensionally or stacked up to now in a three-dimensional manner from the viewpoint of the operator, and the work space can be handled efficiently. it can. In addition, since the real world surrounding us is a three-dimensional space, the GUI can be handled more intuitively than a two-dimensional GUI by making the GUI three-dimensional.

  However, one of the problems that are required in the process of operating the three-dimensional GUI is pointing to objects at various depth positions. In the conventional technology, a new function necessary for moving in the depth direction is added to the operation with the mouse, keyboard, joystick, etc. that has been used in the two-dimensional GUI, so that the pointer within the three-dimensional display space The movement is realized. The position of the pointer in the depth direction is expressed by, for example, enlarging or reducing the pointer, and pointing to an object in the back of the three-dimensional display space (see, for example, Non-Patent Document 2). That is, there has been proposed a method of displaying a large pointer when the pointer is in front in the three-dimensional display space and displaying a small pointer when it is in the back.

  However, in many of these methods, since the pointer displayed in the three-dimensional display space can freely move in the depth direction in the space, the pointer is displayed on the display device and the two-dimensional position is easily understood. However, it is difficult for the operator to clearly perceive which depth position the integrated pointer is and where in the three-dimensional space is pointing.

  As an example of a method for solving this problem, for example, there is a method of displaying a reference such as an xyz axis whose depth position does not change near the pointer so that the depth position can be easily perceived (see, for example, Patent Document 1). ). However, the displayed reference generally has nothing to do with the shape of the pointer, except that it is a clue to the depth position of the pointer. For this reason, it is unnatural, and the reference appears on the display surface, thereby obstructing the operator's view and reducing the work efficiency.

From the above, in the pointing work using the pointer in the three-dimensional GUI, the operator can quickly and accurately perceive where the pointer is located and where the pointer is pointing. There has been a demand for a pointing method that can naturally point without interfering.
JP-A-8-248938 G. Robertson, et al., "The Task Gallery: A 3D Window Manager," CHI Letters, Vol.2, pp.494-501, 2000. Keita Watanabe, Michiaki Yasumura, “RUI: Realization of Information Realization with User Interface Cursor,” Human Interface Society Research Report, vol.6, No.2, pp.35-38, 2004.

  One of the problems to be solved by the present invention is that, as described in the background art, in the pointing operation using the pointer in the conventional three-dimensional GUI, the pointer is located at any depth position in the three-dimensional display space. Therefore, it is difficult for an operator to intuitively and accurately perceive where he is pointing.

  Another problem is that the use of an unnatural reference having a low relevance to the shape of the pointer obstructs the operator's field of view, and in some cases reduces the work efficiency.

  The object of the present invention is to enable the operator to quickly and accurately perceive the depth position of the pointer and the pointing position in the pointing operation using the pointer in the three-dimensional GUI, and further the operation of the operator It is an object of the present invention to provide a pointing method and a pointing device capable of pointing using a pointer having a natural shape that does not hinder the movement.

  The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  In the 3D pointing method of the present invention, when a 3D GUI is displayed on a display device capable of 3D display and an object placed at an arbitrary depth position in the 3D GUI is pointed using a pointer, Point the two-dimensional coordinates with at least one part of the pointer, and point the depth position with at least one part. At the same time, set the position of the at least one part in the two-dimensional plane to the pointing depth position. It is characterized by being changed accordingly. This allows the operator to perceive not only the pointing two-dimensional coordinates but also the depth position.

  Further, when the pointer approaches the object that can be pointed within a certain distance on the two-dimensional projection plane, the depth position of the object that can be pointed is changed to the position of the pointer whose position in the two-dimensional plane changes according to the depth position. An object indicating a position in the two-dimensional plane corresponding to the depth position is displayed at a position where the position on the two-dimensional projection plane matches at least one part. As a result, the operator can quickly and accurately grasp the depth position of the objects arranged in the three-dimensional space, and can quickly and intuitively point these objects.

  At this time, the object indicating the position in the two-dimensional plane corresponding to the depth position is given the shape, color, size, orientation, movement, etc. of the object that can be pointed. Thereby, the operator can grasp | ascertain the depth position of the object arrange | positioned in dimension space more rapidly and correctly. Further, even when the objects overlap in the three-dimensional space, the operator can point to the target object without hesitation.

  Furthermore, in the three-dimensional pointing method of the present invention, a part serving as a reference for knowing the depth position is provided on the pointer itself. This reference part has a fixed depth position when performing pointing, and by comparing it with the part where the depth of the pointer changes, the depth position being pointed can be presented to the operator more quickly and easily. .

  The outline of the three-dimensional pointing method, the three-dimensional pointing device, and the three-dimensional pointing program of the present invention based on the above points will be described as follows.

  (1) A three-dimensional pointing method for pointing an arbitrary point in a three-dimensional display space represented on an external display device, wherein the point of the point in the three-dimensional display space is pointed based on input information from the external input device. Calculating a two-dimensional plane coordinate and a depth position; a first part indicating a two-dimensional plane coordinate of the point to be pointed; a second part indicating a depth position at the point to be pointed; and the depth position. Accordingly, a three-dimensional pointing method includes a step 2 for generating a pointer having a third portion whose position in the two-dimensional plane changes in response, and a step 3 for displaying the generated pointer on an external display device.

  (2) In the step (1), when the distance between the pointer and the object that can be pointed on the two-dimensional projection plane approaches within a certain distance, the depth position of the object that can be pointed Is a three-dimensional pointing method for generating an object indicating a position in a two-dimensional plane corresponding to the depth position where the third part is to be displayed.

  (3) In the above (2), in addition to the steps 1 to 3, at least one of the shape, color, size, orientation, and movement of the object indicating the depth position of the object that can be pointed is selected. , A three-dimensional pointing method comprising a step 4 of changing in accordance with an attribute of the object that can be pointed.

  (4) In any one of (1) to (3), the step 2 is a three-dimensional pointing method in which the size of the pointer is changed and displayed when an arbitrary point in the three-dimensional space is pointed. .

  (5) In any one of the above (1) to (4), in the step 2, the position in the depth direction is fixed in addition to each part of the first part to the third part. This is a three-dimensional pointing method for generating a fourth part that can compare the depth position with another part whose position changes.

  (6) A three-dimensional pointing device for pointing an arbitrary point in a three-dimensional display space represented on the external display device, the input information acquiring unit for acquiring input information from the external input device, and the input information acquiring unit The pointing position calculation means for calculating the two-dimensional plane coordinates and the depth position of the point to be pointed in the three-dimensional display space based on the input information acquired in step (b), and the pointing based on the calculation result of the pointing position calculation means. Pointer generating means for generating a pointer indicating the two-dimensional plane coordinate position and depth position of the point; and display control means for displaying the pointer generated by the pointer generating means on the external display device, the pointer generating means, A first portion indicating a two-dimensional plane coordinate of the point to be pointed; A second portion indicating a depth position in the point of coating in a three-dimensional pointing apparatus for generating a pointer and a third portion which is located in a two-dimensional plane is changed in accordance with the 該奥 bound position.

  (7) In the above (6), when the distance between the pointer and the object that can be pointed on the two-dimensional projection plane is within a certain distance, the pointer generation unit determines the depth of the object that can be pointed to A three-dimensional pointing device comprising means for generating an object indicating a position in a two-dimensional plane corresponding to the depth position at which the third part is to be displayed.

  (8) In the above (7), in addition to the above means, at least one of the shape, color, size, orientation, and movement of the object indicating the depth position of the pointing object is set as the pointing object. It is a three-dimensional pointing device provided with the object production | generation means to change according to an attribute.

  (9) In any one of (6) to (8), the pointer generation means is a three-dimensional pointing device including means for changing the size of the pointer.

  (10) In any one of (6) to (9), the pointer generation means has a fixed position in the depth direction in addition to the first part to the third part. This is a three-dimensional pointing device that generates a fourth part capable of comparing the depth position with other parts whose positions change.

  (11) A three-dimensional pointing program for causing a computer to execute the processing operation of each means in the pointing device according to any one of (6) to (10).

  The three-dimensional pointing method of the present invention includes, for example, as in the means (1), a first part indicating a two-dimensional plane coordinate of a point to be pointed, a second part indicating a depth position at the point to be pointed, A pointer having three parts of the third part whose position in the two-dimensional plane changes according to the depth position is generated and displayed on the external display device. Therefore, the operator can perceive the two-dimensional coordinate position pointing at the first part, and can perceive the depth position at the second part and the third part. As described above, the position of the at least one part (third part) in the two-dimensional plane changes according to the depth position at which the pointing is performed, so that only the two-dimensional coordinates at which the operator is pointing are used. In addition, the depth position can be pointed easily.

  In addition, for example, as in the means (2), when the pointer approaches a pointable object within a certain distance on the two-dimensional projection plane, an object indicating the depth position of the pointable object is displayed. Thus, the operator can quickly and accurately grasp the depth positions of the objects arranged in the three-dimensional space, and can quickly and intuitively point these objects.

  Further, for example, like the means (3), an object indicating a position in a two-dimensional plane corresponding to the depth position has a shape, a color, a size, a direction, a movement, and the like of the object that can be pointed. Thus, the operator can grasp the depth position of the object arranged in the three-dimensional space more quickly and accurately. In addition, even when objects overlap in a three-dimensional space, the operator can point to the target object without hesitation.

  Further, for example, when the size of the pointer is changed and displayed as in the means (4), for example, when the target object is larger than the pointer, the pointer can be enlarged and displayed. Without hiding, the operator can accurately and quickly grasp the depth position of the target object. Therefore, the operator can easily point to an object arranged in the three-dimensional space, and the operation efficiency and operability are improved.

  Still further, for example, if a pointer having a fourth part whose position in the depth direction is fixed as in the means (5), the fourth part can be used as a reference. In this reference, the depth position is fixed when performing pointing, and by comparing it with the part where the depth of the pointer changes, it is possible to present the pointing depth position to the operator more quickly and easily. .

  The pointer generated by the three-dimensional pointing method according to the present invention has a shape that can represent the two-dimensional coordinates and the depth position at the same time, and at the same time, a reference that is a clue to the depth position is naturally used as the pointer shape. It differs from the conventional pointing method in that it has a simple shape. Further, the depth position of the object that can be pointed is set to the position where the position on the two-dimensional projection plane of at least one portion of the pointer whose position in the two-dimensional plane changes according to the depth position is the same. The point which displays and shows the object which shows the position in the two-dimensional plane corresponding to is also different from the conventional pointing method. Further, the object indicating the position in the two-dimensional plane corresponding to the depth position is given an attribute of a pointing object, which is different from the conventional pointing method.

  In order to realize the three-dimensional pointing method such as the means (1) to the means (5), for example, a three-dimensional pointing device including means such as the means (6) to the means (10) is used. Good. The three-dimensional pointing device can also use a general-purpose computer such as a PC. In this case, a program that causes the computer to execute processing operations performed by each means of the three-dimensional pointing device, such as the means (11). Prepare it and let it run on your computer.

Hereinafter, the present invention will be described in detail together with embodiments (examples) with reference to the drawings.
In all the drawings for explaining the embodiments, parts having the same function are given the same reference numerals and their repeated explanation is omitted.

FIG. 1 is a schematic diagram showing a configuration example of a system for realizing the three-dimensional pointing method of the present invention.
In FIG. 1, 1 is a system control device, 101 is input information acquisition means, 102 is pointing position calculation means, 104 is pointer generation means, 105 is display control means, 106 is pointing determination means, 107 is object generation means, and 108 is Processing control means, 109 is storage means, 103 is a pointing possible point determination means, 2 is an input device, and 3 is a display device.

  The three-dimensional pointing method of the present invention is a method of pointing an arbitrary point in a three-dimensional display space represented on an external display device connected to a system control device such as a PC (Personal Computer). At this time, an object (pointer) for pointing in the three-dimensional display space is three-dimensionally operated using an external input device connected to the system control device.

  At this time, for example, as shown in FIG. 1, the system control apparatus 1 includes an input information acquisition unit 101 that acquires input information input from the external input device 2, and the input information acquired by the input information acquisition unit 101 is a pointer. The pointing position calculating means 102 for calculating the moving direction and the moving amount of the pointing point based on the input information, and pointing is possible based on the calculation result of the pointing position calculating means 102. Pointable point determination means 103 for determining whether a point or an object exists, pointer generation means 104 for generating a pointer based on the determination of the pointable point determination means 103, and the pointer generated by the pointer generation means 104 Display control means 105 to be displayed on the external display device 3; Provided.

  In addition to the above-described units 101 to 105, the system control apparatus 1 uses, for example, the coordinates of the point that the pointer generated based on the determination by the pointing possible point determination unit 103 indicates as shown in FIG. For example, a pointing determination unit 106 that determines whether or not there is an object, and an object generation unit 107 that changes the color of the object when the pointing object is present are provided.

  Further, when the system control device 1 is a device that activates and operates software according to input information from the external input device 2 or controls other devices, such as the PC, As shown in FIG. 1, in addition to the means 101 to 107, for example, a processing control means 108 and a storage means 109 are provided. When the information acquired by the input information acquisition unit 101 is different from the information related to the control of the pointer, the input information acquisition unit 101 passes the acquired input information to the processing control unit 108 and performs processing according to the acquired input information. Is executed by the system control apparatus 1.

  The external display device 3 may be any display device that can represent a three-dimensional space. For example, a two-dimensional display device that displays a three-dimensional object such as a CRT display or a liquid crystal display on a two-dimensional plane. A three-dimensional display device capable of displaying a three-dimensional stereoscopic image such as a DFD display device (see, for example, Japanese Patent Nos. 3022558 and 3460671) may be used.

  In other words, the external display device 3 may be any display device as long as the operator (observer) can perceive the displayed pointer or object three-dimensionally.

  2A and 2B are schematic diagrams for explaining the principle of the DFD display device.

  The DFD display device is a three-dimensional display device capable of displaying a three-dimensional stereoscopic image with a simple configuration, and is preferable as the external display device 3.

  For example, as shown in FIGS. 2-1 and 2-2, the DFD display device is arranged so that two or more display surfaces overlap each other when viewed from the viewpoint P of the observer (operator). According to the display method, the display device is usually divided into two types, a transmission type and a luminance modulation type. The detailed configuration and operation principle of the DFD display device are described in, for example, Japanese Patent No. 3022558 and Japanese Patent No. 3460671. Therefore, detailed description thereof is omitted, and here, the DFD display device is described. Only a simple operating principle will be described.

  Here, as shown in FIGS. 2-1 and 2-2, it is assumed that the two display surfaces 3A and 3B overlap each other. At this time, a display object Obj such as a pointer or an object is displayed in a three-dimensional space between the two display surfaces 3A and 3B reflecting the depth position.

For example, as shown in FIG. 2A, the display object Obj displayed on the DFD display device is displayed on both the front display surface 3 </ b> A and the rear display surface 3 </ b> B as viewed from the viewpoint P of the operator. At this time, if the DFD display device is a luminance modulation type, the display object ObjA on the front display surface 3A is displayed with the luminance L _A , and the display target ObjB on the back display surface 3B is displayed with the luminance L _B. the inner, the ratio of the distance from the distance and the back of the display surface 3B from the front of the display surface 3A is L _B: display object Obj the depth position of the L _a appears to be displayed.

  Further, for example, as shown in FIG. 2B, by continuously changing the luminance in the display area of one display object Obj, one display object Obj can be displayed tilted in the depth direction. . In the example shown in FIG. 2B, the brightness of the display object ObjA on the front display surface 3A increases from the top to the bottom of the paper, and the brightness of the display target ObjB on the back display surface 3B increases from the bottom to the top of the paper. It is getting bigger as it goes to. Therefore, the operator can observe a three-dimensional display object Obj in which the upper side of the paper is inclined to the back and the lower side of the paper is inclined to the front.

  Although detailed explanation is omitted, when the DFD display device is a transmissive type, for example, by adjusting the transparency of each point (pixel) in the area displaying the display object Obj on the front display surface 3A. Similarly to the luminance modulation type DFD display device, a stereoscopic image of the display object Obj such as a pointer or an object can be displayed at an arbitrary depth position between the front display surface 3A and the back display surface 3B.

  In this way, the DFD display device is a display device that can easily display a three-dimensional stereoscopic image by changing the luminance ratio or transparency of objects and pointers displayed on a plurality of display surfaces.

  The external display device 3 used in the present invention may be any display device that can express a three-dimensional space as described above. However, if a 3D display device (display), such as a DFD display device, in which a space capable of expressing a 3D stereoscopic image is limited in advance, the depth position pointed to by the pointer can be clearly presented to the operator. ,preferable. Further, in the case of a two-dimensional display device such as a general CRT display or a liquid crystal display, it is necessary to project the displayed three-dimensional space onto a two-dimensional plane, and for example, in the case of a DFD display device Since it is only necessary to set the luminance ratio of points (pixels) on each display surface according to the position in the depth direction of the three-dimensional display space, the load on the system control device 1 (pointing device) can be reduced.

  In addition, in the case of a two-dimensional display device such as a general CRT display or a liquid crystal display, the displayed three-dimensional display space is projected onto a two-dimensional plane and displayed. Although it may be difficult to perform a pointing operation, by using a three-dimensional display device such as a DFD display device, it is possible to perform a pointing operation with a feeling closer to real space. From these facts, by using a 3D display device such as a DFD display device, the operator can achieve better accuracy and speed than a case where a pointing operation is performed using a general 2D display. 3D depth can be pointed.

  The external input device 2 may be, for example, a pen tablet or a joystick (joypad) in addition to an input device generally connected to the PC such as a keyboard or a mouse. Furthermore, the external input device 2 may be configured such that, for example, a touch panel or a tablet is integrated with the external display device 3, and information can be input by touching the display screen of the external display device 3 with a pen or a fingertip.

  3 to 5 are schematic diagrams for explaining the three-dimensional pointing method according to the first embodiment of the present invention. FIG. 3 is a diagram for explaining an example of a pointer used in the present invention. FIG. FIG. 5 is a diagram for explaining, and FIG. 5 is a diagram for explaining a method of pointing an object in the depth direction.

  The three-dimensional pointing method according to the first embodiment includes, for example, a part 4A-1 (4A-2) for performing pointing and parts 4B and 4C whose positions in the depth direction are fixed as shown in FIG. The pointer 4 is displayed in the three-dimensional display space of the external display device 3.

  At this time, the part 4A-1 for performing the pointing has a part 4P indicating the pointing position. This part 4A-1 is a part that moves in the depth direction (z> 0) in accordance with the input information from the external input device 2, and follows the locus 4D when moving in the depth direction. That is, the part 4A-1 rotates 360 degrees around the part 4P while changing the z coordinate, and moves to the position 4A-2. At this time, the xy coordinate of the part 4P does not change, and only the z coordinate changes based on the input information from the external input device 2.

  The locus 4D when the part 4A-1 moves in the depth direction may be displayed or may not be displayed.

  In addition, the parts 4B and 4C whose positions in the depth direction are fixed are parts that can be compared with other parts whose positions in the depth direction change, that is, the reference of the depth position. At this time, the parts 4B and 4C are displayed on, for example, the foremost two-dimensional plane and the rearmost two-dimensional plane of the three-dimensional display space expressed by the external display device 3. If the external display device 3 is a DFD display device having two display surfaces 3A and 3B as shown in FIG. 4, for example, the part 4B is displayed on the front display surface 3A (z = 0), for example. Then, the part 4C is displayed on the back display surface 3B.

  Further, in the first embodiment, the external display device 3 is a DFD display device, and the setting of the three-dimensional display space is a space between the front display surface 3A and the back display surface 3B. As long as it is a space that can be expressed by the display device, any space may be used. In the following embodiments, a DFD display device is used as the external display device 3, but a two-dimensional display device such as a general CRT display or a liquid crystal display, or a three-dimensional stereoscopic display such as an HMD (Head Mount Display). It goes without saying that an apparatus may be used.

  In the three-dimensional pointing method of the first embodiment, as shown in FIG. 4, a pen tablet including a tablet 201 and an input pen 202 is used as the external input device 2. At this time, assuming that the three-dimensional coordinates of xyz as shown in FIG. 4 are taken in the three-dimensional display space of the external display device 3, the operator (observer) looks at the external display device 3 from the −z direction. Suppose that When the pen tablet is used, for example, when the input pen 202 is slid on the tablet 201, the pointer 4 is moved to the outside according to the amount of movement of the input pen 202 in the x-axis direction and the amount of movement in the y-axis direction. It is moved in the xy plane on the three-dimensional display space represented on the display device 3. In addition, the pen tablet generally detects the movement of the input pen 202 by an electromagnetic induction method, and can detect not only the position and amount of movement of the input pen 202 but also the pressure (writing pressure) to press the input pen. (For example, see JP-A-5-73208.) Therefore, for example, according to the pressure when the input pen 202 is pressed against the tablet 201, the part 4A-1 for pointing the pointer 4 is controlled, and the depth position (z coordinate) for pointing is controlled. be able to.

  When using the pen tablet, for example, instead of the magnitude of the pressure (writing pressure) of the input pen 202, the depth position of the pointer is determined according to the number of times the tablet 201 is pressed with the input pen 202, for example. May be.

  A method of pointing an object in the depth direction by operating the pointer 4 using the external input device 2 such as the pen tablet will be described with reference to FIG. In FIG. 5, (a), (b), and (c) are the positional relationship between the pointer and the object before pointing, the positional relationship when the pointer is moved in the depth direction, and when the object is pointed, respectively. It is a figure which shows a positional relationship. In FIG. 5, (a), (b), and (c) show a diagram viewed from the operator's viewpoint, a diagram viewed from diagonally above, and a diagram viewed from the right side, respectively. .

  When pointing in the depth direction, the operator first determines the xy coordinates of the pointer 4 based on the two-dimensional position of the pen tip of the input pen 202, and the position of the pointing part 4 </ b> P of the pointer 4 displayed on the external display device 3. It is possible to perceive the pointing two-dimensional coordinates. Here, on the external display device 3, as shown in FIG. 5A, even when the part 4 </ b> A- 1 for pointing and the object 5 overlap with each other as viewed from the operator's viewpoint, When the depth positions are different, the object 5 is not pointed.

  In such a state, for example, when the input pen 202 is pressed against the tablet 201 and a writing pressure is applied, the portion 4A-1 for performing the pointing as in the state of (b) shown in FIG. It moves in the direction of the object 5 (+ z direction) while rotating around the pointing unit 4P. At this time, the angle about the pointing part 4P of the part 4A-1 for performing the pointing appears different to the operator. In addition, when a stereoscopic display device such as a DFD display device is used, the depth of the part 4A-1 for performing pointing can also be perceived. Therefore, the operator perceives that the part 4A-1 for pointing is pointing deeper than the position before the operation from the depth perceived by the rotation angle of the part 4A-1 for pointing. it can.

  Here, when the pen pressure is applied to the input pen 202 again, as shown in FIG. 5C, the rotation angle of the portion 4A-1 for performing the pointing and the depth perceived by the operator are reduced. Further changes. At this time, as in the state of FIG. 5C, the distance in the xy direction between the point 4P to be pointed and an arbitrary point of the object 5 is a predetermined distance (for example, 10 pixels) or less, and the depth When the distance in the direction is equal to or smaller than the predetermined distance, the object 5 is in a state where it is pointed by the pointer 4. Therefore, in order to inform the operator that the state is the pointing state, for example, the color of the object 5 is changed to indicate the pointing state.

  The distance in the xy direction and the distance in the depth direction with respect to the pointing part 4P that is determined that the object 5 is pointed may be set to an arbitrary distance by the operator. A predetermined distance may be set.

  In this way, the operator can intuitively and accurately perceive the depth position where the pointer 4 is pointing from the rotation angle of the portion 4A-1 for pointing. Whether the part 4A-1 for pointing that overlaps the object 5 is pointing to the object 5 by changing the color of the object 5 when it is pointed by the part 4A-1 for pointing. It can be intuitively and accurately perceived. In addition, a DFD display device is used as the external display device 3 here, and the parts 4B and 4C of the pointer 4 serve as a reference as a clue to perception of the depth position. Therefore, there is a great effect that the pointing can be performed intuitively and accurately by comparing with the depth position of the portion 4A-1 for pointing with the portions 4B and 4C.

  FIG. 6A is a schematic diagram for explaining a first modification of the shape of the pointer. FIG. 6B is a schematic diagram for explaining a second modification of the shape of the pointer.

  In the description so far, the shape of the part 4A-1 for pointing the pointer 4 has been exemplified as a water drop shape as shown in FIG. 3 and FIG. 4, but the shape of the part 4A-1 is an operation. Any shape may be used as long as the two-dimensional coordinates and the depth position that the person is pointing can be simultaneously perceived by one pointer and an arbitrary point in the three-dimensional space can be pointed.

  That is, the shape 4A-1 for performing the pointing may be a shape in which the object is pointed with the index finger as shown in FIG. 6A, for example. At this time, the pointing part 4P is the tip of the index finger. At this time, the part 4A-1 rotates around the part 4P (the tip of the index finger) while changing the z coordinate, and moves to the position 4A-2.

  Further, the shape of the part 4A-1 for performing the pointing may be, for example, a triangle as shown in FIG. At this time, the pointing part 4P is set as one vertex. At this time, the part 4A-1 rotates around the part 4P (one vertex) while changing the z coordinate and moves to the position 4A-2.

  Further, in the first embodiment, the part 4A-1 for performing the pointing is set to be able to rotate 360 degrees around the pointing unit 4P. However, the pointing position can be presented to the operator. This is not necessarily the case. For example, the rotation may be 180 degrees.

  Further, the object 5 is also a folder type object in FIG. 5, but may have any shape as long as it can be pointed by the pointer 4.

  In the pointing method of the first embodiment, when the pen tablet is used, for example, the tablet 201 is not limited to the external pen tablets 201 and 202 shown in FIG. It may be provided integrally. If the tablet 201 is integrated with the external display device 3, the pointer 4 can be operated by touching the display screen of the external display device 3 with the input pen 202, for example. Therefore, the operator can operate the pointer 4 more intuitively. The pointing method of the first embodiment is also suitable for pointing using an input device that can be integrated with the external display device 3, such as a touch panel, instead of the pen tablet.

  In the case of the touch panel, for example, the pointer 4 can be operated by touching the screen of the external display device 3 with an operator's finger instead of the input pen 202. Therefore, the touch panel is more than a pen tablet using the input pen 202. The pointer 4 can be operated intuitively. Furthermore, depth pointing may be performed when a predetermined operation is performed with a mouse (mouse wheel), a keyboard, a touch panel, a joystick, or the like.

  FIG. 7 is a flowchart for explaining a processing procedure in the system control apparatus of the three-dimensional pointing method according to the first embodiment.

  Hereinafter, the three-dimensional pointing method of the first embodiment will be described using the system control apparatus 1 having the configuration shown in FIG. At this time, the system control device 1 first causes the display control means 105 to display the object 5 and the pointer 4 on the external display device 3 as shown in FIG. 7 (step 601). At this time, a plurality of the objects 5 may be displayed, and may be displayed at any position within the three-dimensional space represented by the external display device 3.

  Next, information input by the operator using, for example, the tablet 201 of the pen tablet and the input pen 202 is acquired by the input information acquisition unit 101 (step 602). At this time, the input information acquired by the input information acquisition unit 101 also acquires input information that causes the processing control unit 108 to perform processing in addition to the information related to the operation of the pointer 4. Suppose that it was acquired. When the input information acquisition unit 101 acquires the input information related to the operation of the pointer 4, the input information acquisition unit 101 passes the input information to the pointing position calculation unit 102.

  When the pointing position calculation unit 102 receives the input information, first, the pointing position calculation unit 102 calculates the moving direction, the moving amount, and the like of the pointer 4 based on the input information (step 603). In step 603, for example, the movement direction and movement amount of the pointer 4 in the xy plane shown in FIG. 4 are calculated.

  Next, based on the calculation result in step 603, the pointer 4 displayed on the external display device 3 is moved and displayed (step 604). In step 604, for example, the pointer 4 is generated by the pointer generation unit 104 based on the movement direction, movement amount, etc. of the pointer 4 in the xy plane, and then generated by using the display control unit 105. Is displayed on the external display device 3. If the input information does not include information for moving the pointer 4 in the xy plane, the operation in step 604 is omitted, and the processing in the next step 605 is performed.

  When the processing of step 603 is completed, the pointing position calculation unit 102 next calculates the depth position of the part 4A-1 to be pointed based on the input information (step 605). In step 605, the depth position is determined from the writing pressure information of the input pen 202, for example.

  Next, based on the calculation result in step 605, the part 4A-1 of the pointer 4 displayed on the external display device 3 is rotated and displayed (step 606). Step 606 is shown in FIG. 5 using the display control means 105 after changing the depth position of the part 4A-1 by the pointer generating means 104 based on the calculation result of the depth position of the part 4A-1, for example. Thus, the pointer 4 with the depth position of the part 4A-1 changed is displayed on the external display device 3. If the information to change the depth position of the pointing part 4A-1 is not included in the input information, the operation in step 606 is omitted and the processing in the next step 607 is performed.

  When the pointing position calculation unit 102 performs the processing of step 603 and step 606, the pointing position calculation unit 102 passes the calculation result to the pointer generation unit 104 and also to the pointing determination unit 106. At this time, the pointing determination means 106 determines whether there is an object within a predetermined range from the pointed portion 4A-1 after the operation based on the received calculation result (step 407). Specifically, in step 407, for example, the distance in the xy direction between the point indicated by the part 4P of the pointer 4 and an arbitrary point of the object is equal to or less than a predetermined distance (for example, 10 pixels), and the depth It is determined whether or not the direction distance is equal to or less than a predetermined distance. If there is an object in which both the distance in the xy direction and the distance in the depth direction are equal to or less than the predetermined distance, it is determined that the object is pointed. At this time, if there is no pointing object, the process returns to step 602 and waits until the next input information is acquired.

  If there is the pointing object 5, for example, the object generation means 107 generates an object in which the color of the pointing object is changed and displays it on the external display device 3 (step 608). Then, after displaying the object whose color has been changed, the process returns to step 602 and waits until the next input information is acquired. In this embodiment, the color is changed when the object is pointed. However, the present invention is not limited to this as long as the same effect can be obtained such as sounding at the time of pointing.

  Further, after the depth position of the part 4A-1 to be pointed is changed and displayed in the above-described procedure, information on the operation of the depth position of the pointer 4 is acquired from the external input device 2 and moved in the xy plane, for example. In this case, the depth position of the part 4A-1 to be pointed may be returned to the original position, or may be moved while the depth position of the part 4A-1 to be pointed is changed.

  Although illustration is omitted, when input information other than information related to the operation of the pointer 4 is acquired in step 602, the input information acquisition unit 101 passes the acquired input information to the processing control unit 108. At this time, examples of input information other than information related to the operation of the pointer 4 include activation of software associated with the object pointed by the pointer 4 and input information of numerical values or character strings. In this case, the processing control unit 108 performs processing such as activation of software associated with the object based on the input information, and causes the external display device 3 to display the processing result using the display control unit 105.

  As described above, according to the three-dimensional pointing method of the first embodiment, an operation of viewing the pointer 4 by changing the depth position of one part 4A-1 of the pointer 4 and causing the external display device 3 to display it. A person can intuitively and accurately perceive the two-dimensional coordinates and the depth position at which the pointer 4 is pointing.

  8 to 10 are schematic diagrams for explaining the first application example of the first embodiment, FIG. 8 is a diagram for explaining an example of the shape of the pointer, and FIG. 9 is an object in the depth direction. FIG. 10 is a diagram for explaining a modification of this application example.

  In the first embodiment, for example, as shown in FIG. 5, the portion 4B whose position in the depth direction is fixed is formed into an annular shape, and the portion 4A-1 for pointing that is visible on the inside moves in the depth direction while rotating. An example of the pointer 4 having a shape to be used is given. However, the shape of the pointer 4 may be any shape as long as an effect equivalent to the shape of the pointer described in the first embodiment can be obtained. Therefore, an example of a pointer having a different shape will be described below and its operation will be described. The pointing using the pointer 4 described below and the pointing using the pointer 4 described in the first embodiment differ only in the pointer shape and the method of presenting the pointing position to the operator. Therefore, other input methods and operation methods, input information control methods, display methods, and the flow at the time of pointing are the same as in the case of the pointer 4 described above, and thus the description thereof is omitted.

  An example of a pointer having a different shape is a pointer 4 having a shape as shown in FIG. The pointer 4 includes a reference portion 4B that is a clue to depth perception, and a portion 4A-1 'that indicates the depth position while pointing. The pointer 4 represents a two-dimensional coordinate (xy coordinate) at the center of the reference part 4B. Further, the part 4A-1 'representing the depth position has a plurality of parts arranged spirally in the depth direction, and the depth position is presented to the operator by the change of the part 4A-1'. In FIG. 8, the part 4A-1 'representing the depth position is illustrated in 13 stages, but the part 4A-1' representing the depth position may be increased to present a finer depth position.

  Hereinafter, the pointing method using the pointer 4 shown in FIG. 8 will be described with reference to FIG. In FIG. 9, (a), (b), and (c) are the positional relationship between the pointer and the object before pointing, the positional relationship when the pointer is moved in the depth direction, and when the object is pointed, respectively. It is a figure which shows a positional relationship. In FIG. 9, (a), (b), and (c) show a diagram viewed from the operator's viewpoint, a diagram viewed from diagonally above, and a diagram viewed from the right side, respectively, from the top. .

  When pointing in the depth direction, the operator first moves the center of the reference portion 4B of the pointer 4 so as to overlap the object 5 to be pointed, as in the state of (a) shown in FIG. At this time, even if the center of the reference part 4B and the object 5 overlap with each other as viewed from the operator's viewpoint, the object 5 is not pointed when the depth position is different.

  In this state, for example, when an operation of moving the pointer 4 in the depth direction, such as pressing the input pen 202 against the tablet 201 and applying writing pressure, is performed, for example, the state of (b) shown in FIG. In this way, the part 4A-1 ′ representing the depth position of the pointer 4 disappears in order from the one arranged in front of the operator (z <0). Therefore, it can be known which depth position is currently pointing by the number of the parts 4A-1 'representing the displayed depth positions.

  Here, when the operation of moving the pointer 4 in the depth direction is performed again, for example, among the parts 4A-1 ′ representing the depth position of the pointer 4 as shown in FIG. 9C, Only the part located at the back from the same depth position as the object 5 is displayed. Then, when the object 5 is located at a position corresponding to the depth position of the nearest part among the displayed parts 4A-1 ′, the object 5 is determined to be pointing, and the object being pointed to An object with a different color is generated and displayed on the external display device 3.

  At this time, the part 4A-1 'representing the depth position displayed without disappearing becomes a reference as a clue to depth perception. Therefore, the pointer 4 ′ can be a reference to the pointed part itself. Therefore, the display can be simplified, and the load on the display of the system control apparatus 1 (computer) can be reduced. Furthermore, the pointer 4 is a part 4A-1 ′ representing the depth position without erasing the part 4A-1 ′ representing the depth position, for example, as shown in FIGS. It is also possible to present the depth position to the operator by changing the color of. In FIG. 10, (a), (b), and (c) are views showing the positional relationship between the pointer and the object before pointing from the viewpoint of the operator.

  FIG. 11 is a schematic diagram for explaining a second application example of the first embodiment.

  In the first embodiment and the first application example, the portion 4A-1 (4A-1 ′) representing the depth position of the pointer 4 changes in the depth direction while rotating. The depth position can be perceived with a realistic expression. This method will be described with reference to FIG. In FIG. 11, (a) in the upper stage represents before the depth position change, and (b) in the lower stage represents after the depth position change. In FIGS. 11A and 11B, (a) and (b) show a diagram viewed from the upper side, a diagram viewed from the viewpoint of the operator, and a diagram viewed from the right side, respectively, from the left.

  The pointer 4 shown in FIG. 11 has a pointing unit 4A having a part 4P for pointing two-dimensional coordinates and a depth position. Moreover, it has the site | part 4D showing the locus | trajectory which the pointing part 4A follows when pointing a depth position. Since this part 4D is inclined in the depth direction, the depth positions of the parts at both ends thereof are different. Therefore, it also serves as a reference for clues to depth perception. Further, the pointing portion 4P has a portion 4E indicating the depth position at which the pointing is performed. Here, the part 4E may move in the depth direction in the three-dimensional space, or may move only in the two-dimensional plane corresponding to the pointing depth position. Moreover, since the part 4E moves within the xy plane when the operator views the pointer 4 from the front, the depth position can be perceived two-dimensionally.

  FIG. 12 is a schematic diagram for explaining a third application example of the first embodiment.

  In the first embodiment and the first application example, the portion 4A-1 (4A-1 ′) representing the depth position of the pointer 4 changes in the depth direction while rotating, but the present invention is not limited to this. The depth position can be perceived by rotating the part. This method will be described with reference to FIG. In FIG. 12, (a), (b), and (c) are the positional relationship between the pointer and the object before pointing, the positional relationship when the pointer is moved in the depth direction, and when the object is pointed, respectively. It is a figure which shows a positional relationship. In FIG. 12, (a), (b), and (c) show a diagram viewed from the operator's viewpoint, a diagram viewed from obliquely above, and a diagram viewed from the right side, respectively, from the top. .

  The pointer 4 shown in FIG. 12 has a pointing unit 4A having a part 4P for pointing two-dimensional coordinates and a depth position. Moreover, it has the site | part 4B showing the depth position of 4 A of pointing parts. As shown in FIG. 12, this part 4B is a ring that is not closed, and the depth positions at both ends thereof are different, and the part 4B rotates as the operator changes the depth position. When the operator changes the depth position of the pointer, the pointing unit 4A moves in the depth direction along the depth position where the rotating portion 4B exists. Here, as the depth position changes, the operator can perceive the depth position by rotating (changing the position) the two-dimensional position of the unclosed portion of the part 4B. Furthermore, since the depth positions at both ends of the part 4B do not always change and remain at a constant depth, the operator can perceive both ends as a reference for depth perception.

  When the DFD type display device is used as the external display device 3, the pointer shown above is the position of the part 4A-1 ′, the position of the part 4E for performing the pointing displayed on the external display device 3, the part 4B Not only can the depth be recognized by the two-dimensional position of the unclosed portion of the object, but also the depth of each object can be actually perceived, so that the depth position of the pointer can be perceived intuitively and accurately.

  The second embodiment is another application example of the three-dimensional pointing method of the first embodiment, and in the pointing two-dimensional coordinate (xy coordinate), there is a pointable object (or point) in the z coordinate. The pointing method presents an object 4F representing the depth position of the object that can be pointed to a pointer. Since there is no difference from the first embodiment except that the object 4F representing the depth position of the object that can be pointed is displayed, description of the necessary system and operation method is omitted.

  FIG. 13 is a flowchart for explaining a processing procedure in the system control apparatus of the three-dimensional pointing method according to the second embodiment.

  Also in the case of the three-dimensional pointing method of the second embodiment, the system control device 1 first displays the object 5 and the pointer 4 on the external display device 3 as shown in FIG. 13 (step 601). Then, for example, information input by the operator using the tablet 201 and the input pen 202 of the pen tablet is acquired by the input information acquisition unit 101 (step 602).

  Next, after calculating the moving direction, moving amount, etc. of the pointer 4 in step 603, an object that can be pointed in the depth direction of the pointer 4 is a two-dimensional projection plane (xy coordinates) of the point to which the pointer is pointing at that time. The pointing possible point determination means 103 determines whether or not it is within a predetermined distance (here, 10 pixels) (step 609). If there is an object that can be pointed, the object 4F representing the depth position of the object that can be pointed is displayed (step 610). If there is no object that can be pointed, step 610 is skipped and the process of step 604 is performed.

  Note that the processing from step 604 to step 608 shown in FIG. 13 may be the same as the processing from step 604 to step 608 shown in FIG.

  FIG. 14 is a schematic diagram for explaining the concept of the second embodiment.

  When the concept of the second embodiment is applied to the pointing operation of the pointer 4 and the object 5 shown in FIG. 5, for example, the result is as shown in FIG. In FIG. 14, (a), (b), and (c) are the positional relationship between the pointer and the object before pointing, the positional relationship when the pointer is moved in the depth direction, and when the object is pointed, respectively. It is a figure which shows a positional relationship.

  In the pointing method of the second embodiment, for example, as shown in FIG. 14A, the part 4A-1 (pointing part 4P) where the pointer 4 is pointed is moved onto the object 5. In step 609 shown in FIG. 13, it is determined that there is an object that can be pointed. Therefore, in step 610, the object 4F representing the depth position of the object that can be pointed is generated, and the object 4F is displayed on the external display device 3 as in the state of FIG. As a result, the operator can know the relationship between the depth position of the pointer 4 and the depth position of the object 5 that can be pointed regardless of the depth position of the pointer 4, thereby enabling more efficient pointing. .

  From this state, when the operation of moving the pointing portion 4A-1 in the depth direction is performed, the pointing portion 4A-1 is rotated as shown in FIGS. 14B and 14C. Move in the depth direction. Then, when the pointing part 4A-1 is rotated to the position where the object 4F is displayed, the object 5 is in a pointing state. For example, the color of the object 5 as shown in FIG. Changes.

  In the second embodiment, the object 4F representing the depth position of the object that can be pointed has a small triangular shape. However, the object 4F is not limited as long as the depth position that can be pointed to the operator can be presented.

  In the second embodiment, as a condition for displaying the object 4F representing the depth position of the object that can be pointed, a certain distance on the two-dimensional projection plane (xy coordinates) of the point to which the pointer 4 is pointing is set. Although the distance is within 10 pixels, this distance may be set to an arbitrary value in advance by an operator or an administrator, or may be automatically set to a predetermined value by the system control device 1. To do.

  FIG. 15 is a schematic diagram for explaining the three-dimensional pointing method according to the third embodiment of the present invention.

  The third embodiment is a pointing method for one object in a case where the objects are arranged at a depth position overlapped when viewed from the operator. Hereinafter, the three-dimensional pointing method of the third embodiment will be described with reference to FIG. In FIG. 15, (a), (b), (c), and (d) respectively indicate the positional relationship between the pointer and the object before pointing, the positional relationship when the pointer is moved in the depth direction, and the object in front. It is a figure which shows the positional relationship when a is pointed, and the positional relationship when a back object is pointed. In FIG. 15, (a), (b), (c), and (d) are views from the top of the operator in order from the top, a view from obliquely above, and a view from the right side, respectively. Is shown.

  In the third embodiment, in order to simplify the description, for example, a case where two objects 5A and 5B are overlapped as in the state of (a) shown in FIG. 15 is taken as an example. At this time, it is assumed that the operator is pointing to select the object 5B behind.

  Further, using the pointing method of the first embodiment, the operator recognizes the depth position of the pointer and the object from the rotation angle of the pointing unit 4A-1 and the perceived depth position, and points a plurality of objects individually. Although possible, here, in order to make better use of the characteristics of the pointer, the pointing method of the second embodiment is applied. When a plurality of objects are overlapped as in the third embodiment, the object 4F representing the depth position of the object that can be pointed in the second embodiment is displayed by the number of objects that can be pointed. Therefore, for example, when the two objects 5A and 5B are overlapped as in the state of FIG. 15A, the object 4F-1 representing the depth position of the front object 5A and the depth position of the back object 5B An object 4F-2 representing is displayed.

  In the third embodiment, the operator moves the pointer on the xy coordinates, and an object that can be pointed in the depth direction of the pointer 4 is on the two-dimensional projection plane (xy coordinates) of the point that the pointer is pointing at that time. In this case, the objects 4F-1 and 4F-2 representing the depth positions of the objects that can be pointed are displayed. The operator recognizes that there are two objects that can be pointed out from the objects 4F-1 and 4F-2 representing the depth positions of the two displayed objects that can be pointed, and determines the depth position of the object to be selected. Recognition and pointing can be performed without being confused with the depth position of other objects.

  In such a state, for example, when an operation of moving the pointer 4 in the depth direction is performed, the portion 4A-1 for pointing is pointed within the xy plane as in the state of (b) shown in FIG. It moves in the direction of the object 5 (+ z direction) while rotating around the part 4P.

  Here, when the operation of moving the pointer 4 further in the depth direction is performed once again, as shown in FIG. 15C, the part 4A-1 for performing the pointing becomes the pointing unit 4P in the xy plane. It moves in the direction of the object 5 (+ z direction) while rotating around. Then, when the object 4F-1 representing the depth position of the object 5A in front is rotated to the position where the object 4F-1 is displayed, the object 5A in front is pointed and the color of the object 5A in front changes.

  Thereafter, when the operation of moving the pointer 4 further in the depth direction is performed again, as shown in FIG. 15D, the part 4A-1 for performing the pointing operation moves the pointing unit 4P in the xy plane. It moves in the direction of the object 5 (+ z direction) while rotating to the center. Then, when the object 4F-2 representing the depth position of the back object 5B is rotated to a position where the object 4F-2 is displayed, the back object 5B is pointed and the color of the back object 5B changes. At this time, the object 5A in the foreground returns to the color that is not pointed.

  In the third embodiment, the objects 4F-1 and 4F-2 representing the depth positions of the objects that can be pointed have a shape like a small triangle. However, it is possible to present the depth position that can be pointed to the operator. This is not the case. In addition, since there is no difference from the first embodiment and the second embodiment except that a plurality of objects representing the depth positions of objects that can be pointed are displayed, description of necessary systems and operation methods is omitted.

  FIG. 16 is a schematic diagram for explaining an application example of the third embodiment.

  In the example shown in FIG. 15, the two overlapping objects 5A and 5B are both folder-type objects. However, in an actual three-dimensional GUI, not only folder-type objects but also objects of various shapes are displayed. The Therefore, when the object 4F representing the depth position of the object that can be pointed is displayed, the object having the shape corresponding to the attribute assigned to the corresponding object is displayed, so that the operator can select the target object to be selected. The depth position can be recognized and pointed without being confused with the depth position of other objects.

  As an example of displaying an object 4F having a shape corresponding to the attribute of the object that can be pointed, as shown in FIG. 16, the two objects 5A and 5B are overlapped, the object 5A in the foreground is a folder type object, A case where the object 5B is a mail type object will be described. At this time, the object 4F-1 representing the depth position of the object 5A in the foreground is made into a folder shape, and the object 4F-2 representing the depth position of the object 5B in the back is made into a mail shape. In this way, when selecting the front object 5A, it is understood that the external input device 2 may be operated so that the part 4A-1 for pointing rotates to the position of the object 4F-1.

  In the third embodiment, the objects 5A and 5B to be pointed and the objects 4F-1 and 4F-2 representing the depth positions of the objects that can be pointed to are the folder type and the mail type, respectively. This is not necessarily the case if In particular, an object representing the depth position of an object that can be pointed to may be displayed with its shape, color, size, orientation, movement, etc. changed according to the attribute of the object to be pointed. In addition, the position where the index is displayed is not limited to the depth position at which pointing is possible, and the depth position can be presented to the operator in a two-dimensional position. Any position can be used.

  17 and 18 are schematic diagrams for explaining the three-dimensional pointing method according to the fourth embodiment of the present invention. FIG. 17 is a diagram for explaining the problems of the respective embodiments, and FIG. 18 is the fourth embodiment. It is a figure explaining the principle of.

  In the three-dimensional pointing method of the present invention, for example, the objects 5A and 5B that the operator is trying to select may be larger than the size of the pointer 4 as in the state of (a) shown in FIG. At this time, for example, when the pointer 4 is moved onto the object 5A and the part 4A-1 for pointing is moved in the depth direction, the part 4A for pointing as shown in FIG. 17B. -1 may be hidden and you may not know where you are pointing.

  Therefore, in the present embodiment, a pointer is described in which the pointer is not hidden when an object larger than the pointer is pointed, and the operator can perceive the depth position as in the other embodiments.

  In the fourth embodiment, for example, when the pointer 4 is at a position where there is no object as in the state of (a) shown in FIG. 18, it is assumed that the pointer 4 is displayed with an initial size. At this time, the objects 5A and 5B that the operator is trying to select may be larger than the size of the pointer 4.

  For example, when it is desired to point an object 5B larger than the pointer 4, the pointer 4 is moved onto the object 5B as shown in FIG. 18B. And operation which moves the part 4A-1 which performs pointing to a depth direction is performed.

  At this time, if the pointer 4 has an initial size, if the depth position of the part 4A-1 to which the pointing is performed becomes deeper than the depth position of the object 5A in the foreground, the part 4A-1 to which the pointing is performed is the front object 5A. It hides in. Therefore, when there is an overlapping object 5A that is in front of the part 4A-1 where pointing is performed, the size of the pointer 4 in the xy plane is set in advance as shown in FIG. The display is enlarged to the size set by the operator or the size where the pointer can be seen without being hidden by the object. As a result, the operator can recognize the depth position of the part where the pointer is to be pointed, and can point to the target object.

  FIG. 19 is a flowchart for explaining a processing procedure in the system control apparatus of the three-dimensional pointing method according to the fourth embodiment.

  Also in the case of the three-dimensional pointing method of the fourth embodiment, the system control device 1 first displays the object 5 and the pointer 4 on the external display device 3 as shown in FIG. 19 (step 601). Then, for example, information input by the operator using the tablet 201 and the input pen 202 of the pen tablet is acquired by the input information acquisition unit 101 (step 602).

  Next, the moving direction and moving amount of the pointer 4 are calculated (step 603), the pointer 4 is moved to the two-dimensional coordinate position based on the calculation result, and displayed on the external display device 3 (step 604). Thereafter, the depth coordinate of the part to be pointed is calculated (step 605), the part to be pointed is rotated while being moved in the depth direction, and displayed on the external display device 3 (step 606).

  Note that the processing from step 601 to step 606 shown in FIG. 19 may be the same as the processing from step 601 to step 606 shown in FIG.

  In the fourth embodiment, after step 606, as shown in FIG. 19, it is determined whether or not the area of the pointer 4 visible to the operator is smaller than a predetermined size (step 611). In step 611, for example, it is determined whether or not there is an object in front of the pointer 4, and if there is an object in front, whether or not the entire pointer 4 is hidden. At this time, if the entire pointer 4 is hidden, the size of the pointer 4 in the xy plane is enlarged to a predetermined size and displayed on the external display device 3 (step 612). If the pointer 4 is not hidden, the next process (step 607) is performed without changing the size.

  Note that the processing in step 607 and step 608 shown in FIG. 19 may be the same as the processing in step 607 and step 608 shown in FIG.

  However, in the fourth embodiment, not only the pointer is enlarged when the entire pointer 4 is hidden, but also when a part of the pointer 4 is hidden, for example, the part 4A-1 or the reference part 4B for pointing. The whole may be enlarged in the xy plane. Further, when the pointer 4 is enlarged, the pointer 4 may be enlarged not only in the xy plane but also in only one of the x direction and the y direction. Also, this is not necessary as long as the same effect can be obtained, such as enlarging only a part of the pointer. Also, the enlargement ratio may be set dynamically, such as changing the enlargement ratio depending on the area where the pointer is hidden.

  FIG. 20 is a schematic diagram illustrating an example of a method for setting the enlargement ratio of the pointer.

  When the enlargement ratio of the pointer is dynamically set, for example, as shown in FIG. 20, the area of the pointer 4 or the portion 4A-1 for performing the pointing as viewed from the operator performs the pointer or the pointing. When the ratio is smaller than a certain ratio t with respect to the entire area of the part 4A-1 or the like, the area of the part 4A-1 or the like for performing the pointer or pointing is enlarged, and the ratio of the area of the pointer visible to the operator is Keep t. This ratio t may be set by the system in advance or may be arbitrarily set by the operator. The enlargement method is not limited to the example, such as doubling the area when the pointer is completely hidden.

  The present invention has been specifically described above based on the above-described embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention. is there.

  For example, the pointing method as described in the first to fourth embodiments is described as a program to be executed by the computer, installed in the computer, and executed, so that the present invention can be applied to a general personal computer. This pointing method can be realized. At this time, for example, the program can be provided by being recorded on a magnetic, electrical, or optical recording medium, or can be provided via a network such as the Internet.

It is a schematic diagram which shows the structural example of the system for implement | achieving the three-dimensional pointing method of this invention. It is a schematic diagram for demonstrating the principle of a DFD display apparatus. It is a schematic diagram for demonstrating the principle of a DFD display apparatus. It is a schematic diagram for demonstrating the three-dimensional pointing method of Example 1 by this invention, and is a figure for demonstrating the example of the pointer used for this invention. It is a schematic diagram for demonstrating the three-dimensional pointing method of Example 1 by this invention, and is a figure for demonstrating a principle. It is a schematic diagram for demonstrating the three-dimensional pointing method of Example 1 by this invention, and is a figure for demonstrating the pointing method of the object in a depth direction. FIG. 10 is a schematic diagram for explaining a first modified example of a pointer shape. FIG. 10 is a schematic diagram for explaining a first modified example of a pointer shape. It is a flowchart for demonstrating the process sequence in the system control apparatus of the three-dimensional pointing method of the present Example 1. It is a schematic diagram for demonstrating the 1st application example of the said Example 1, and is a figure for demonstrating the example of the shape of a pointer. It is a schematic diagram for demonstrating the 1st application example of the said Example 1, and is a figure for demonstrating the pointing method of the object in a depth direction. It is a schematic diagram for demonstrating the 1st application example of the said Example 1, and is a figure for demonstrating the modification of this application example. FIG. 4 is a schematic diagram for explaining a second application example of the first embodiment. FIG. 6 is a schematic diagram for explaining a third application example of the first embodiment. It is a flowchart for demonstrating the process sequence in the system control apparatus of the three-dimensional pointing method of the present Example 2. It is a schematic diagram for demonstrating the concept of the present Example 2. FIG. It is a schematic diagram for demonstrating the three-dimensional pointing method of Example 3 by this invention. FIG. 10 is a schematic diagram for explaining an application example of the third embodiment. It is a schematic diagram for demonstrating the three-dimensional pointing method of Example 4 by this invention, and is a figure explaining the problem of each said Example. It is a schematic diagram for demonstrating the three-dimensional pointing method of Example 4 by this invention, and is a figure explaining the principle of this Example 4. FIG. It is a flowchart for demonstrating the process sequence in the system controller of the three-dimensional pointing method of the present Example 4. It is a schematic diagram which shows an example of the setting method of the magnification factor of a pointer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... System control apparatus 101 ... Input information acquisition means 102 ... Pointing position calculation means 103 ... Pointing possible point determination means 104 ... Pointer generation means 105 ... Display control means 106 ... Pointing determination means 107 ... Object generation means 108 ... Processing control means 109 ... Storage means 2 ... External input device 201 ... Tablet 202 ... Input pen 3 ... External display device 3A ... Front display surface 3B ... Back display surface 4 ... Pointers 4A, 4A-1, 4A-1 '... for pointing Part 4B, 4C ... part where the position in the depth direction is fixed (reference part)
4P: Part indicating the position where pointing is performed 4D: Trajectory of the part where pointing is performed 4E: Reference part 4F, 4F-1, 4F-2: Object representing depth position 5, 5A, 5B: Object

Claims (11)

A three-dimensional pointing method for pointing an arbitrary point in a three-dimensional display space expressed on an external display device,
Calculating a two-dimensional plane coordinate and a depth position of a point to be pointed in the three-dimensional display space based on input information from an external input device;
A first part indicating a two-dimensional plane coordinate of the point to be pointed, a second part indicating a depth position at the point to be pointed, and a third part whose position in the two-dimensional plane changes according to the depth position Generating a pointer having a portion; and
And a step 3 for displaying the generated pointer on an external display device.   In the step 2, when the distance between the pointer and the object capable of pointing on the two-dimensional projection plane approaches within a certain distance, the depth position of the object capable of pointing is displayed by the third part. The three-dimensional pointing method according to claim 1, wherein an object indicating a position in a two-dimensional plane corresponding to the depth position to be performed is generated.   In addition to the above steps 1 to 3, at least one of the shape, color, size, orientation and movement of the object indicating the depth position of the pointing object is set as an attribute of the pointing object. The three-dimensional pointing method according to claim 2, further comprising a step 4 of changing correspondingly.   The three-dimensional according to any one of claims 1 to 3, wherein the step 2 displays the pointer while changing the size of an arbitrary point in the three-dimensional space. Pointing method.   In step 2, in addition to the first part to the third part, the position in the depth direction is fixed, and the depth position can be compared with other parts whose position in the depth direction changes. The three-dimensional pointing method according to any one of claims 1 to 4, wherein a fourth part is generated. A three-dimensional pointing device for pointing an arbitrary point in a three-dimensional display space represented on an external display device,
Input information acquisition means for acquiring input information from an external input device;
A pointing position calculating means for calculating a two-dimensional plane coordinate and a depth position of a point to be pointed in the three-dimensional display space based on the input information acquired by the input information acquiring means;
Pointer generation means for generating a pointer indicating the two-dimensional plane coordinate position and depth position of the point to be pointed based on the calculation result of the pointing position calculation means;
Display control means for displaying the pointer generated by the pointer generation means on the external display device,
The pointer generating means includes a first part indicating a two-dimensional plane coordinate of the point to be pointed, a second part indicating a depth position at the point to be pointed, and a position in a two-dimensional plane according to the depth position. A three-dimensional pointing device that generates a pointer having a third portion where the angle changes.   When the distance between the pointer and the object that can be pointed on the two-dimensional projection plane approaches within a certain distance, the pointer generation unit determines the depth position of the object that can be pointed by the third part. The three-dimensional pointing device according to claim 6, further comprising means for generating an object indicating a position in a two-dimensional plane corresponding to the depth position to be displayed.   In addition to the above means, an object that changes at least one of the shape, color, size, orientation, and movement of the object indicating the depth position of the pointing object corresponding to the attribute of the pointing object The three-dimensional pointing device according to claim 7, further comprising a generation unit.   The three-dimensional pointing device according to any one of claims 6 to 8, wherein the pointer generation means includes means for changing the size of the pointer.   In addition to the first part to the third part, the pointer generating means has a fixed position in the depth direction, and compares the depth position with other parts whose position in the depth direction changes. The three-dimensional pointing device according to any one of claims 6 to 9, wherein a fourth portion that can be generated is generated.   11. A three-dimensional pointing program for causing a computer to execute the processing operation of each means in the pointing device according to claim 6.

Images