WO2000004440A1

WO2000004440A1 - Virtual 3d object control

Info

Publication number: WO2000004440A1
Application number: PCT/EP1999/004709
Authority: WO
Inventors: David V. Keyson; Paul P. Thursfield; Peter G. Matthews; Albertus J. Bongers
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 1998-07-13
Filing date: 1999-07-02
Publication date: 2000-01-27
Also published as: JP2002520966A; EP1034471A1

Abstract

The invention relates to a method of enabling a user to select among a plurality of options, such as the channels in a TV system. The method presents an object (200) with a three-dimensional visual appearance on a display. The available options are arranged on the object (200) by allocating to each option an area (202, 204) on a surface of the object (200). By changing an orientation of the object (200) with respect to the display through an input device, the user can put the desired area (202; 204) in a particular position with respect to the display. When the desired area (202; 204) is in the particular position, the option to which it is allocated is selectable via a further action. A desired change of orientation of the object (200) with respect to the display is provoked by the user via a corresponding change of orientation of a member of the input device with respect to a housing thereof. As the user is in direct contact with the member, he gets the impression of having the object (200) beneath his fingers.

Description

Virtual 3d object control.

The invention relates to a method of interacting with a virtual reality object, the method comprising the steps of: presenting the virtual reality object on a display, said object having a three- dimensional visual appearance; changing a virtual orientation of the object with respect to the display in response to a manipulation of an input device having a member that is moveable with respect to a reference frame.

Such a method is known from IBM Technical Disclosure Bulletin Vol. 38, No. 02, February 1995, page 423. According to the known method, an object having the visual appearance of a three-dimensional cylinder is presented to a user on a display. The user can perform a virtual rotation in either direction around the horizontal axis of the cylinder by dragging with a mouse across the cylinder providing a spinning motion, or by clicking the mouse at the upper or lower part of the cylinder. A drawback of the known method is that control over the motion of the cylinder is not intuitive for the user. First a mouse controlled cursor has to be positioned on a particular part of the cylinder and subsequently a mouse button has to be pushed or a dragging action has to be performed.

It is an object of the invention to provide a method according to the preamble, which provides more intuitive control over the object. To this end, the method according to the invention is characterized in that the manipulation of the input device comprises the step of changing an orientation of the member with respect to the reference frame in correspondence with a desired change of the virtual orientation of the object with respect to the display, the step of changing the orientation of the member being performed by the user being in direct contact with the member. Hence, the virtual movements of the object resemble the movements of the input device, which gives the user the impression that he is in actual contact with the object he sees on the display, and which enhances his understanding of the manipulation process. A change of the virtual orientation of the object with respect to the display can be suggested in a number of ways. Firstly, if the three-dimensional shape of the object is such that there is little or no axial symmetry with respect to a certain axis, then a change of the virtual orientation of the object is obvious from virtually rotating the object itself around that axis. Secondly, on the three-dimensional surface one or more fields can be mapped, for example a picture field, a texture field or a transparency field. Then, if there is a fixed one-to- one mapping of elements of the field to three-dimensional coordinates on the surface of the object, again a change of the virtual orientation of the object is obvious from virtually rotating the object itself. Alternatively, the field can be moved across the surface of the object in response to a change of orientation of the member, without the one-to-one mapping of elements of the field to the surface being maintained. Although in principle this is not possible in the real world, such a change of the outlook of the object still suggests to the user that in response to a change of orientation of the member the object itself undergoes a similar change of its virtual orientation. Note that a requirement for an input device to be applicable in combination with the method of the present invention is that it comprises a member, the movements of which the user can associate with the movements of the object on the display, as if he can move the object directly. Examples of such input devices are a trackball and a rotatable cylinder. In case of a trackball, the member of the input device is the ball, and the reference frame is formed by the housing of the trackball. Clearly, a mouse does not lend itself for usage in combination with the method of the invention. The member of the mouse, i.e. the rotatable ball underneath it, is not directly manipulatable by the user, whereas to movements of the housing of the mouse, i.e. the part of the mouse that should be taken in the hand, the user will only associate two-dimensional movements. Other input devices that can be used in combination with the method of the invention are, for example, those devices that use a reference frame that is external to the input device. Examples of such devices are devices with a earth-magnetic force sensor or a gyroscope. By turning such a device, the sensor will record a change of orientation of the device with respect to the earth-magnetic force lines or the angular momentum of the gyroscope, which change the method subsequently transforms into a corresponding change of orientation of the object on the display. Such a way of operating the input device the user will readily associate with changes of orientation of the object on the display.

Advantageously, the member of the input device has a form that resembles the appearance of the object. This measure further increases the impression of actually having the object on the display in one's grasp. It should be noted, however, that the resemblance does not have to be very close to provoke this impression: e.g. a trackball and a polyhedral object , sufficiently resemble each other. It is found that even three-dimensional objects with a very irregular shape can be effectively controlled by a trackball, whereas still the impression is provoked of having the object under one's hand.

In accordance with an aspect of the invention, the method of the invention is used for enabling the user to select among a plurality of options. In the known method, on the surface of the object, having the shape of a cylinder, the options are represented as text fields. The cylinder shape suggests to the user that by rotation of the cylinder additional choices are revealed. The option corresponding to the text field that is frontally presented to the user is selectable. Thus, after virtually rotating the cylinder until the desired text field is in the front position, the user may select the corresponding option. An advantage of using a three- dimensional object for laying out a plurality of available choices is that it takes up less screen real estate than ordinary list boxes. The method of manipulating the object in accordance with the invention is more intuitive than the known method. Thus, in one of its aspects, the invention provides a more user-friendly method of enabling selection among a plurality of options.

Advantageously, the visual appearance of the object is spherical. On a spherical surface a lot of options can be efficiently organized. In combination with a trackball input device, a method for enabling selection of options is provided, that is visually attractive and easy to understand and operate by the user.

Advantageously, the method is applied in a data processing environment for receiving television (TV) signals for enabling selection of channels or channel sets. Such an environment lends itself particularly well for using the method of the invention. Especially in the TV environment it is crucial that the communication with the user is simple and straightforward. It cannot be expected from the users that they are willing to learn to handle a complex interaction process with their TV, as perhaps they would in the PC environment. On the other hand, the current way of choosing channels by remembering channel numbers is simple but awkward. By enabling the user to select a channel m accordance with the method of the invention, he no longer has to remember the channel numbers, but can select a channel at a glance. If the representation of the channels comprises a visual display of what is currently broadcast in that channel, the user can additionally look for channels currently broadcasting interesting content, for example, a live football match. Preferably, the input device is a trackball integrated in a remote control of the TV. The invention also relates to a system suited for application of a method according to the invention.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

In the drawings:

Figure 1 shows a widely used data processing system in which the invention can advantageously be applied,

Figure 2 shows an arrangement of areas on an object according to a preferred embodiment of the invention, and

Figure 3 shows a block diagram of a data processing system according to a preferred embodiment of the invention.

Figure 1 shows a widely used data processing system 100 in which the method of the invention can advantageously be applied. It is a system for presenting TV data to a user, the TV data entering the user's living room via a wall plug 102 after being transmitted to him by a service operator via a cable, terrestrial, satellite link, or otherwise. This data processing system 100, further referred to as TV system, comprises a set-top box 104 and a TV set 106, the set-top box converting the transmitted signal to a TV-signal that can be fed to the TV set 106. The TV system 100 further comprises a remote control 108, working for example on the principle of infrared light or ultrasound.

The active channel is the channel that is presented in a full screen view and is heard in the foreground as the single auditory source. For selecting the active channel, the user has a plurality of channels, i.e. options, to select among. Most often, the user does this by typing a unique number associated to the channel of his choice on the remote control 108.

Consequently, he has to remember an ever increasing list of numbers, that list also being likely to change from time-to-time when the operator reorganizes the available bandwidth. This problem is solved by the method of the invention, in that to each channel is allocated a respective area on an object with a three-dimensional visual appearance on a display of the TV set 106. Each area has an indication to which channel it is allocated, for example in the form of a channel logo or in the form of a projection on that area of the content that is currently delivered through the channel. In order to select a particular channel, the user rotates the object with the aid of an input device, which is, in a preferred embodiment, a trackball integrated in the remote control 108, until the area corresponding to the desired channel is in a particular position. Subsequently, the TV system 100 switches to that channel immediately or in response to a further user action, such as the user pushing a button or pushing the trackball downwardly. The input device comprises a member and a housing that are moveable with respect to each other. In the preferred embodiment, the member is a ball which can freely rotate in a housing. The housing constitutes a reference frame with respect to which the ball has a certain orientation. The process of changing the orientation of the member with respect to the housing in correspondence with a desired change of orientation of the object with respect to the display, gives the user the impression of actually having the object under his hand. It is noted that instead of, or in addition to presenting the object on the display of the TV set 106, the object can be shown on a display integrated elsewhere in the TV system 100, such as in the remote control 108.

Figure 2 shows an arrangement of areas 202, 204 on an object 200 according to the preferred embodiment of the invention. The object 200 has the form of a globe, and thus closely resembles the member of the input device, i.e. of the trackball, which further enhances the impression of the user of actually having the object under his hand. Each one of the areas 202, 204 comprises a textual indication of the channel to which it is allocated. The area 204 is in the aforementioned particular position, which is the front position in the preferred embodiment, in which an area has to be moved for enabling selection of the corresponding channel. In a further embodiment of the invention, the area that is in the front position can additionally be highlighted or otherwise given a visual indication in order to aid the selection process. The actual selection is performed by pressing a button on the remote control 108, whereas many alternative ways are possible, e.g. pressing the trackball itself.

The object 200 is divided into areas by "meridians" and lines parallel to the "equator" of the globe, said meridians and equator being defined with respect to an axis of the globe that is vertical with respect to the display. Such an arrangement of areas increases the three-dimensional impression of the object. However, when the number of channels that is simultaneously represented on the surface of the globe is large, areas near the "poles" are small and indications contained therein are hard to identify. Therefore, in the preferred embodiment the areas near the poles are not allocated to channels. Alternatively, the division into areas of the surface of the globe 200 can be done such, that areas near the poles are scaled up so that indications are still identifiable. As a further alternative, the mapping of the areas on the surface of the object can be done in a flexible manner, in the sense that the areas are not glued to the surface, as it were, but rather are projected on the surface in a way that depends on the orientation of the member of the input device. For example, a rectangular maze of areas can be frontally projected on the globe, with the perspective being such as if the maze were wrapped around the globe, whereas rotating the input device merely moves the center of the maze. with respect to the front position, without tilting the maze. In this way, it can be avoided that areas become tilted with respect to the display after a manipulation of the input device. Moreover, the whole globe can be virtually covered by areas, whereas whichever area is in the front position, the look of the front position area will be independent of the orientation of the member of the input device. Although this would not be possible with a real-world object, the user does get the impression that he is actually moving the object itself, when manipulating the member of the input device. An advantage of using a three-dimensional object for laying out a plurality of available choices is that it takes up less screen real estate than ordinary list boxes. In the background, for example, the active channel can be displayed, while enabling the user to select a new channel. The visual feedback overcomes the problem of having to remember the channel numbers associated to the channels. The three-dimensional appearance of the object can be further enhanced by applying shading and further visual effects.

In the preferred embodiment, there is a one-to-one correspondence between the orientation of the object and the orientation of the member of the input device. It is noted, however, that this is not required for a method in accordance with the invention. For example in a configuration like the one of the preferred embodiment, it could be imagined that a rotation of the trackball corresponds to less than or more than a rotation of the globe, or that the rotation of the globe also depends on the rotation speed of the input device.

Up to now, it is assumed that to each channel there is allocated an area on the globe 200, which is a straightforward and obvious choice. This could very well work in the situation that the number of available channels is sufficiently low, say, not more than 20. Nowadays, however, the number of channels available to most TV users is rather large and still increasing, certainly in the light of the advent of digital TV broadcast. Consequently, if each channel has a respective area on the surface of the object allocated to it, this would lead to rather small areas that are hard to distinguish one from another, which is disadvantageous to the user's overview on the available channels and at the same time requires very precise movement of the object in order to select a desired channel. In the preferred embodiment, the number of areas that are simultaneously present on the surface of the object is limited, thereby restraining the dexterity required from the user in handling the trackball. Hereto, the available channels are classed in a number of categories, e.g. Sports, News, Music, Movies, Children, etc. First a globe is presented, on which the various categories are shown. In the preferred embodiment, the areas display textual indications of respective available categories. Alternatively or additionally, descriptive graphics of the categories could be used. The user can select a desired category in the manner described before. Subsequently, a different object is presented, on which the channels pertaining to the selected category are presented, the number of which being of course considerably lower than the total number of available channels. Then, the user can select the channel of his choice.

In a further embodiment, the user is supplied with tactual feedback in the form of forces applied to him via the member of the input device, thus facilitating the positioning of any desired area in the front position. Hereto, the trackball of the remote control 108 should have force feedback. When manipulating the trackball, the user experiences a force on the trackball as a function of the orientation of the object. The force could be such that the object appears to be drawn towards orientations in which any one of the areas is in the front position. Alternatively or additionally, the forces on the trackball could simulate inertia of the object, thereby further increasing the impression that, instead of manipulating the trackball, the user is actually moving the object. The use of such intuitive navigational guidance considerably lowers the complexity of the interaction process.

In a further embodiment, the object has a default orientation, such that a particular area is by default in the front position. After the user having moved the object away from this default orientation, the object automatically again assumes the default orientation. This could further be supported by providing forces to the user via the member, apparently drawing the object toward this default orientation.

In a further embodiment, auditory feedback is provided. This feedback is presented in the form of audio cues, representative of the various areas. When the orientation of the object is such that a particular area is close to the front position, a corresponding auditory cue is played in the background, for example laughing children upon approaching the category Children, or part of the CNN tune when approaching the target zone corresponding to the CNN channel. Alternatively or additionally, when an area approaches the front position, the actual sound of the channel corresponding to that area as it is broadcast at that moment is output. Further auditory feedback could be aimed at increasing the impression of the user of working with a real-world object, e.g. by rolling sounds or soughing sounds while the orientation of the object is changing.

In the preferred embodiment, the globe is not shown on the display until the trackball is manipulated, upon which the object with the available channel categories is overlaid on the display, without severely disturbing the viewing of the active channel. Initially, at the front position the area corresponding to the current category is displayed. After selection of the desired category, the object comprising the channels belonging to the selected category is displayed. Initially, the area of the active channel is located at the front position. After selection of the desired channel, the object disappears and the desired channel is shown on the display. After bringing up the globe on the screen, the globe automatically disappears again if no further actions are performed by the user. Alternatively or additionally, the globe can be made to disappear by an appropriate gesture, i.e. by a movement of the member that is recognized by the TV system 100 as being different from an ordinary orientation adjustment, e.g. on the basis of the velocity of the movement. For example, a sufficiently fast movement of the ball, corresponding to a movement of the globe around its vertical axis, could be interpreted as a command for making the globe disappear. This correspondence between gesture and subsequent action could be made visible to the user by showing an indication of the action, such as "exit", next to the globe in the direction that the globe should be moved. For the above example, this would mean some indication next to the globe near to the equator. The same principle can be used for bringing up the globe in the first place, or for switching between the various globes. Further gestures that could be imagined are, amongst others, movements in the direction of either pole of the globe, or in any other direction.

In the preferred embodiment, the areas display textual indications, i.e. names of available categories or channels. Alternatively, the channel corresponding to a particular area can be identified through a projection of the content that is currently delivered through that channel. The latter can be implemented by deducing bitmaps from the current video content of the channels corresponding to the visible areas, and using an X,Y lookup table for mapping the bitmaps to the areas on the globe.

Figure 3 shows a block diagram of a data processing system 300 according to a preferred embodiment of the invention. The data processing system 300 is a TV system comprising a set-top box 304, a TV set 306 and a remote control 308. The set-top box 304 receives a transmitted signal from a service provider via link 310. The set-top box 304 generates a TV signal that is subsequently fed to the TV set 306 through a link 312. At this point it should be mentioned that, for the invention, it is not essential that the functionalities of set-top box and TV set are put into separate boxes; they can be integrated into one box, or, alternatively, be distributed over the boxes in a different manner. In the preferred embodiment, the user can interact with the TV system through remote control 308, comprising multiple buttons 320 and a trackball 322, the remote control 308 sending signals to the set-top box in response to manipulations of the trackball. In the preferred embodiment, set-top box 304 comprises the necessary means for implementing the invention. As the details of the invention have already been extensively discussed in connection with the previous figures, only a concise overview of these means is given. A decoder 350 is provided for demultiplexing and decoding a single channel from a plurality of channels received via link 310. A graphics generator 352 is arranged for generating a graphical representation of a globe. The output signals from the decoder 350 and the graphics generator 352 are mixed by a mixer 356 and presented to the TV set 306 in the form of a TV signal. A device control unit 358 is provided for receiving signals from the remote control 308 in response to the user manipulating the trackball 322. The decoder 350, the graphics generator 352 and the device control unit 358 are under control of a microcontroller 380, that microcontroller 380 being further connected to a memory 382 for storing software programs and code describing attributes of the globe, such as the various indications to be displayed on the surface of the object. The microcontroller 380 converts a manipulation of the trackball to a change of orientation of the globe and detects whether an appropriate one of the buttons 320 is pressed for selection of the channel corresponding to the area presently being in the front position, in response to which it causes the decoder 350 to decode the desired channel and to provide the TV set 306 with the required TV signal.

The programming of diverse aspects of the object can be left to the user and/or is done dynamically, the latter meaning that aspects of the object, such as the layout of areas over its surface, are adjustable by the service operator, or are adjusted automatically in the course of use according to the measured preferences of the user. For instance, if the operator chooses to add a channel, information is sent along with the TV data, automatically allocating an area to the new channel. Or, when after some time the set-top box has noticed that the user is a sports fan, the presentation and ease of selection of the sports category and sports channels could be enhanced.

Other options that could advantageously be made selectable with the aid of the method according to the invention regard user preferences with respect to adjustable system- options. These options could be arranged on the surface of the object along with the channels or the categories or, alternatively, a separate object could be used for accommodating them.

Although the method in accordance with the invention is particularly useful for enabling the selection among a plurality of options, and the embodiments particularly refer to that application, the method is generally applicable to the problem of how to control the virtual orientation of three-dimensional objects. It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design many alternative, embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware.

Claims

CLAIMS:

1. Method of interacting with a virtual reality object, the method comprising the steps of: presenting the virtual reality object (200) on a display, said object having a three-dimensional visual appearance; changing a virtual orientation of the object (200) with respect to the display in response to a manipulation of an input device (108; 308) having a member that is moveable with respect to a reference frame, characterized in that said manipulation of said input device (108; 308) comprises the step of changing an orientation of said member with respect to said reference frame in correspondence with a desired change of the virtual orientation of said object (200) with respect to said display, said step of changing said orientation of said member being performed by the user being in direct contact with said member.

2. Method as claimed in Claim 1, said member having a shape that resembles the appearance of the object (200).

3. Method as claimed in Claim 1, wherein the input device (108; 308) comprises a trackball (322).

4. Method as claimed in Claim 1, the method further enabling a user to select among a plurality of options through the step of allocating to each option an area (202, 204) on a surface of the object (200), the orientation of the object (200) determining which one of the options is selectable.

5. Method as claimed in Claim 4, wherein said visual appearance of the object

(200) is spherical.

6. Method as claimed in Claim 4, the method being applied in a data processing environment (100; 300) for receiving television signals, a number of options of the plurality of options corresponding to respective channels or channel sets.

7. Data processing system (100; 300) comprising means (106, 108; 306, 308) enabling a selection among a plurality of options by a user, the means comprising: display means (106; 306) for presenting an object (200) with a three- dimensional visual appearance, areas (202, 204) on a surface of the object (200) being allocated to respective ones of the plurality of options; an input device (108; 308) having a member that is moveable with respect to a reference frame; a control unit (104; 304) for receiving input data from the input device (108; 308) and for outputting display data to the display means (106; 306), the control unit (104; 304) being arranged for changing an orientation of the object (200) with respect to the display means (106; 306) in response to a manipulation of the input device (108; 308), the orientation of the object (200) determining which one of the options is selectable, characterized in that said control unit (104; 304) is arranged for transforming a change of an orientation of said member with respect to said housing into a corresponding change of orientation of said object (200) with respect to said display means (106; 306), said member being arranged for direct contact with the user.

8. Data processing system as claimed Claim 7, wherein the input device (108;

308) comprises a trackball and the object (200) has a spherical appearance.

9. Data processing system as claimed in Claim 7, wherein the data processing system is arranged for receiving television signals and a number of options of the plurality of options correspond to respective channels or channel sets.