Method and Apparatus for Improving the Virtual Desktop
FIELD OF THE INVENTION
This invention relates a method and an apparatus for improving the quality of the work interface between people and computers.
BACKGROUND OF THE INVENTION
Often tools are intrinsically related to specific tasks. For example, the screwdriver and the screw are specific interrelated. While each is independently useful, the major benefit of their use is in combination. Computers are intrinsically also combinations of tool and task. However, in the evolution of computer technology, the innovation of the tool and the innovation of the relate-able task often develop asynchronously.
The priman' format of computer presentation displays has evolved from the Telex-type printout, to the printout emulation on a CRT, to today's assembly of icon-objects on a simulated worktable. Likewise, the interface tools for the computer have shifted from the cardpunch. to the terminal, to the mouse driven menu control method: during the same brief "evolutionary" span.
There is a need in the art for a method that delivers a synergetic benefit between manual input and visual output in a computer system. More specifically, there is a need in the art for a computer system interface that allows facile hand-eye coordination transactions. Equivalently, there is a need in the art for either improving the man-to-machine input interface, or the machine-to-man output interface, or both.
The currently popular two or three button mouse device dominates present man-to-machine input interface prior art. The mouse device is substantially a palm shaped roller-ball holder that usually requires a mouse-pad or some higher friction surface to help facilitate the user deterministically rotating the roller-ball at desired increments. One clear failing of the mouse device is with users who lack sufficient dexterity to translate the roller-ball at the desired increments. Another clear failing of
the mouse device is with users who lack sufficient ''desk space" on which to translate the roller-ball.
The currently popular computer display window formats dominate present machine-to-man output interface prior art. These formats substantially allow icon-objects to be arranged on a rectangular background area. Except for allowing a minimum icon-to-icon separation to be defined or allowing the set of icon-objects to be ordered into rows according to a descriptive field such as name or date, little has been developed in the way of tools for helping the user to organize the representation of icon-object visualization.
Thus, given the present states of man-to-machine input interfaces and of machine-to-man output interface, it is not surprising to note that little progress has been reported in delivering other than serendipitous combinations of synergetic benefits between manual input modalities and visual output modalities in a computer system; such as in the market's preferred electing to use a standard mouse device with a standard window-type computer display facility.
SUMMARY OF THE INVENTION
One aspect of the present invention relates to a facile command-control system, including:
■ a computer workstation having therein at least one presentation format wherein a plurality of icon-objects are stored for representation on an associated graphic display device, and dynamic icon-object management software of the workstation configures the presentation format icon-objects into a virtual desktop of the icon-objects; and
■ a command-control apparatus, interconnected to the workstation, for transmitting control commands to the workstation, wherein at least one of the control commands elicits an action by the dynamic icon-object management software.
In the context of the present invention, the "virtual desktop" (hereinafter
"desktop") is a graphic representation of all or part of a stored plurality of
icon-objects, wherein the graphic representation's space is substantially larger than that of a single graphic display device when the icon-objects are presented at a human-recognizable resolution. The dynamic icon-object management software maps stored icon-objects onto the representation space according to definable scales of resolution for regions of the display presentation, allows the user to elect a region of the representation space for viewing, and facilitates regular spacing of icon-objects in the viewed display. Thus icon-objects, individually or in mapped proximate plurality, are dynamically scalable.
By linking multiple facile command-control systems of the present invention, new multi-user tasks may be accomplished. For example, a plurality of surgeons, each using a facile command-control systems of the present invention, may link together (e.g. via the Internet, an intra-net. a LAN. etc.) to concurrently participate in a multi-robot surgical operation. For these surgeons, icon-objects may correspond to video views of the operation, patient history, patient physiology monitors, search engines and search-bots looking for relevant surgical sub-routines, etc. Equivalently, multi-participant sports-type events may be coordinated, etc.
According to an embodiment of this aspect of the present invention, the virtual desktop is a mapping of the plurality of icon-objects onto a geometric manifold or a projection thereof.
Generally a "Geometric Manifold" is understood to be a topological space which looks locally homeo-morphic to Rm for some m. It is a generalization of n-dimensional space in which a neighborhood of each point, called its chart, looks like Euclidean space. The charts are related to each other by Cartesian coordinate transformations and comprise an atlas for the manifold. The atlas may be non-trivially connected; there are round-trip tours of a manifold that cannot be contracted to a point.
Practically, a central visualization problem in pure geometry is to create pictures of manifolds situated in space, as well as images showing how they might look from the inside. Therefore, informally speaking, surfaces are 2-manifolds and volumes are 3-manifolds. A few examples of 2-manifolds constructed by gluing and twisting strips of paper may help clarity the definition in the Glossary. When we glue
the ends of a strip of paper together, without a twist, we get a cylinder. When we glue the ends of a strip of paper together after a twist, we get a (one-sided) surface called a Mobius band. If we glue opposite pairs of edges of a square together, we get the 2-torus, a 2-manifold that looks like the surface of a donut. A manifold is embedded when it is situated in a space without self-intersections or singularities. When we give one of the edges of a square a twist before gluing, we get a Klein bottle, which, being a closed, one-sided surface, cannot be embedded in 3-space. We can embed it in 4D space, but its projection to 3D must be self-intersecting. When we glue volumes instead of surfaces, we construct 3-manifolds. Gluing together the walls of a cube in opposite pairs yields a 3-torus. If we twist one volume before gluing, we obtain a 3-dimensional Klein bottle.
It is worth while mentioning that the manifolds can change in their size without changing their topology (shape) and that the size may grow dynamically and without limits while maintaining the topology (shape). This is substantially the reason that mapping and transforming of data-structures onto manifolds is a powerful method of organizing representations for visualization; which is the virtual desktop of the present invention on a workstation's dynamic display device.
Originally, the presentation format for a computer workstation's output was configured to produce a visual display either using a line printer or using a pen plotter. At this stage the geometric manifold of the display was a finite two-dimensional rectangle.
Subsequently, the presentation formats became configured to produce a visual display on an associated graphic display device, such as a cathode ray tube. At this stage too. the geometric manifold of the display was a finite two-dimensional rectangle. However, quickly numerous mappings and projections of icon-objects have been developed. The manifolds for these mappings and projections have primarily been for a finite two-dimensional rectangle, or a plurality of parallel layered two-dimensional rectangles. Independently, certain video effects systems and CAD-CAM systems have allowed the display to be warped, rotated, etc. According to embodiments of the present invention, any geometric manifold or projection thereof may be used as an innate base coordinate system for locating (mapping) icon-objects.
Non-limiting examples of geometric manifolds include conic section surfaces, surfaces of regular geometric solids, Cartesian coordinate unit "cubes" of any number of dimensions, prisms, cylinders, etc.
According to another embodiment of this aspect of the present invention, the virtual desktop is a mapping of the plurality of icon-objects onto a multi layer organization. This organization, for simple two-dimensional rectangles, is already familiar to computer cartographers, to computer animation studios, and to multi-mask lithograph process designers. This allows the user to differentiate a view of visually foreground icon-objects from visually background icon-objects, or to open a foreground layer of icon-objects to view a subsequent background layer of icon-objects: with more than two layers being allowed according to one embodiment of the present invention. In this context, there are many visualization representation techniques that may be applied. For example, layers may be assigned different background colors, textures, "wall papers", or even degrees of "transparency" according to their "strata-graphic depth" beneath a top-most layer of the desktop.
According to another embodiment of this aspect of the present invention, two elected topological edges of the virtual desktop logically shunt together into a single virtual edge. In a simple continuous scrolling example, the two elected edges may be the upper and lower edges of a computer screen. However, in a more flexible sense, any two line segments or curves or like dimensional regions may be defined as logically attached to each other, forming a shortcut across the base manifold or equivalently forming a linkage in the storage representation.
According to yet another embodiment of this aspect of the present invention, at least one icon-object of plurality of icon-objects is mapped to more than one location of the virtual desktop. For example, an infinite two-dimensional manifold is used as the basis for presentation. This mainfold is subdivided into a tessellation of regular rectangular regions, each of which corresponds to a potential presentation on a display screen. A single icon-object, showing the current date and time, may be logically assigned to have a base position in each of these rectangular regions.
According to another embodiment of this aspect of the present invention, at least one of the icon-objects is an electronic mail box.
According to another embodiment of this aspect of the present invention, at least one of the icon-objects is shared with another computer workstation.
According to a different embodiment of this aspect of the present invention, representational scaling of an icon-object on the virtual desktop depends on a positional metric of icon-object. For example, a rectangular section of the base manifold may be scaled such that representation of icon-object size decreases exponentially according to distance from an elected "origin", etc.
According to still another embodiment of this aspect of the present invention, the dynamic icon-object management software maintains a region of the virtual desktop as an archive of past or preferred virtual desktop representations. Simply stated, a portion of the base representation or manifold is reserved for maintaining at least one previous representational view of icon objects. This should allow a user to view his icon-object mapping as it was a few views ago, yesterday, etc.
According to another embodiment of this aspect of the present invention, the workstation pre-fetches icon-objects peripheral to a current graphic device representation of a portion of the virtual desktop. A processor of the workstation may store neighboring regions of the manifold or of the base representation that are proximate to a current display view. This accelerates the user's navigation from a current view region across to an adjacent one.
According to another embodiment of this aspect of the present invention, the command-control apparatus includes at least one device selected from the list: mouse, joy-stick, voice-activation software, scrolling-mouse, roller-ball, mixing board, or touch-pad.
Another aspect of the present invention relates to a program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine, these instructions dynamically managing icon-objects of an associated computer workstation having therein at least one presentation format wherein a plurality of icon-objects are stored for representation on an associated graphic display device, and these instructions include configuring the presentation format icon-objects into a virtual desktop of the icon-objects.
Yet another aspect of the present invention relates to system for facilitating efficient command-control operations on an organized virtual desktop, including: a virtual desktop organized using the method of the present invention and a mouse-like device according to the device of the present invention.
The present invention also relates to a method of organizing a virtual desktop, for use with a computer system having at least one presentation format wherein a plurality of icon-objects are stored for representation on a graphic display device, the method including the steps of:
(a) in a presentation format for graphically representing a plurality of icon-objects contained therein, wrapping a vertical or horizontal dimension of the representation in the format;
(b) selecting an icon-object from the plurality of icon-objects:
(c) scrolling the representation along its wrapped dimensions until the selected icon-object is on the central axes of those dimensions of the display;
(d) selecting a vertical or a horizontal space between a pair adjacent icon-objects in the representation, wherein the selected space is greater than a predefined nominal space;
(e) modifying the representation by compressing the selected space into a nominal space interval, wherein the compressing is by choosing the icon-object of the pair of icon-objects that is furthest from the central axes, and moving the chosen icon-object toward one or both of the central axes until the space between the selected pair of icon-objects has a nominal space interval; and
(f) repeating steps "d" and "e" until all of the icon-objects fit within the display area or until the display includes a predefined maximum number of icon-objects or until all spaces between icon-objects have been reduced to nominal space intervals.
Simply stated, the method of the present invention allows the user to select a most important icon. This most important icon becomes central to the users view of
his virtual desk top, will substantially all other icons "migrating" to the peripheral region of the desktop.
The present invention also relates to a scrolling-mouse device including:
• a base-plate for comfortably resting a hand thereon and the base-plate's upper surface is shaped like the inner surface of the hand;
• at least one pressure-actuated sensor located under at least one fingertip region of the base-plate;
• a rotation-actuation sensor located under the thumb-tip region of the base-plate;
• wherein said sensors are accessible through the base-plate; and
• a connector for connecting the sensors to a command-control interface.
Simply stated, the scrolling-mouse device of the present invention provides a facile interface while simultaneously providing multiple actuators and multiple degrees of command control variable designation.
The system for facilitating efficient command-control operations of the present invention provides an optimized man -"machine" interface for workers who interact via an organized virtual desktop.
On the one hand, the method organizes the virtual desktop so that its central region is arranged without cluttering while the remainder of the desktop is rearranged into the peripheral region. This allows the used a facile means to transform his functional frame of reference since choosing of any desktop object transforms the desktop into a central organization related to the chosen object with the new remainder set of objects conveniently rearranged into the peripheral region.
On the other hand, the device of the present invention offers its user a low-stress ergonomic set of arranged actuators so that his command and control of the desktop can achieve high transaction throughput rates. The device design has configuration flexibility for both digital and analog interface actuators. Actuators may be two state (e.g. on off), multi state, linear or rotational. Furthermore, interpretation
of actuator state combinations can be dynamically assigned using substantially standard software transformation tools.
The method and device of the present invention, when used in conjunction, provide a system of hand - eye - mind coordination which is easy to learn and easy to use. This system is also easy to master, which means that its synergetic combination facilitates new avenues of transactional dynamics, which have not been otherwise forthcoming.
BRIEF DESCRIPTION OF THE DRAWINGS
In order to understand the invention and to see how it may be carried out in practice, a preferred embodiment will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:
Fig. 1 1-VII illustrates steps in organizing a virtual desktop;
Fig. 2 illustrates a basic mouse-like device;
Fig. 3-6 illustrate various actuator configurations for a mouse-like device;
Fig. 7 illustrates a system for facilitating efficient command-control operations on an organized virtual desktop; and
Fig. 8 illustrates a schematic diagram of a facile command-control system.
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT
The present invention relates to a method of organizing a virtual desktop, for use with a computer system having at least one presentation format wherein a plurality of icon-objects are stored for representation on a graphic display device; for example with Microsoft windows, Apple's Macintosh, Silicon Graphics, Sun workstations, etc.
The method including the steps of:
(a) in a presentation format for graphically representing a plurality of icon-objects (shown as A-G in Figure 1) contained therein, wrapping a vertical (Figure 1 1(a)) or horizontal (Figure 1 111(a)) dimension of the representation in the format;
(b) selecting an icon-object (e.g. A) from the plurality of icon-objects (A-G);
(c) scrolling the representation along its wrapped dimensions until the selected icon-object is on the central axes (Figure 1 11(c) or Figure 1 III(c')) of those dimensions of the display;
(d) selecting a vertical or a horizontal space between a pair adjacent icon-objects (e.g. (d) between A and B in Figure 1 I or III) in the representation, wherein the selected space is greater than a predefined nominal space;
(e) modifying the representation by compressing the selected space into a nominal space interval (Figure 1 II or IV), wherein the compressing is by choosing the icon-object of the pair of icon-objects that is furthest from the central axes, and moving the chosen icon-object toward one or both of the central axes until the space between the selected pair of icon-objects has a nominal space interval: and
(f) repeating steps "d" and "e" until all of the icon-objects fit within the display area or until the display includes a predefined maximum number of icon-objects or until all spaces between icon-objects have been reduced to nominal space intervals (e.g. Figure 1 V).
According to one embodiment of the method of the present invention, the size of at least one of the nominal space intervals or the size of at least one of the icon-objects is reduced according to a predetermined metric, and the metric is a function of a measurable distance from the selected icon-object of step "b" to the interval or icon-object whose size is reduced. For example in Figure 1 VI there are three regions defined: a central region 101, an intermediate region 102. and a peripheral region 103. According to one type of spatial transformation, the central region only contains the selected icon-object from step (b). The intermediate region contains objects whose representation is fixed to be much smaller than that of the icon-object in the central region, and the spaces between icon-objects in the intermediate region are proportional to the size of the icon-objects. In the peripheral
region, the icon-objects can be about as tiny as the resolution of the display device will allow and the interstitial intervals may be down to that of a physical pixel.
The present invention also relates to a mouse-like device (e.g. Figure 2) including: a base-plate 20 for comfortably resting a hand 21 thereon and the base-plate's upper surface is shaped like the inner surface of the hand; at least one pressure-actuated sensor 22 located under at least one fingertip region of the base-plate: a rotation-actuation sensor 23 located under the thumb-tip region of the base-plate: wherein the sensors are accessible through the base-plate; and a connector 24-25 for connecting the sensors to a command-control interface 26. The rotation actuation sensor may be a roller-ball, a touch-pad, a joystick, etc. The pressure-actuated sensor may be an on-off switch, a galvanic skin response electrode, a piezoelectric switch, a micro-switch, a potentiometer, a rheostat, a solenoid based diaphragm, or the like.
According to one embodiment (e.g. Figure 3) of the device of the present invention, there are eight fingertip regions: for each finger of the four fingers of the hand there is a finger extended location 31-34 and a finger contracted location 35-38. According to another embodiment (e.g. Figure 4) of the device of the present invention, there are four fingertip regions: one region for each finger 41-44 or 45-48.
According to another embodiment (e.g. Figure 5) of the device of the present invention, at least one of the pressure-actuated sensors 50 measures a finger tip's pressure from the finger extended position to the finger contracted position or from the finger contracted position to the finger extended position. For example, there are four fingertip regions 51-54: one region for the first and second fingers extended, one region for the first and second fingers contracted, one region for the third and fourth fingers extended, and one region for the third and fourth fingers contracted.
According to one variation of the device of the present invention, the base-plate is shaped for a left hand (e.g. Figures 2-5). According to one variation of the device of the present invention, the base-plate is shaped for a right hand (e.g. Figure 6).
The present invention also relates to a command-control apparatus (e.g. Figure 7) including both a left handed mouse-like device 71 and a right handed mouse-like device 72 .
The device of the present invention relates to embodiments wherein the base-plate is sized to fit an adult's hand, an adolescent's hand, a child's hand, an infant's hand, or a large adult's hand. Furthermore, in the device of the present invention the connector for connecting the sensors to a command-control interface is selected from the list: computer interface, arcade game interface, powered wheel chair navigation interface, vehicle command-control interface, robotic command-control interface, etc. Likewise, in the device of the present, the connector for connecting the sensors to a command-control interface is a multi-pin plug, a multi-pin socket, a transmitter, a software configured mapping of the sensors into a command-control specification, etc.
The present invention also relates to a system for facilitating efficient command-control operations on an organized virtual desktop, including: a virtual desktop 73 organized using the method of the present invention and a mouse-like device of the present invention 71; or the virtual desktop coupled with a command control apparatus according to the present invention 71 72. According to the preferred embodiment of the system of the present inventions, an icon-object is an open window. Furthermore, according to the preferred embodiment of the system of the present inventions IF the central icon-object is an open window THEN selecting the size of this open window, as represented on the graphics display device, automatically cause a reduction in the representation size of it's adjacent icon-objects; such that the open central icon-object window does not graphically overlay any adjacent icon-object in the representation. Alternatively, the system of the present inventions has a representation selection command wherein at least one of a predetermined number of previous representations is accessible. According to another variation, in the system of the present inventions areas of the representation on the graphic device are reserved for non-icon-object applications, effectively reducing the screen size wherein the method is applied. In the preferred embodiment of the system of the
present invention, at least one of the icon-objects is an HTML-object or a VRML-object or a Java Applet or a Java Application or an HTTP program, or at least one of the icon-objects is a video-stream, or at least two of the icon-objects form a stereoscopic pair.
Furthermore according to the preferred embodiment (e.g. Figure 1 VII) of the method of the present invention, step "c" includes grouping the plurality of icon-objects into sets of approximately equal numbers of icon-objects such that the sets are symmetric with respect to central axes of the wrapped dimensions, and scrolling of the representation maintains said grouping.
Figure 8 schematically presents a facile command-control system 81. including:
• a computer workstation 82 having therein at least one presentation format 83 wherein a plurality of icon-objects 84-85 are stored for representation on an associated graphic display device 86, and dynamic icon-object management software 87 of the workstation configures the presentation format icon-objects into a virtual desktop 88 of the icon-objects: and
• a command-control apparatus 89. interconnected to the workstation, for transmitting control commands to the workstation, wherein at least one of the control commands elicits an action by the dynamic icon-object management software.
The present invention also relates to a computer program product including: a computer usable medium having computer readable program code embodied therein for organizing a virtual desktop, for use with a computer system having at least one presentation format wherein a plurality of icon-objects are stored for representation on a graphic display device, the computer readable program code in said product includes:
■ computer readable program code for causing the computer system to, in said at least one presentation format for graphically representing a plurality
of icon-objects contained therein, wrapping a vertical or horizontal dimension of the respective representation;
■ computer readable program code for causing the computer system to accept a manually inputting for a selecting of an icon-object from the plurality of icon-objects;
■ computer readable program code for causing the computer system to perform a scrolling of the representation along its wrapped dimensions until the selected icon-object is on the central axes of those dimensions of the display;
■ computer readable program code for causing the computer system to perform an automatically selecting of a vertical or a horizontal space between a pair adjacent icon-objects in the representation, wherein the selected space is greater than a predefined nominal space;
■ computer readable program code for causing the computer system to perform a modifying of the representation by choosing the icon-object of the pair of icon-objects that is furthest from the central axes, and moving the chosen icon-object toward one or both of the central axes until the space between the selected pair of icon-objects has a nominal space interval; and
■ computer readable program code for causing the computer system to perform a repeating of steps "d" and "e" as required, until all of the icon-objects fit within the display area or until the display includes a predefined maximum number of icon-objects or until all spaces between icon-objects have been reduced to said nominal space intervals.
The present invention has been described to a certain degree of peculiarity using substantially non-binding examples. Those versed in the art will recognize that the present invention may be manifest alternatively without traversing beyond its scope or spirit, as defined in the following claims: