CA2274786A1 - Computer control system - Google Patents
Computer control system Download PDFInfo
- Publication number
- CA2274786A1 CA2274786A1 CA002274786A CA2274786A CA2274786A1 CA 2274786 A1 CA2274786 A1 CA 2274786A1 CA 002274786 A CA002274786 A CA 002274786A CA 2274786 A CA2274786 A CA 2274786A CA 2274786 A1 CA2274786 A1 CA 2274786A1
- Authority
- CA
- Canada
- Prior art keywords
- computer
- cursor
- function
- control according
- sensor range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
- Circuit Arrangement For Electric Light Sources In General (AREA)
- Toys (AREA)
Abstract
This invention concerns a control system for at least one function (10) of a computer (1). The control system has sensors (5) for detecting the position (8) of at least one cursor (3) on a computer (1) display and elements for detection of a dwell time of the cursor (3) at the position (8). Furthermore, the control system has elements which, in relation (9) to the cursor position (3) and the dwell time of the cursor (3) in at least one 1 or 2 dimensional determined zone (sensor zone (2)) of the computer (1) display, influence at least one function (10) of the computer (1). This creates a control system with which computers (1) can be flexibly and differentially controlled and, in relationship to the behavior of the cursor (3), certain functions (10) of the computer (1) can be controlled.
Description
03~JUN-1999 16~26 FROM DUMMETT COPP TO MRIKOtJSKI P.03i31 COMPUTER CONTROL
DESCRIPTION '_~
The invention relates to a control for a computer.
Controls for multi functional systems, such as for example computers, have been known in a simple form for some time.
In particular computers are known wherein a user triggers or influences functions of the computer through a trigger, e.g. a mouse-supported cursor on a screen or a data glove for manipulating three-dimensional objects.
Thus with the nowadays widely used multi media systems which are designed as computer programs, the trigger of a function (e.g. the display of a picture, the retrieval of a program or the playing of a video) is produced by selecting a certain point on the display screen. By multi media systems are understood systems where the senses of a person are influenced by various media, such as for example by text) pictures, videos, sounds, noises or music. In principle multi media systems also include however contact (tactile) or smell (olfactory? stimulus which has an effect on a person.
From IBM Technical Disclosure Bulletin, Vol. 36, no. 11, November 1993, page 57 a method is known for activating user-interface functions of a computer. The time spent by a cursor in a small part of the display of the computer is detected. After a certain dwell time a certain function 03-JUN-1999 16:26 FPOM DUMMETT COPP TO MAIKOIJSKI P.04i31 is executed which is assigned to this part of the display.
A drawback with these computers is that the operation of the computer is only carried out through the position and where applicable the dwell time of the cursor which represents a considerable restriction on the functionality. Only one function can ever be Executed in a certain manner and way since the trigger of the function is produced by clicking on the mouse in a fixed predetermined area of the screen or by the cursor dwelling in this fixed predetermined area. The potential functionality of the computer is severely restricted by this non-differentiated use of the cursor.
The obj ect of the invention is to provide a control for a computer and a method for computer control wherein the functions of the computer can be controlled in specially differentiated ways and the functionality of the computer is increased.
This is achieved according to the invention through a control with the features of claim 1 and through a method for control with the features of claim 19.
The control of a computer according to the invention has sensor means with which the position of at least one cursor on a display of the computer is detected. If the position of the cursor is within a certain partial area of the display, the sensor range, then the dwell time and the position of the cursor in the sensor range are measured.
The information on the dwell time and the position is used for influencing at least one function of the computer.
The dwell time of the cursor outside of the sensor range is not measured.
03-JUN-1999 16:27 FFOM DUMMETT COPP TD MAIKOIJSKI P.05i31 By computer is meant any device for data processing which is fitted inter alia .With a screen and a device for controlling a cursor (e.g. mouse) digitalizer). The control of the computer according to the invention can be designed for example in the form of a processor or a program. Generally the functional units described below can be produced selectively as soft or hardware.
Thus compared with the known controls for computers not only the position of the cursor but also -the dwell time of the cursor at a spot on the screen are significant for the function of the computer. Through these additional parameters (namely position and dwell time in the sensor range) it is possible to increase the functionality (e. g_ control of audio-visual signals) of the computer and to provide new control possibilities.
One advantageous embodiment of the control according to the invention detects the speed and/or acceleration of at least one cursor and uses this information to influence at least one function of the computer. The functionality of the computer is improved through these additional parameters of the trigger. Thus a control for a computer according to the invention can react differently for example to a rapid or slow movement of a cursor.
In a further advantageous embodiment the path of movement is detected which is described by at least one cursor on the screen of the computer. At least one function of the computer is influenced in dependence on this data. Thus for example the paths of movement of the cursor on the display of the computer are characteristic for certain situations in the operation of the computer or for certain 0~ JUN-1999 16:27 FROM DUMMETT COPP TO MRIKOWSKI P.06i31 users. This information can be used to adapt the computer better to a user.
In a particularly advantageous design the control according to the invention has means with which the kinematic (dynamic) behaviour of the cursor is quantified.
sy kinematic behaviour of the cursor is meant here generally the space-time behaviour of the cursor on the computer screen, which includes in particular the dwell time, the position, the speed and the acceleration of the cursor. By quantification is meant that the kinematic behaviour of the cursor is detected through parameters or functions which describe for example the dwell time or shape of the path of movement. These parameters and functions form the input values for functional connections which link the kinematic behaviour of the cursor directly with a function of the computer. These functional connections can be stored read-only in a data bank or can be changed during the course of time. Through the quantification of the kinematic behaviour of the cursor it is possible to influence the functions of the computer in very different ways.
The quantification is used with special advantage in combination with a random generator so that particularly With computers having multi media applications or games.
novel effects can be achieved again and again.
The control advantageously has means to change the position, shape and/or function of at least one sensor range in the computer in predetermined or random way.
Thus the sensor ranges can be adapted to changing situations which increases the flexibility of the control and of the computer.
03-JUN-1999 16:27 FROM DUMMETT COPP TO MAIKOIJSKI P.07~31 In a further advantageous development of the control according to the invention a data bank serves to store the kinematic behaviour of at least one cursor. It is also advantageous to detect and store the spatial, time and/or functional changes of at least one sensor range in a data bank . Certain movements or movement patterns of the cursor can thereby be stored for example and used in a particularly advantageous way for influencing functions of the computer and/or at least one sensor range.
In a particularly advantageous design of the control according to the invention a continuous transition (fading) takes place between two different functions of the computer. In particular a number of aesthetic or useful effects can thereby be achieved between visual and/or acoustic functions.
A particularly advantageous development of the control according to the invention has a data bank in which objects are stored for influencing at least one function of the computer. At least one of these objects thereby has an attribute which describes a property of the object.
This attribute can describe for example the type of object (e. g. text) or also the contents of the object (e_g.
verse). By using attributes the control is able to easily make relationships between different objects.
In a further advantageous design of the control according to the invention at least one object and/or attribute of the object stored in the data bank has a modifier. This modifier is a standard by which the control can compare different objects or attributes with each other. A
modifier can be stored in predetermined manner in a data 03-JUN-1999 16:28 FROM DUMMETT COPP TO MRIKOIJSKI P.0Bi31 bank or can be changed by the control over the course of time_ Advantageously the control according to the invention has means by which at least one function of the computer can be controlled by the kinematic behaviour of the cursor in connection with attributes and/or modifiers of at least one object. It is thus possible that the kinematic behaviour of the cursor and the properties of the objects ZO influence the function of the computer whereby a very flexible control of the computer is possible.
The control according to the invention likewise advantageously has means with which objects, more Z5 particularly media, can be stored in the data bank automatically sorted according to type. The detection of objects (e.g. texts or pictures) which are to be used as functions of a computer can thus be considerably speeded up. The control can thus assign e.g. automatically 20 determined attributes to the objects.
At least one of the objects more advantageously has information on a sensor range, a picture, a text, a noise, a piece of music, control data far external equipment, 25 data for a three-dimensional representation, a modifier, an attribute or a group of objects. Through the general treatment of the most varied information as object it is possible to use this information in uniform way for influencing the computer.
Advantageously with a control according to the invention for a computer at least one sensor range for a cursor is filed invisible behind the display of the computer. The display thereby appears in the u5ua1 form for the user.
03-JUN-1999 16:2E FROM DUMMETT COPP TO MRIKOIJSKI P.09i31 _ 7 _ In a likewise advantageous development of the control according to the invention the cursor is controllable on the display of the computer by the eye movements of a user of the computer. The detection of the eye movements can thereby take place for example through video monitoring of the pupils or a derivative of action potentials of face muscles. The activation of the cursor through eye movements is particularly suitable for people who cannot use their hands when working at the computer.
l0 With the method according to the invention for controlling at least one function of a computer first the position of at least one cursor on a display of the computer is detected by sensor means. Through the at least one cursor the functions of the computer such as e.g. audio visual signals can be activated. The information captured by the sensor means are then transferred to the control. The control then establishes whether the at least one cursor is located in a 1- or 2- dimensional partial area (sensor range) of the display of the computer. If the position is inside the sensor range then the control determines the dwell time and the position of the at least one cursor inside the sensor range. The control finally influences the at least one function of the control in dependence on the dwell time and the position of the at least one cursor.
The invention will now be explained with reference to the embodiments shown in the drawings in which:
Figure 1 shows a diagrammatic view of a display of a computer whereby the functions of the computer can be influenced by a cursor and sensor ranges for the cursor;
03-JUN-1999 16:2E FROM DUMMETT COPP TO MRIKOIJSKI P.10i31 _ g Figure 2 is a diagrammatic illustration of a sensor range on a computer display;
Figure 3 is a diagrammatic illustration of a functional connection between the position of a cursor on the computer display and a function of the computer (interaction graph) ;
Figure 4 is a diagrammatic illustration of a functional connection between the time behaviour of a cursor and a function of the computer;
Figure 5 is an illustration of the selection of multi media objects through an interaction graph;
Figure 6 is a diagrammatic view of the time path of the positioning of a cursor.
Figure 1 shows by way of example a diagrammatic view of a computer display 1 which is equipped with a control according to the invention.
A cursor 3 thereby serves as a cursor of functions 10 of the computer 1. Functions 10 are for example the volume of a sound clip or the display sites of program menus.
The cursor 3 is moved by means of a mouse or another handling device over the display of the computer 1.
The control of the computer 1 according to the invention defines at some sites of the display certain partial areas in which it is registered when a position 8 of the cursor 3 lies within this partial area. These partial areas are called sensor ranges 2 in the following. The control ~3-.TUN-1999 16:29 FRDM DUMMETT COPP TO MAIKDWSKI P.lli31 _ g _ detects and stores in addition to the position 8 of the cursor 3 also the dwell time of the cursor 3 in a sensor range 2.
The shape of a sensor range 2 is thereby not rigid but can be adapted in position, shape and/or function on the displays of the computer 1 in any way to meet the requirements. It is also possible that the entire display of the computer 1 is covered with sensor ranges 2 so that 1o the dwell time of the cursor 3 is measured at any point on the screen whereby depending on the sensor range 2 different functions 10 of the computer 1 are triggered.
An overlap of sensor ranges 2 is also possible whereby the control of the computer 1 then determines in which way the dwell times are processed (e.g. evaluation, addition of the dwell times).
With the normal course of the control according to the invention the sensor ranges 2 are invisible on the display, i.e. they are hidden under the usual displays of a multi media application or text processing system. The sensor ranges 2 can however be made visible during program setting up or debugging of a program in order to check the function 10.
The method of operation of the sensor ranges 2 is described below with reference to multi media programs.
A multi media encyclopaedia according to the invention indicates for example on a screen, text, pictures and videos whereby sensor ranges 2 are placed behind at certain points of the screen) A user of the multi media encyclopaedia guides the cursor 3 into the area of the displays which are of particular 63-JUN-1999 16:29 FROM DUMMETT COPP TO MAIKOWSKI P.12i31 interest to him. If the cursor 3 is thereby guided into a sensor range 2 then the control detects the position 8 of the cursor 3 and its dwell time in this sensor range 2.
For this the control has timer functions.
The control interprets the time spent by the cursor 3 in a sensor range 2 as interest of the user and quantifies this interest as so-called energy value. In this way the performance of a user is described by a standard. The 0 energy value is stored in a data bank and thus serves as memory for the interest of a user. The control thereby ensures that the energy value is changed after some time so that a forgetting or waning interest is simulated. The control keeps an "energy household" with which it can always be ascertained which energy was used in which sensor ranges 2.
Overall the position of a cursor 3 is detected at any time through the momentary position, the momentary speed and the time spent at its momentary position 8. In a two dimensional display of a computer 1 the state can accordingly be described by five values.
The control determines the further behaviour of the computer 1 (see also Figures 2 to 4 ) in dependence on the position a and dwell time of the cursor 3. After a certain time (reaching a threshold value for the energy) cross references to theme fields used are displayed for example or a piece of music fitting in context is played.
It is thereby possible that the newly displayed pictures or pieces of music played aze each superimposed on each other and thus a continuous transition between scenes is produced (fading) .
03-JUN-1999 16:29 FROM DUMMETT COPP TO MAIKOWSKI P.13i31 The control can however control the behaviour of the computer 1 not only in deterministic dependence on the kinematic behaviour of the cursor 3. Rather multi media contents can also be selected and presented through a random generator. With an electronic encyclopaedia the possibility of "leafing" through is thereby provided for example. Through a combination of the deterministic and random-controlled selection of the contents it is possible to take into account specific associations of the user.
For example in connection with the random control in the case of artistic mufti media programs pictures and atmospheres can be created which are not repeatable and which bring out the creativity of a user. In particular random-controlled pictures and text can be used in games which thereby disclose new aspects again and again.
It is also possible in the case of mufti-person games to show several cursors 3 on a computer display 1. Sensor ranges 2 can thereby be allocated to different players which widely expands the playing possibilities_ A sensor range 2 can also be far example a menu point of an operating system of the computer 1. If the cursor 3 remains for a longer time on this sensitive menu point then this is identified by the control as increased interest and a help text is displayed for this menu point.
Additional functions can then be addressed through the position of the cursor 3 in the sensor range 2.
The control of the computer 1 according to the invention can detect and utilise the kinematic or dynamic behaviour of the cursor 3 in an even different way. Thus the control of the computer 1 not only registers the position 8 of the cursor 3 but it also measures the speed, 03-JUN-1999 16:30 FROM DUMMETT COPP TO MAIKOIJSKI P.14i31 acceleration and path of movement of the cursor 3 on the display of the computer 1. Furthermore the regions are also detected which a cursor 3 includes by opening up a window.
By detecting the path of movement of the cursor 3 the control identifies in which sequence the cursor 3 has stopped in certain sensor ranges 2. The control thereby triggers different functions of the computer 1 according to the sequence which is run through.
The control of the computer 1 can also execute at certain points of the path of movement numerical differentiation through which the speeds and accelerations are calculated at the points of the path of movement. Thus the kinematic behaviour of the cursor 3 is fully detected. These measurements of the kinematic behaviour of the cursor are also quantified as energy values.
If for example a cursor 3 is moved particularly quickly over a text then the control evaluates this as a low output of energy, i.e. the interest of the user ie evaluated as slight and only a text is displayed. If a cursor 3 is however moved slowly over a text then more energy is used. The interest is evaluated as higher which leads to a different behaviour of the computer 1, for example to the playing of a video_ Since different people also have different habits when 3o using computers 1 the kinematic behaviour of the cursor 3 depends decisively on the type of person who is using the computer. To personalise the computer 1 the kinematic behaviour of a user is stored in a data bank. The control can thus match functions to of the computer 1 to a specific user (e. g. through an expert system or a neuronal 03-JUN-1999 16:30 FROM DUMMETT COPP TO MAIKOIJSKI P.1Si31 network). It is also possible that it identifies from the kinematic behaviour of the cursor 3 that a certain behaviour of a user is not efficient and it adapts a function 10 of the computer 1 accordingly yr refers the user to the inefficiency. In the case of learning software for example this caa lead to a considerable improvement in the rate of learning.
It is likewise possible that the cursor 3 of the computer 7.0 1 is controlled through the control according to the invention by eye movements of a user. The eye movements can be detected for example through video monitoring of the pupils. Through the control of the kinematic behaviour of the cursor 3 through the eye movements and the sensor ranges 2 it is possible for persons, particularly those who dv not have full use of their hands (e.g. disabled people) to operate the computer in an efficient and flexible way.
The functions of the sensor range 2 will now be described with reference to Figures 2 to 4.
In Figure 2 a circular sensor range 2 is displayed on the display of a computer 1 with a radius 7. zf a cursor 3 such as shown in Figure 2 is located inside the sensor range 2 then the kinematic behaviour of the cursor 3 and its dwell time in the sensor range 2 is detected by the control of the computer I according to the invention.
In this case the position 8 of the cursor 3 is thereby shown in a golar co-ordinate system, with the centre point as the reference point 6 of the sensor range 2. Fixing the position 8 of the cursor 3 is produced from the distance of the cursor 3 from the reference point 6 and an angle (not shown here ) to a reference line.
03-JUN-1999 16:30 FROM DUMMETT COPP TO MAIKOIJSKI P.16~31 In differently shaped sensor ranges 2 a corner of the sensor range 2 or the centre of gravity of the sensor range 2 serves as the reference point 6 for example . In an alternative embodiment the position 8 of the cursor 3 is given in an absolute co-ordinate system of the display of the computer 1, i.e. the co-ordinates are counted from the corner of the screen display.
The control according to the invention evaluates in addition the angle co-ordinate and the dwell time at various spots of the sensor range 2 and determines from this at least one function l0 of the computer 1.
The connection between the kinematic behaviour of the cursor 3 and a funetion 10 of the computer is shown in Figures 3 and 4.
Figure 4 shows a functional connection 9 between a position 8 of the cursor 3 and a function 10 of a computer 1. The functional connection 9 is part of the control according to the invention.
Reference is thereby made to the circular shaped sensor range of Figure 2. For simplification only the influence of the radial distance of the position 8 from the reference point 6 on the function 10 (e.g. the volume of a piece of music) is shown.
Outside of the sensor range 2 (i.e. the distance of the cursor 3 from the reference point 6 is greater than the radius 7) no functions 10 of the computer 1 are triggered.
After the cursor 3 enters into the sensor range 2 the volume l0 of the piece of music is increased. If the cursor 3 reaches the reference point 6 then the volume 10 is maximum. If the cursor 3 is placed anywhere in the 03-JUN-1999 16:31 FROM DUMMETT COPP TO MAIKOIJSKI P.1?i31 sensor range 2 then the position 8 in the sensor range is detected and the functional connection 9 is evaluated at this point.
The functional connections 9 between a function 10 of the computer 1 and the position 8 of a cursor 3 can be both linear and non-linear.
In other embodiments there is a functional connection 9 in an analogous way between the speed, acceleration or dwell time of the cursor 3 in a sensor range 2. The control of the computer 1 then evaluates the various information on the kinematic behaviour of the cursor 3 and then assigns a certain function 10. In further embodiments a random generator is used in addition to determine the function 10.
Since the functional connection 9 describes the interaction of the kinematie behaviour of a cursor 3 with a function 10 of a computer these connections are also called interaction graphs.
The following input values are used typically by the control . pressing keys, mouse movements, track ball movements, data glove actions, sensor information, camera information. The input values are linked by the control through interaction graphs to the functions 10 of the computer 1. The output values (i.e. functions 10) are thereby typically . visual 2D and 3D illustrations, video information, slide projections, sound, tactile information through active sensors in data gloves.
Figure 4 shows a further functional connection 9~ between a position of the cursor 3 and a function 10'. In this 03-JUN-1999 16:31 FROM DUMMETT COPP TO MRIKOIJSKI P.1Bi31 case the function 10' exists in the opacity of a picture in a mufti media application.
Different from Figure 3, here it is nvt the spatial distance between a reference point 6 and the position 8 of the cursor 3, but the time distance from a starting time point 11 which is used as the reference point 6.
The starting time point 11 is defined by a certain action (e.g, by pressing a key, exceeding a certain dwell time of the cursor 3 in a sensor range 2) . From this time point the opacity of a picture is determined by the function connection 9', i.e. the opacity rises and reduces again after a while. If the cursor 3 is removed from the sensor range 2 at any time point 13 then the opacity 10' of the picture assigned to this time point 13 remains for a little while-Both the spatial (see Figure 3} and the time evaluation of 2o interaction graphs (see Figure ~} can be used combined.
Several functions 10 can thereby be influenced dependent on or independently of each other.
In Figure 5 it is shown from an example how a control of the computer 1 according to the invention influences functions 10 ", 10'~~ of a mufti media system through interaction graphs 9 ", Essential for the function of the control according to the invention is a data bank in which are stored all the signals measured by the control and issued by the control.
In particular the data bank contains objects 14 such as e.g. pictures, text, music, sounds, videos, programs) control commands for external apparatus, which are made 03-JUN-1999 16~31 FROM DUMMETT COPP TD MRIKOIJSKI P.19i31 accessible to the user of the computer 1. Information on sensor ranges 2 are also processed by technical programs as objects 14.
Media are stored as objects 14 of different kinds in the data bank_ The objects 14 are thereby combined by program technology in a container 15 whereby the objects 14 stored in the container 15 are grouped by contents (i.e.
pictures, texts, music vn one theme). Also a container 15 Io is in turn an object 14 with regard to programming technology. An object 14 can thereby be a member of different containers 15.
The selection of an object 14 or a certain number of objects 14 is carried out in dependence on the position 8 " , 8 " ' of the cursor 3 through the interaction graphs 9 " , 9 "'. A standard is determined from the positions 8 " , 8 " ' and/or another kinematic parameter of the cursor 3 through the interaction graphs 9 ", 9 " ' which are valid 2o at the relevant sites and/or at the relevant time. From this standard the control according to the invention establishes which object 14 or which group of objects 14 is displayed or played from the appropriate container 15.
Each object 14 thereby has attributes 16 which describe properties of the object 14. From these attributes 16 the control determines inter alia which objects 14 are displayed.
In the data bank is for example stored the picture of a Greek temple which has the attribute 16 "Building", "Greece", "Religion" and "Antiques". Depending on the context which is displayed on the display of the computer 1 the control shows the picture of the temple. If the control has determined for example that a user requires information on Greece, then it is determined in dependence 03-JUN-1999 16:32 FRDM DUMMETT COPP TO MAIKOIJSKI P.20i31 on the kinematic behaviour of the cursor 3 in the sensor ranges 2~ whether for example the picture of the temple is also displayed in addition to travel information on Greece. If a user is obtaining information on antiques through the computer 1 then again the picture of the temple can be displayed in dependence on the kinematic behaviour of the cursor 3. The attributes 16 thus produce cross connections between various objects 14 stored in a data bank. Since through program technology all information is stored as objects 14 in the data bank a multitude of interactions can be produced between the information and the kinematic behaviour of the cursor 3.
Unlike the known systems, such as for example WWW pages, the control according to the invention provides nv fixed information hierarchy where for example under the title Greece only the sub-headings "Travel Information and "Pictures" can be retrieved. Rather the information offer is fixed on the display through the control dynamically in dependence on the kinematic behaviour of the cursor 3.
Solely by the cursor 3 dwelling at a certain spot in a sensor range 2, the focus, (see Figure 6) can different information be displayed or played little by little; the control interprets the time spent in the sensor range 2 as increased interest and controls the display of the computer 1 from the relevant energy values.
Furthermore each object 14 has a modifier 17 which assigns a standard (e.g. in the range 1 to 100) to the object 14.
Frvm the modifier 17 it can be established for example with which transparency a picture is displayed, In the case of a modifier 17 with the value 100 the control show$
the picture with full opacity, the background of the display is completely obscured. With a value of 10 the picture can only be seen transparently on the screen so 03-JUN-1999 16:32 FROM DUMMETT COPP TO MRIKOLJSKI P.21i31 that elements lying behind the picture show through the picture. With a modifier 17, far example also the volume of a noise, the frequency of the display of pictures or the playing of music, the selection of a picture from a container or the sensitivity of the energy output or energy absorption can also be influenced.
Hoth the attributes 16 and the modifier 17 can be changed by the control in a predeterminable manner. It is likewise possible that attributes 16 or modifier 17 are changed by the kinematic behaviour of the cursor 3 and thus are influenced directly by the behaviour of the user_ An example here is an audio system where the playing of pieces of music is controlled in dependence on the movement of a cursor 3. If the cursor 3 interacts timewiee in succession with different sensor ranges 2, one Sensor range 2 when recalled need not necessarily play the same pieces of music as the f first time . There is rather the possibility of playing pieces of music used with a theme. In certain circumstances the interaction of the cursor 3 which has meanwhile taken place with the control signals that the interest of the user has changed. After evaluation of the information on the energy, attributes 16 and modifier Z7 the container contents are therefore compiled anew and the pieces of music then included are played.
In this way it is possible that a user of the computer 1 3o also controls the display through movements of the cursor 3 but at the same time the control leads the user from the kinematic behaviour of the cursor 3 through a program.
The navigation of the user is thus carried out in a permanent interplay between the user and the computer control according to the invention whereby the kinematic 93-JUN-1999 16:32 FROM DUMMETT COPP TO MAIKOIJSKI P.2~i31 behaviour of the cursor 3 represents the binding link.
The control according to the invention controls the interaction of the data bank and the evaluation of the kinematic behaviour of the cursor 3 so that new information is always being displayed. In this way a knowledge browser is provided with completely new properties , namel.y the creation and consideration of data rooms as well as the possibility of interaction with a cursor.
Figure 6 shows diagrammatically the influence of a function 10 of a computer 1 through the time spent by a cursor (not shown here) at the position 8.
Figure 6 thereby shows the time curve, symbolised by a time axis 18, when the cursor 3 dwells at the position s.
At first the control of the computer 1 does not react to the presence of the cursor 3 in a sensor range 2 (not shown here). Thus no functions of the computer 1 are executed.
After exceeding a certain time certain objects 14 are displayed whereby the control determines these from the attributes 16 and modifier 17. The displayed objects 14 stand in close affinity with the object 14 at position 8.
After a certain time which the control interprets as increased interest, the objects 14 are displayed whose contents have a further connection with the object 14 at the position 8. The degree of remoteness is determined through the attributes 16 and modifier 17.
If for example the position 8 is in a sensor range 2 which is assigned to the picture of a church then after a certain time different views of the church are shown, i.e.
information which iB directly connected with the selected ~3-JUN-1999 16:33 FROM DUMMETT COPP TO MAIKOWSKI P.23i31 object 14. As the time increases so pictures of churches are displayed which are to be associated with the same style. Still later church music is played from the corresponding era. In this way the user is guided by the control according to the invention through a multi media application whereby the control according tv the invention offers the user at any time the possibility of influEncing the information offered by moving the cursor 3 (i.e.
through the time spent and position inside the sensor range ) .
The direction of the time axis 18 and the alignment of a so-called focal funnel thus indicates the "direction of interest", thus the focus of the user of the computer 1.
The increasing interest is shown in Figure 6 therefore by a widening focal funnel 19;.more and more objects 14 are detected. Moving the position 8 into another sensor range 2 therefore corresponds to a change of alignment of the focal funnel 19.
The invention is not restricted in its design to the preferred embodiments indicated above. Rather a number of variations is possible which utilise the computer control according to the invention even in quite different designs.
DESCRIPTION '_~
The invention relates to a control for a computer.
Controls for multi functional systems, such as for example computers, have been known in a simple form for some time.
In particular computers are known wherein a user triggers or influences functions of the computer through a trigger, e.g. a mouse-supported cursor on a screen or a data glove for manipulating three-dimensional objects.
Thus with the nowadays widely used multi media systems which are designed as computer programs, the trigger of a function (e.g. the display of a picture, the retrieval of a program or the playing of a video) is produced by selecting a certain point on the display screen. By multi media systems are understood systems where the senses of a person are influenced by various media, such as for example by text) pictures, videos, sounds, noises or music. In principle multi media systems also include however contact (tactile) or smell (olfactory? stimulus which has an effect on a person.
From IBM Technical Disclosure Bulletin, Vol. 36, no. 11, November 1993, page 57 a method is known for activating user-interface functions of a computer. The time spent by a cursor in a small part of the display of the computer is detected. After a certain dwell time a certain function 03-JUN-1999 16:26 FPOM DUMMETT COPP TO MAIKOIJSKI P.04i31 is executed which is assigned to this part of the display.
A drawback with these computers is that the operation of the computer is only carried out through the position and where applicable the dwell time of the cursor which represents a considerable restriction on the functionality. Only one function can ever be Executed in a certain manner and way since the trigger of the function is produced by clicking on the mouse in a fixed predetermined area of the screen or by the cursor dwelling in this fixed predetermined area. The potential functionality of the computer is severely restricted by this non-differentiated use of the cursor.
The obj ect of the invention is to provide a control for a computer and a method for computer control wherein the functions of the computer can be controlled in specially differentiated ways and the functionality of the computer is increased.
This is achieved according to the invention through a control with the features of claim 1 and through a method for control with the features of claim 19.
The control of a computer according to the invention has sensor means with which the position of at least one cursor on a display of the computer is detected. If the position of the cursor is within a certain partial area of the display, the sensor range, then the dwell time and the position of the cursor in the sensor range are measured.
The information on the dwell time and the position is used for influencing at least one function of the computer.
The dwell time of the cursor outside of the sensor range is not measured.
03-JUN-1999 16:27 FFOM DUMMETT COPP TD MAIKOIJSKI P.05i31 By computer is meant any device for data processing which is fitted inter alia .With a screen and a device for controlling a cursor (e.g. mouse) digitalizer). The control of the computer according to the invention can be designed for example in the form of a processor or a program. Generally the functional units described below can be produced selectively as soft or hardware.
Thus compared with the known controls for computers not only the position of the cursor but also -the dwell time of the cursor at a spot on the screen are significant for the function of the computer. Through these additional parameters (namely position and dwell time in the sensor range) it is possible to increase the functionality (e. g_ control of audio-visual signals) of the computer and to provide new control possibilities.
One advantageous embodiment of the control according to the invention detects the speed and/or acceleration of at least one cursor and uses this information to influence at least one function of the computer. The functionality of the computer is improved through these additional parameters of the trigger. Thus a control for a computer according to the invention can react differently for example to a rapid or slow movement of a cursor.
In a further advantageous embodiment the path of movement is detected which is described by at least one cursor on the screen of the computer. At least one function of the computer is influenced in dependence on this data. Thus for example the paths of movement of the cursor on the display of the computer are characteristic for certain situations in the operation of the computer or for certain 0~ JUN-1999 16:27 FROM DUMMETT COPP TO MRIKOWSKI P.06i31 users. This information can be used to adapt the computer better to a user.
In a particularly advantageous design the control according to the invention has means with which the kinematic (dynamic) behaviour of the cursor is quantified.
sy kinematic behaviour of the cursor is meant here generally the space-time behaviour of the cursor on the computer screen, which includes in particular the dwell time, the position, the speed and the acceleration of the cursor. By quantification is meant that the kinematic behaviour of the cursor is detected through parameters or functions which describe for example the dwell time or shape of the path of movement. These parameters and functions form the input values for functional connections which link the kinematic behaviour of the cursor directly with a function of the computer. These functional connections can be stored read-only in a data bank or can be changed during the course of time. Through the quantification of the kinematic behaviour of the cursor it is possible to influence the functions of the computer in very different ways.
The quantification is used with special advantage in combination with a random generator so that particularly With computers having multi media applications or games.
novel effects can be achieved again and again.
The control advantageously has means to change the position, shape and/or function of at least one sensor range in the computer in predetermined or random way.
Thus the sensor ranges can be adapted to changing situations which increases the flexibility of the control and of the computer.
03-JUN-1999 16:27 FROM DUMMETT COPP TO MAIKOIJSKI P.07~31 In a further advantageous development of the control according to the invention a data bank serves to store the kinematic behaviour of at least one cursor. It is also advantageous to detect and store the spatial, time and/or functional changes of at least one sensor range in a data bank . Certain movements or movement patterns of the cursor can thereby be stored for example and used in a particularly advantageous way for influencing functions of the computer and/or at least one sensor range.
In a particularly advantageous design of the control according to the invention a continuous transition (fading) takes place between two different functions of the computer. In particular a number of aesthetic or useful effects can thereby be achieved between visual and/or acoustic functions.
A particularly advantageous development of the control according to the invention has a data bank in which objects are stored for influencing at least one function of the computer. At least one of these objects thereby has an attribute which describes a property of the object.
This attribute can describe for example the type of object (e. g. text) or also the contents of the object (e_g.
verse). By using attributes the control is able to easily make relationships between different objects.
In a further advantageous design of the control according to the invention at least one object and/or attribute of the object stored in the data bank has a modifier. This modifier is a standard by which the control can compare different objects or attributes with each other. A
modifier can be stored in predetermined manner in a data 03-JUN-1999 16:28 FROM DUMMETT COPP TO MRIKOIJSKI P.0Bi31 bank or can be changed by the control over the course of time_ Advantageously the control according to the invention has means by which at least one function of the computer can be controlled by the kinematic behaviour of the cursor in connection with attributes and/or modifiers of at least one object. It is thus possible that the kinematic behaviour of the cursor and the properties of the objects ZO influence the function of the computer whereby a very flexible control of the computer is possible.
The control according to the invention likewise advantageously has means with which objects, more Z5 particularly media, can be stored in the data bank automatically sorted according to type. The detection of objects (e.g. texts or pictures) which are to be used as functions of a computer can thus be considerably speeded up. The control can thus assign e.g. automatically 20 determined attributes to the objects.
At least one of the objects more advantageously has information on a sensor range, a picture, a text, a noise, a piece of music, control data far external equipment, 25 data for a three-dimensional representation, a modifier, an attribute or a group of objects. Through the general treatment of the most varied information as object it is possible to use this information in uniform way for influencing the computer.
Advantageously with a control according to the invention for a computer at least one sensor range for a cursor is filed invisible behind the display of the computer. The display thereby appears in the u5ua1 form for the user.
03-JUN-1999 16:2E FROM DUMMETT COPP TO MRIKOIJSKI P.09i31 _ 7 _ In a likewise advantageous development of the control according to the invention the cursor is controllable on the display of the computer by the eye movements of a user of the computer. The detection of the eye movements can thereby take place for example through video monitoring of the pupils or a derivative of action potentials of face muscles. The activation of the cursor through eye movements is particularly suitable for people who cannot use their hands when working at the computer.
l0 With the method according to the invention for controlling at least one function of a computer first the position of at least one cursor on a display of the computer is detected by sensor means. Through the at least one cursor the functions of the computer such as e.g. audio visual signals can be activated. The information captured by the sensor means are then transferred to the control. The control then establishes whether the at least one cursor is located in a 1- or 2- dimensional partial area (sensor range) of the display of the computer. If the position is inside the sensor range then the control determines the dwell time and the position of the at least one cursor inside the sensor range. The control finally influences the at least one function of the control in dependence on the dwell time and the position of the at least one cursor.
The invention will now be explained with reference to the embodiments shown in the drawings in which:
Figure 1 shows a diagrammatic view of a display of a computer whereby the functions of the computer can be influenced by a cursor and sensor ranges for the cursor;
03-JUN-1999 16:2E FROM DUMMETT COPP TO MRIKOIJSKI P.10i31 _ g Figure 2 is a diagrammatic illustration of a sensor range on a computer display;
Figure 3 is a diagrammatic illustration of a functional connection between the position of a cursor on the computer display and a function of the computer (interaction graph) ;
Figure 4 is a diagrammatic illustration of a functional connection between the time behaviour of a cursor and a function of the computer;
Figure 5 is an illustration of the selection of multi media objects through an interaction graph;
Figure 6 is a diagrammatic view of the time path of the positioning of a cursor.
Figure 1 shows by way of example a diagrammatic view of a computer display 1 which is equipped with a control according to the invention.
A cursor 3 thereby serves as a cursor of functions 10 of the computer 1. Functions 10 are for example the volume of a sound clip or the display sites of program menus.
The cursor 3 is moved by means of a mouse or another handling device over the display of the computer 1.
The control of the computer 1 according to the invention defines at some sites of the display certain partial areas in which it is registered when a position 8 of the cursor 3 lies within this partial area. These partial areas are called sensor ranges 2 in the following. The control ~3-.TUN-1999 16:29 FRDM DUMMETT COPP TO MAIKDWSKI P.lli31 _ g _ detects and stores in addition to the position 8 of the cursor 3 also the dwell time of the cursor 3 in a sensor range 2.
The shape of a sensor range 2 is thereby not rigid but can be adapted in position, shape and/or function on the displays of the computer 1 in any way to meet the requirements. It is also possible that the entire display of the computer 1 is covered with sensor ranges 2 so that 1o the dwell time of the cursor 3 is measured at any point on the screen whereby depending on the sensor range 2 different functions 10 of the computer 1 are triggered.
An overlap of sensor ranges 2 is also possible whereby the control of the computer 1 then determines in which way the dwell times are processed (e.g. evaluation, addition of the dwell times).
With the normal course of the control according to the invention the sensor ranges 2 are invisible on the display, i.e. they are hidden under the usual displays of a multi media application or text processing system. The sensor ranges 2 can however be made visible during program setting up or debugging of a program in order to check the function 10.
The method of operation of the sensor ranges 2 is described below with reference to multi media programs.
A multi media encyclopaedia according to the invention indicates for example on a screen, text, pictures and videos whereby sensor ranges 2 are placed behind at certain points of the screen) A user of the multi media encyclopaedia guides the cursor 3 into the area of the displays which are of particular 63-JUN-1999 16:29 FROM DUMMETT COPP TO MAIKOWSKI P.12i31 interest to him. If the cursor 3 is thereby guided into a sensor range 2 then the control detects the position 8 of the cursor 3 and its dwell time in this sensor range 2.
For this the control has timer functions.
The control interprets the time spent by the cursor 3 in a sensor range 2 as interest of the user and quantifies this interest as so-called energy value. In this way the performance of a user is described by a standard. The 0 energy value is stored in a data bank and thus serves as memory for the interest of a user. The control thereby ensures that the energy value is changed after some time so that a forgetting or waning interest is simulated. The control keeps an "energy household" with which it can always be ascertained which energy was used in which sensor ranges 2.
Overall the position of a cursor 3 is detected at any time through the momentary position, the momentary speed and the time spent at its momentary position 8. In a two dimensional display of a computer 1 the state can accordingly be described by five values.
The control determines the further behaviour of the computer 1 (see also Figures 2 to 4 ) in dependence on the position a and dwell time of the cursor 3. After a certain time (reaching a threshold value for the energy) cross references to theme fields used are displayed for example or a piece of music fitting in context is played.
It is thereby possible that the newly displayed pictures or pieces of music played aze each superimposed on each other and thus a continuous transition between scenes is produced (fading) .
03-JUN-1999 16:29 FROM DUMMETT COPP TO MAIKOWSKI P.13i31 The control can however control the behaviour of the computer 1 not only in deterministic dependence on the kinematic behaviour of the cursor 3. Rather multi media contents can also be selected and presented through a random generator. With an electronic encyclopaedia the possibility of "leafing" through is thereby provided for example. Through a combination of the deterministic and random-controlled selection of the contents it is possible to take into account specific associations of the user.
For example in connection with the random control in the case of artistic mufti media programs pictures and atmospheres can be created which are not repeatable and which bring out the creativity of a user. In particular random-controlled pictures and text can be used in games which thereby disclose new aspects again and again.
It is also possible in the case of mufti-person games to show several cursors 3 on a computer display 1. Sensor ranges 2 can thereby be allocated to different players which widely expands the playing possibilities_ A sensor range 2 can also be far example a menu point of an operating system of the computer 1. If the cursor 3 remains for a longer time on this sensitive menu point then this is identified by the control as increased interest and a help text is displayed for this menu point.
Additional functions can then be addressed through the position of the cursor 3 in the sensor range 2.
The control of the computer 1 according to the invention can detect and utilise the kinematic or dynamic behaviour of the cursor 3 in an even different way. Thus the control of the computer 1 not only registers the position 8 of the cursor 3 but it also measures the speed, 03-JUN-1999 16:30 FROM DUMMETT COPP TO MAIKOIJSKI P.14i31 acceleration and path of movement of the cursor 3 on the display of the computer 1. Furthermore the regions are also detected which a cursor 3 includes by opening up a window.
By detecting the path of movement of the cursor 3 the control identifies in which sequence the cursor 3 has stopped in certain sensor ranges 2. The control thereby triggers different functions of the computer 1 according to the sequence which is run through.
The control of the computer 1 can also execute at certain points of the path of movement numerical differentiation through which the speeds and accelerations are calculated at the points of the path of movement. Thus the kinematic behaviour of the cursor 3 is fully detected. These measurements of the kinematic behaviour of the cursor are also quantified as energy values.
If for example a cursor 3 is moved particularly quickly over a text then the control evaluates this as a low output of energy, i.e. the interest of the user ie evaluated as slight and only a text is displayed. If a cursor 3 is however moved slowly over a text then more energy is used. The interest is evaluated as higher which leads to a different behaviour of the computer 1, for example to the playing of a video_ Since different people also have different habits when 3o using computers 1 the kinematic behaviour of the cursor 3 depends decisively on the type of person who is using the computer. To personalise the computer 1 the kinematic behaviour of a user is stored in a data bank. The control can thus match functions to of the computer 1 to a specific user (e. g. through an expert system or a neuronal 03-JUN-1999 16:30 FROM DUMMETT COPP TO MAIKOIJSKI P.1Si31 network). It is also possible that it identifies from the kinematic behaviour of the cursor 3 that a certain behaviour of a user is not efficient and it adapts a function 10 of the computer 1 accordingly yr refers the user to the inefficiency. In the case of learning software for example this caa lead to a considerable improvement in the rate of learning.
It is likewise possible that the cursor 3 of the computer 7.0 1 is controlled through the control according to the invention by eye movements of a user. The eye movements can be detected for example through video monitoring of the pupils. Through the control of the kinematic behaviour of the cursor 3 through the eye movements and the sensor ranges 2 it is possible for persons, particularly those who dv not have full use of their hands (e.g. disabled people) to operate the computer in an efficient and flexible way.
The functions of the sensor range 2 will now be described with reference to Figures 2 to 4.
In Figure 2 a circular sensor range 2 is displayed on the display of a computer 1 with a radius 7. zf a cursor 3 such as shown in Figure 2 is located inside the sensor range 2 then the kinematic behaviour of the cursor 3 and its dwell time in the sensor range 2 is detected by the control of the computer I according to the invention.
In this case the position 8 of the cursor 3 is thereby shown in a golar co-ordinate system, with the centre point as the reference point 6 of the sensor range 2. Fixing the position 8 of the cursor 3 is produced from the distance of the cursor 3 from the reference point 6 and an angle (not shown here ) to a reference line.
03-JUN-1999 16:30 FROM DUMMETT COPP TO MAIKOIJSKI P.16~31 In differently shaped sensor ranges 2 a corner of the sensor range 2 or the centre of gravity of the sensor range 2 serves as the reference point 6 for example . In an alternative embodiment the position 8 of the cursor 3 is given in an absolute co-ordinate system of the display of the computer 1, i.e. the co-ordinates are counted from the corner of the screen display.
The control according to the invention evaluates in addition the angle co-ordinate and the dwell time at various spots of the sensor range 2 and determines from this at least one function l0 of the computer 1.
The connection between the kinematic behaviour of the cursor 3 and a funetion 10 of the computer is shown in Figures 3 and 4.
Figure 4 shows a functional connection 9 between a position 8 of the cursor 3 and a function 10 of a computer 1. The functional connection 9 is part of the control according to the invention.
Reference is thereby made to the circular shaped sensor range of Figure 2. For simplification only the influence of the radial distance of the position 8 from the reference point 6 on the function 10 (e.g. the volume of a piece of music) is shown.
Outside of the sensor range 2 (i.e. the distance of the cursor 3 from the reference point 6 is greater than the radius 7) no functions 10 of the computer 1 are triggered.
After the cursor 3 enters into the sensor range 2 the volume l0 of the piece of music is increased. If the cursor 3 reaches the reference point 6 then the volume 10 is maximum. If the cursor 3 is placed anywhere in the 03-JUN-1999 16:31 FROM DUMMETT COPP TO MAIKOIJSKI P.1?i31 sensor range 2 then the position 8 in the sensor range is detected and the functional connection 9 is evaluated at this point.
The functional connections 9 between a function 10 of the computer 1 and the position 8 of a cursor 3 can be both linear and non-linear.
In other embodiments there is a functional connection 9 in an analogous way between the speed, acceleration or dwell time of the cursor 3 in a sensor range 2. The control of the computer 1 then evaluates the various information on the kinematic behaviour of the cursor 3 and then assigns a certain function 10. In further embodiments a random generator is used in addition to determine the function 10.
Since the functional connection 9 describes the interaction of the kinematie behaviour of a cursor 3 with a function 10 of a computer these connections are also called interaction graphs.
The following input values are used typically by the control . pressing keys, mouse movements, track ball movements, data glove actions, sensor information, camera information. The input values are linked by the control through interaction graphs to the functions 10 of the computer 1. The output values (i.e. functions 10) are thereby typically . visual 2D and 3D illustrations, video information, slide projections, sound, tactile information through active sensors in data gloves.
Figure 4 shows a further functional connection 9~ between a position of the cursor 3 and a function 10'. In this 03-JUN-1999 16:31 FROM DUMMETT COPP TO MRIKOIJSKI P.1Bi31 case the function 10' exists in the opacity of a picture in a mufti media application.
Different from Figure 3, here it is nvt the spatial distance between a reference point 6 and the position 8 of the cursor 3, but the time distance from a starting time point 11 which is used as the reference point 6.
The starting time point 11 is defined by a certain action (e.g, by pressing a key, exceeding a certain dwell time of the cursor 3 in a sensor range 2) . From this time point the opacity of a picture is determined by the function connection 9', i.e. the opacity rises and reduces again after a while. If the cursor 3 is removed from the sensor range 2 at any time point 13 then the opacity 10' of the picture assigned to this time point 13 remains for a little while-Both the spatial (see Figure 3} and the time evaluation of 2o interaction graphs (see Figure ~} can be used combined.
Several functions 10 can thereby be influenced dependent on or independently of each other.
In Figure 5 it is shown from an example how a control of the computer 1 according to the invention influences functions 10 ", 10'~~ of a mufti media system through interaction graphs 9 ", Essential for the function of the control according to the invention is a data bank in which are stored all the signals measured by the control and issued by the control.
In particular the data bank contains objects 14 such as e.g. pictures, text, music, sounds, videos, programs) control commands for external apparatus, which are made 03-JUN-1999 16~31 FROM DUMMETT COPP TD MRIKOIJSKI P.19i31 accessible to the user of the computer 1. Information on sensor ranges 2 are also processed by technical programs as objects 14.
Media are stored as objects 14 of different kinds in the data bank_ The objects 14 are thereby combined by program technology in a container 15 whereby the objects 14 stored in the container 15 are grouped by contents (i.e.
pictures, texts, music vn one theme). Also a container 15 Io is in turn an object 14 with regard to programming technology. An object 14 can thereby be a member of different containers 15.
The selection of an object 14 or a certain number of objects 14 is carried out in dependence on the position 8 " , 8 " ' of the cursor 3 through the interaction graphs 9 " , 9 "'. A standard is determined from the positions 8 " , 8 " ' and/or another kinematic parameter of the cursor 3 through the interaction graphs 9 ", 9 " ' which are valid 2o at the relevant sites and/or at the relevant time. From this standard the control according to the invention establishes which object 14 or which group of objects 14 is displayed or played from the appropriate container 15.
Each object 14 thereby has attributes 16 which describe properties of the object 14. From these attributes 16 the control determines inter alia which objects 14 are displayed.
In the data bank is for example stored the picture of a Greek temple which has the attribute 16 "Building", "Greece", "Religion" and "Antiques". Depending on the context which is displayed on the display of the computer 1 the control shows the picture of the temple. If the control has determined for example that a user requires information on Greece, then it is determined in dependence 03-JUN-1999 16:32 FRDM DUMMETT COPP TO MAIKOIJSKI P.20i31 on the kinematic behaviour of the cursor 3 in the sensor ranges 2~ whether for example the picture of the temple is also displayed in addition to travel information on Greece. If a user is obtaining information on antiques through the computer 1 then again the picture of the temple can be displayed in dependence on the kinematic behaviour of the cursor 3. The attributes 16 thus produce cross connections between various objects 14 stored in a data bank. Since through program technology all information is stored as objects 14 in the data bank a multitude of interactions can be produced between the information and the kinematic behaviour of the cursor 3.
Unlike the known systems, such as for example WWW pages, the control according to the invention provides nv fixed information hierarchy where for example under the title Greece only the sub-headings "Travel Information and "Pictures" can be retrieved. Rather the information offer is fixed on the display through the control dynamically in dependence on the kinematic behaviour of the cursor 3.
Solely by the cursor 3 dwelling at a certain spot in a sensor range 2, the focus, (see Figure 6) can different information be displayed or played little by little; the control interprets the time spent in the sensor range 2 as increased interest and controls the display of the computer 1 from the relevant energy values.
Furthermore each object 14 has a modifier 17 which assigns a standard (e.g. in the range 1 to 100) to the object 14.
Frvm the modifier 17 it can be established for example with which transparency a picture is displayed, In the case of a modifier 17 with the value 100 the control show$
the picture with full opacity, the background of the display is completely obscured. With a value of 10 the picture can only be seen transparently on the screen so 03-JUN-1999 16:32 FROM DUMMETT COPP TO MRIKOLJSKI P.21i31 that elements lying behind the picture show through the picture. With a modifier 17, far example also the volume of a noise, the frequency of the display of pictures or the playing of music, the selection of a picture from a container or the sensitivity of the energy output or energy absorption can also be influenced.
Hoth the attributes 16 and the modifier 17 can be changed by the control in a predeterminable manner. It is likewise possible that attributes 16 or modifier 17 are changed by the kinematic behaviour of the cursor 3 and thus are influenced directly by the behaviour of the user_ An example here is an audio system where the playing of pieces of music is controlled in dependence on the movement of a cursor 3. If the cursor 3 interacts timewiee in succession with different sensor ranges 2, one Sensor range 2 when recalled need not necessarily play the same pieces of music as the f first time . There is rather the possibility of playing pieces of music used with a theme. In certain circumstances the interaction of the cursor 3 which has meanwhile taken place with the control signals that the interest of the user has changed. After evaluation of the information on the energy, attributes 16 and modifier Z7 the container contents are therefore compiled anew and the pieces of music then included are played.
In this way it is possible that a user of the computer 1 3o also controls the display through movements of the cursor 3 but at the same time the control leads the user from the kinematic behaviour of the cursor 3 through a program.
The navigation of the user is thus carried out in a permanent interplay between the user and the computer control according to the invention whereby the kinematic 93-JUN-1999 16:32 FROM DUMMETT COPP TO MAIKOIJSKI P.2~i31 behaviour of the cursor 3 represents the binding link.
The control according to the invention controls the interaction of the data bank and the evaluation of the kinematic behaviour of the cursor 3 so that new information is always being displayed. In this way a knowledge browser is provided with completely new properties , namel.y the creation and consideration of data rooms as well as the possibility of interaction with a cursor.
Figure 6 shows diagrammatically the influence of a function 10 of a computer 1 through the time spent by a cursor (not shown here) at the position 8.
Figure 6 thereby shows the time curve, symbolised by a time axis 18, when the cursor 3 dwells at the position s.
At first the control of the computer 1 does not react to the presence of the cursor 3 in a sensor range 2 (not shown here). Thus no functions of the computer 1 are executed.
After exceeding a certain time certain objects 14 are displayed whereby the control determines these from the attributes 16 and modifier 17. The displayed objects 14 stand in close affinity with the object 14 at position 8.
After a certain time which the control interprets as increased interest, the objects 14 are displayed whose contents have a further connection with the object 14 at the position 8. The degree of remoteness is determined through the attributes 16 and modifier 17.
If for example the position 8 is in a sensor range 2 which is assigned to the picture of a church then after a certain time different views of the church are shown, i.e.
information which iB directly connected with the selected ~3-JUN-1999 16:33 FROM DUMMETT COPP TO MAIKOWSKI P.23i31 object 14. As the time increases so pictures of churches are displayed which are to be associated with the same style. Still later church music is played from the corresponding era. In this way the user is guided by the control according to the invention through a multi media application whereby the control according tv the invention offers the user at any time the possibility of influEncing the information offered by moving the cursor 3 (i.e.
through the time spent and position inside the sensor range ) .
The direction of the time axis 18 and the alignment of a so-called focal funnel thus indicates the "direction of interest", thus the focus of the user of the computer 1.
The increasing interest is shown in Figure 6 therefore by a widening focal funnel 19;.more and more objects 14 are detected. Moving the position 8 into another sensor range 2 therefore corresponds to a change of alignment of the focal funnel 19.
The invention is not restricted in its design to the preferred embodiments indicated above. Rather a number of variations is possible which utilise the computer control according to the invention even in quite different designs.
Claims (19)
1. Control for at least one function (10) of a computer (1) with a) sensor means (5) for detecting the position (8) of at least one cursor (3) on a display of the computer (1), b) means for detecting the time spent by the cursor (3) in at least a 1- or 2- dimensional partial area (sensor range 2) of the display of the computer (1) c) means which influence the function (10) of the computer (1) in dependence on the position (8) and the dwell time of the cursor (3) in the sensor range (2), characterised in that d) the function (10) of the computer (1) is influenced in dependence on the relevant momentary position (8) of the cursor (3) in the sensor range (2).
2. Control according to claim 1 characterised by means for detecting the speed and/or the acceleration of the cursor (3) whereby this kinematic information can be used to influence at least one function (10) of the computer (1).
3. Control according to one of claims 1 or 2 characterised by means for detecting a path of movement described by the cursor (3) on the screen of the computer (1) whereby this information can be used to influence at least one function (l0) of the computer (1).
4. Control according to at least one of the preceding claims characterised by means for quantifying the kinematic behaviour of the cursor (3) wherein at least one function (10) of the computer (1) can be purposely influenced in dependence on the quantification through a predeterminable or variable functional connection (9).
5. Control according to at least one of the preceding claims characterised by means for quantifying the kinematic behaviour of the cursor (3) wherein at least one function (10) can be purposely influenced in dependence on the quantification and through a random generator.
6. Control according to at least one of the preceding claims characterised by means for the predeterminable change in the position, configuration and/or function of at least one sensor range (2).
7. Control according to at least one of the preceding claims characterised by means for the random-controlled change in the position, configuration and/or function of at least one sensor range (2).
8. Control according to at least one of the preceding claims characterised by a data bank for storing the kinematic behaviour of at least one cursor (3).
9. Control according to at. least one of the preceding claims characterised by a data bank in which time, spatial and/or functional changes of at least one sensor range (2) can be stored.
10. Control according to one of the preceding claims characterised by means of using the data stored in the data bank to influence at least one function (10) of the computer (1) and/or at least one sensor range (2).
11. Control according to at least one of the preceding claims characterised by means for providing a fading of the visual and/or acoustic functions of the computer (1).
12. Control according to at least one of the preceding claims, characterised in that at least one object (14) stored in the data bank has for influencing at least one function (10) of the computer (1) an attribute (16) which describes a property of the object (14).
13. Control according to at least one of the preceding claims characterised in that at least one of the objects (14) stored in the data bank and/or at least one of the attributes (16) of the object (14) for influencing at least one function (10) of the computer (1) has a modifier (17) which assigns to the object (14) and/or to the attribute (16) a predeterminable or changeable standard with which a qualitative comparison is possible between the objects (14).
14. Control according to at least one of the preceding claims characterised by means with which at least one function (10) of the computer (1) can be controlled through the kinematic behaviour of the cursor (3) in connection with attributes (16) and/or modifiers (17).
15. Control according to at least one of the preceding claims characterised by means with which objects (14) can be stored in the data bank automatically sorted according to type.
16. Control according to at least one of the preceding claims characterised in that the object (14) is information on a sensor range (2), a picture, a text, a noise, a piece of music, control data for external apparatus, data for a three-dimensional representation, a modifier (17), an attribute (16) or a group of objects (14).
17. Control according to at least one of the preceding claims characterised in that at least one sensor range (2) is invisible behind the display of the computer (1).
18. Control according to at least one of the preceding claims characterised in that the cursor (3) is controllable on the display of the computer (1) through eye movements of a user of the computer (1).
19. Method for the control of at least one function (10) of a computer (1) wherein a) the position (8) of at least one cursor (3) is detected on a display of the computer (1), b) the time spent by the cursor (3) in at least a 1- or 2- dimensional partial area (sensor range 2) of the display of the computer (1) is detected and c) the function (10) of the computer (1) is influenced in dependence on the position (8) and the dwell time of the cursor (3) in the sensor range (2), characterised in that d) the function (10) of the computer (1) is influenced in dependence on the relevant momentary position (8) of the cursor (3) in the sensor range (2).
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE19653682.0 | 1996-12-13 | ||
| DE1996153682 DE19653682C2 (en) | 1996-12-13 | 1996-12-13 | Control device and method for at least one device of a room, and room with control device |
| PCT/DE1997/002970 WO1998026346A1 (en) | 1996-12-13 | 1997-12-15 | Computer control system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2274786A1 true CA2274786A1 (en) | 1998-06-18 |
Family
ID=7815787
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002274702A Abandoned CA2274702A1 (en) | 1996-12-13 | 1997-12-15 | Device for control of rooms |
| CA002274786A Abandoned CA2274786A1 (en) | 1996-12-13 | 1997-12-15 | Computer control system |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002274702A Abandoned CA2274702A1 (en) | 1996-12-13 | 1997-12-15 | Device for control of rooms |
Country Status (5)
| Country | Link |
|---|---|
| EP (2) | EP1015960A1 (en) |
| JP (2) | JP2000512467A (en) |
| CA (2) | CA2274702A1 (en) |
| DE (1) | DE19654944A1 (en) |
| WO (2) | WO1998026345A1 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6393407B1 (en) * | 1997-09-11 | 2002-05-21 | Enliven, Inc. | Tracking user micro-interactions with web page advertising |
| DE10125309C1 (en) * | 2001-05-21 | 2002-12-12 | Humatic Gmbh | Method and arrangement for controlling audiovisual media content |
| KR100575906B1 (en) | 2002-10-25 | 2006-05-02 | 미츠비시 후소 트럭 앤드 버스 코포레이션 | Hand pattern switching apparatus |
| JP2005242694A (en) | 2004-02-26 | 2005-09-08 | Mitsubishi Fuso Truck & Bus Corp | Hand pattern switching apparatus |
| DE102007057799A1 (en) * | 2007-11-30 | 2009-06-10 | Tvinfo Internet Gmbh | Graphical user interface for computer, comprises switching surface for activating function and pointing unit with pointer for selecting switching surface |
| DE102011102038A1 (en) * | 2011-05-19 | 2012-11-22 | Rwe Effizienz Gmbh | A home automation control system and method for controlling a home automation control system |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4109145A (en) * | 1974-05-20 | 1978-08-22 | Honeywell Inc. | Apparatus being controlled by movement of the eye |
| AU6409986A (en) * | 1985-10-07 | 1987-04-24 | Sigalov, H. | Light beam control signals for musical instruments |
| US4896291A (en) * | 1988-05-20 | 1990-01-23 | International Business Machines Corporation | Valuator menu for use as a graphical user interface tool |
| CA2012796C (en) * | 1989-06-16 | 1996-05-14 | Bradley James Beitel | Trigger field display selection |
| US5107746A (en) * | 1990-02-26 | 1992-04-28 | Will Bauer | Synthesizer for sounds in response to three dimensional displacement of a body |
| JP3138058B2 (en) * | 1992-05-25 | 2001-02-26 | 東芝キヤリア株式会社 | Ventilation fan control device |
| US5196838A (en) * | 1990-12-28 | 1993-03-23 | Apple Computer, Inc. | Intelligent scrolling |
| US5534917A (en) * | 1991-05-09 | 1996-07-09 | Very Vivid, Inc. | Video image based control system |
| JPH05108258A (en) * | 1991-10-14 | 1993-04-30 | Nintendo Co Ltd | Coordinate data generator |
| US5326028A (en) * | 1992-08-24 | 1994-07-05 | Sanyo Electric Co., Ltd. | System for detecting indoor conditions and air conditioner incorporating same |
| US5448693A (en) * | 1992-12-29 | 1995-09-05 | International Business Machines Corporation | Method and system for visually displaying information on user interaction with an object within a data processing system |
| DE4406668C2 (en) * | 1993-04-27 | 1996-09-12 | Hewlett Packard Co | Method and device for operating a touch-sensitive display device |
| US5452240A (en) * | 1993-11-23 | 1995-09-19 | Roca Productions, Inc. | Electronically simulated rotary-type cardfile |
| JP3546337B2 (en) * | 1993-12-21 | 2004-07-28 | ゼロックス コーポレイション | User interface device for computing system and method of using graphic keyboard |
| US5704836A (en) * | 1995-03-23 | 1998-01-06 | Perception Systems, Inc. | Motion-based command generation technology |
-
1996
- 1996-12-13 DE DE19654944A patent/DE19654944A1/en not_active Withdrawn
-
1997
- 1997-12-15 JP JP10526101A patent/JP2000512467A/en active Pending
- 1997-12-15 WO PCT/DE1997/002969 patent/WO1998026345A1/en not_active Application Discontinuation
- 1997-12-15 EP EP97953649A patent/EP1015960A1/en not_active Withdrawn
- 1997-12-15 CA CA002274702A patent/CA2274702A1/en not_active Abandoned
- 1997-12-15 WO PCT/DE1997/002970 patent/WO1998026346A1/en not_active Application Discontinuation
- 1997-12-15 EP EP97953648A patent/EP1015959A1/en not_active Withdrawn
- 1997-12-15 JP JP10526102A patent/JP2000512415A/en active Pending
- 1997-12-15 CA CA002274786A patent/CA2274786A1/en not_active Abandoned
Also Published As
| Publication number | Publication date |
|---|---|
| WO1998026346A1 (en) | 1998-06-18 |
| CA2274702A1 (en) | 1998-06-18 |
| DE19654944A1 (en) | 1998-06-25 |
| JP2000512467A (en) | 2000-09-19 |
| EP1015959A1 (en) | 2000-07-05 |
| EP1015960A1 (en) | 2000-07-05 |
| WO1998026345A1 (en) | 1998-06-18 |
| JP2000512415A (en) | 2000-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11886694B2 (en) | Apparatuses for controlling unmanned aerial vehicles and methods for making and using same | |
| KR102357633B1 (en) | Conversation detection | |
| US11775074B2 (en) | Apparatuses, systems, and/or interfaces for embedding selfies into or onto images captured by mobile or wearable devices and method for implementing same | |
| Cipolla et al. | Computer vision for human-machine interaction | |
| CN101482794B (en) | Mode-based graphical user interfaces for touch sensitive input devices | |
| US11663820B2 (en) | Interfaces, systems and apparatuses for constructing 3D AR environment overlays, and methods for making and using same | |
| US9348421B2 (en) | Gesture and touch-based interactivity with objects using 3D zones in an interactive system | |
| US20090125824A1 (en) | User interface with physics engine for natural gestural control | |
| CN108495165B (en) | Bullet screen play control method, mobile terminal and bullet screen play control system | |
| KR20130112061A (en) | Natural gesture based user interface methods and systems | |
| US7577915B2 (en) | Dynamic reconfiguration of inputs based on contextual focus | |
| WO2022055821A1 (en) | Method of displaying user interfaces in an environment and corresponding electronic device and computer readable storage medium | |
| CN110147191A (en) | A kind of method, apparatus and storage medium controlling window | |
| CA2274786A1 (en) | Computer control system | |
| DE19653682A1 (en) | Control panel | |
| US11782548B1 (en) | Speed adapted touch detection | |
| EP2219097A1 (en) | Man-machine interface method executed by an interactive device | |
| Hwang | Exploring real-time video interactivity with Scratch | |
| D’Amico et al. | Natural human–computer interaction | |
| Rocchesso et al. | Contact sounds for continuous feedback | |
| GB2560344A (en) | Method of control | |
| KR101630404B1 (en) | Apparatus and method for display cartoon data | |
| Nam-gu | The User Experience of Smart-Phone Information Hierarchy and Screen Transition Patterns | |
| Sundar et al. | 3D interactive interface using a kinect sensor | |
| Bowers et al. | The Lightwork performance: algorithmically mediated interaction for virtual environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued |