RU2242043C2 - Method for operation of user interface of portable data processing device - Google Patents

Abstract

FIELD: portable data processing devices.

SUBSTANCE: method includes changing data on display of device for search or selection of data, while data on display change in response to certain displacements of device in three-dimensional space of its use, while said displacements of device and respective changes of data are determined in such a way, that displacement and respective change of data on display have causation, which is analogical to causation of three-dimensional space of senses and actions of user.

EFFECT: simplified operation.

10 cl, 16 dwg

Description

The present invention relates to a method of operating a user interface of a portable data processing device. By a portable device is meant a miniature or PDA, mobile phone, communicator or other device, the essential features of which include displaying data on the device’s display and in which the data on the device’s display is changed to find and select the required data.

The most common type of user interfaces of the above devices currently operates on the principle of "point and click." The usual way this user interface works is to browse and select data using the keys. Handheld computers are usually controlled by touch displays, virtual keyboards, a stylus, and often with a few extra hard-coded keys or buttons. Firstly, this type of user interface is difficult to use with small sizes of devices and, accordingly, keys or virtual keys. On the other hand, use becomes more difficult as the devices are continuously provided with new applications and services that must be used with the same number of small keys. Typical of these user interfaces, like the user interfaces of computers and other data processing devices, is that they generally have their own artificial logic and rules that are fully accepted and understandable only to a small number of technically oriented users.

It is an object of the present invention to provide a method for operating a user interface of portable devices that substantially eliminates many of the aforementioned problems.

To achieve this result, the method of operation of the user interface of a portable device, such as a miniature or PDA, mobile phone, communicator, etc. a device in which the data on the display of the device is changed to search and select the required data, in accordance with the invention differs in the features set forth in paragraph 1 of the claims. Other points are various embodiments of the invention.

An advantage of the invention is that the user can easily accept the operating rules of the user interface since they are “natural”. Using the device is ergonomically simpler, so there is no need to use small keys or, for example, auxiliary devices.

The invention and some embodiments thereof are explained in more detail below with reference to the drawings, in which the following is presented:

figure 1-10 is a schematic spatial representation of examples of the implementation of the method of operation of the user interface corresponding to the invention;

11-13 are schematic front views of the device and the data presented on its display, illustrating some additional examples of the method of operation of the user interface corresponding to the invention;

FIG. 14 is a schematic diagram of a data configuration and operation of one of the user interface applications in accordance with the invention; FIG.

FIG. 15 is a schematic diagram of the operation of yet another user interface application in accordance with the invention; FIG.

Fig is a block diagram representing a possible implementation of the method of operation of the user interface in accordance with the invention.

As shown in figures 1-5, the user holds the portable device 2 in his hand 3, and the device has a display 2 '. The display 2 'is a touch display, and the user is able to enter a signal into the device by pressing the display with his thumb 4. The movements of the device 2 are detected by the acceleration measurement circuit installed, for example, in the device itself, and the data obtained from the circuit are used during operation user interface. Some possible ways to implement the user interface operation method in accordance with the invention are described below. Figure 1-5 presents examples of the user interface.

As shown in FIG. 1, pages 5a, 5b, 5c, 5d, 5e, which may be, for example, weather maps resulting from a web search, are viewed on the display 2 'in response to the movement of the device 2 essentially perpendicular to it display 2 ', as shown by arrow F. It is easy for the user to imagine that he has a stack of images 5a-5e in front of him, and that by moving the images in the stack he can see the image currently in front of him. In the same way you can imagine viewing the sequential pages of a book or other document. The viewing speed can be made dependent on the speed or intensity of movement, i.e. from the magnitude of the acceleration when moving. The principle of operation can also be implemented in such a way that a corresponding small movement forward or backward and an immediate stop will transfer the next page in the corresponding direction to the display. In addition, this can be done in such a way that pressing the touch display 2 'introduces a signal into the display control device, and in response to this signal, the data change on the display is stopped. This type of virtual use of depth greatly increases the capacity of a limited sized display.

As shown in FIG. 2, the image scale on the display 2 ′ increases to image 6b or decreases to image 6a in response to moving the device 2 in the same way as described above, essentially perpendicular to the display 2 ′, as shown by arrow Z. This the way the user interface works is completely analogous to the action of a person when he wants to look closer at the details of an object, such as an image. After finding the desired magnification, the data change on the display can be stopped, for example, by pressing the display 2 'again. The functions shown in figures 1 and 2 can be used alternately in the user interface according to the application, and there is no difference what happens in response to the movement of the device in this way. The functions illustrated in FIGS. 1 and 2 can also be combined during the operation of the user interface, so that, for example, the data is first viewed, then its scale is changed using the same movement. In this case, it is preferable to provide an appropriate selector for selecting one or the other function. The zoom selector can be implemented, for example, by pressing a specific area on the display, and the transition from the zoom mode to the viewing mode can occur automatically by selecting the desired zoom ratio.

3, reference numeral 7a denotes image data whose dimensions are significantly larger than the dimensions of the display. The display capacity of a portable device is not large enough for a large collection of factual data or image data. In response to the lateral movement of the device essentially in the plane of the display 2 ', as shown by the arrows N, E, S, W, the data 7b on the display changes as if some window were moving along the large image formed by the image data 7a. After finding the desired location, the change of data 7b on the display can be stopped again by pressing the display. It is logical to perform the functions according to FIG. 2 and 3 together, wherein the image data can be viewed more thoroughly after scaling by selecting the data in such a manner as described in FIG. 3. Moreover, in this way it is possible to implement, for example, selecting a number without keys by moving the device over a large virtual keyboard and selecting the desired numbers one at a time. This function allows you to read full-length web pages, for example, using a miniature computer.

As shown in FIG. 4, in response to the rapid pivoting movement of the device 2 as if around an axis formed essentially by its edge 9, i.e. in response to the rocking of the device, like turning the pages of a book, as shown by arrow P, the data 8a, 8b, 8c on the display 2 'changes accordingly, for example, the page of the electronic book is turned over. By selecting the swing direction, the page turning direction is selected. The function can be implemented in such a way that a small swing with a quick stop flips one page in the direction of swing, and in response to a more intense swing, the pages are viewed more quickly until the viewing is stopped by pressing the display.

FIG. 5 illustrates an application in which any part 9b of image data 9a forming a panoramic image spanning the full scope of the perceptual space (or, accordingly, image data forming a cylindrical image) can be viewed at any given time. In the method of operating the user interface according to the invention, in response to moving the device 2, as if moving the image on a corresponding spherical surface or in response to a change in the orientation of the device, that part 9b of the image data 9a that corresponds to the display orientation is displayed. At the same time, stopping and zooming can be applied in the same way as described above. It is also possible that the panoramic image is rotated by pressing together with moving the device in the desired direction, and the panoramic image accordingly continues to move until it is stopped, for example, by pressing the display.

6-10 represent embodiments of a user interface according to the invention, in which the data on the display is changed by tilting the device. FIG. 6 shows an example in which the data illustrated by the data D1 on the display 2 ′ of the device 2 scrolls along the path shown by the arrow R1 in the direction in which the device tilts, as shown by the arrow T1. The initial position of the device is shown by a dashed line, and the inclined position is shown by a solid line. In other words, in response to the device tilting, the data on the display scrolls or runs in the direction in which the device tilts, which is completely analogous to the cause-effect relationships in real space. This can be realized in such a way that, for example, the higher the rate of change, the more the device tilts relative to the initial position. Scrolling or data run can be stopped by returning to the initial position or, for example, using the key on the device.

7 shows an embodiment suitable, for example, for moving and searching for objects on a map. On the display 2 'is placed the cursor C, which retains its position; and tilting the device in such a manner as shown by arrow T2 causes the data shown as data D2 to begin to move from the tilt direction toward cursor C as indicated by arrow M2. In other words, the cursor moves along the data forming, for example, a map, in the direction in which the device is tilted.

In the example of FIG. 8, data on the display 2 ′ of device 2 is represented by data D3. When the user tilts the device 2 from the position indicated by the dashed line to the side away from itself, similar to approaching the data on the display, the data on the display is enlarged, which is illustrated by a graph showing the increase in data D3 and arrow Z2. In the example shown in Fig. 9, the data D4 on the display is respectively reduced, as shown by a graph illustrating the reduction, and by the arrow Z3, which is realized by tilting the device to the user in the manner shown by the arrow T4.

The options shown in FIGS. 6, 7, 8, and 9 can be combined in the user interface of the device in such a way, for example, that the device has a key for selecting the operating mode (scrolling or scaling), while first searching for the object by tilting to scrolling mode, and after finding the object, it enters the zoom mode, and the tilt leads to scaling of the data on the display. Scrolling by tilting, of course, can be combined with scaling in such a way as shown in figure 2.

10 schematically represents an embodiment of a user interface according to the invention for selecting data objects on a display. Data objects D5-D10 may be, for example, menu options or selection buttons on a page. In the initial position of the device 2, shown by the dashed line, the data object D5 shown in the figure is shown by hatching. By tilting the device in the direction of arrow T5, the selection is shifted in the direction of tilt, as shown by arrow M3, and thus shifted to the data object D9. Having selected the desired data object, it can be fixed, for example, by means of a key on the device or by backward movement of the device. For simplicity, the tilt of the device in only one direction and the selection of data objects located one below the other are presented here, however, of course, you can move between the selected objects in the transverse direction in the same way.

FIG. 11 is another example of an embodiment described above with reference to FIG. 3. D11 widescreen data consists of objects that are described by letters arranged in matrix form. By moving the device in the manner shown by the arrows N, E, S and W, the user moves as if he were moving over the data and viewing them through the window formed by the display 2 '. Once the desired data is found, in this case the letter q, they can be selected using, for example, the keys on the device. Another comparable function can also be implemented, in which case instead of moving the device in the transverse direction, it tilts in the direction in which it is desirable for the user to move when viewing data.

12 shows another embodiment of a user interface according to the invention for selecting an object on a display. On display 2 'of device 2 there are objects 01-09 from which a selection must be made. In the initial state in the drawing on the left, the cursor is located in the upper right corner of the display. In this embodiment, cursor C seems to be locked in place in real space. If it is desirable for the user to select an object, for example 07, then the device moves laterally, as if moving the image under the cursor, to move the desired object 07 to the position under cursor C, as shown in the drawing to the right. The device has keys 21, 22 and 23, the action of which corresponds to the action of the mouse buttons. In fact, the embodiments of FIGS. 10-12 are solutions that, in portable devices, replace functions similar to mouse functions used in conventional computers and necessary to move the cursor or select an object on the display.

On Fig presents a variant in which data objects, pages, maps, etc. C1, ..., Ci, ..., Cn are arranged in a stack or one above the other on the display T of the device, like, for example, pages in a real book. Data objects can be, for example, maps representing products, or web pages. Objects are viewed in such a way as indicated by the arrow M4 forward and backward, by tilting the device, respectively, either forward in the direction shown by arrow T6, or backward in the direction shown by arrow T7. Using the keys 24-26, objects can be selected and files, programs, etc. related to them can be opened. Various modes can be provided in the user interface of the device, such as viewing, scrolling, selecting, scaling, as described above, for example, key 24 can be used to select a mode, and other keys 25, 26, 27 can act like buttons of a normal mouse .

On Fig presents an application of the user interface corresponding to the invention, on which data objects are arranged in radial stacks S11-S18, extending outward from the circle, in the center of which are the device 2 and the user. The first page of each stack, such as P12 or P16, defines the data behind it. The front pages can be viewed by rotating the device in the transverse direction, for example, along the arrow T8, so as to ensure the choice of the direction A12 and the front page P12. The stack of data objects behind this page can be viewed either by moving or by tilting device 2 back and forth in the direction of arrow F1. Thus, for example, a user interface for electronic commerce can be configured in order to facilitate the search for points of sale and orientation in the virtual store.

Fig. 15 shows a further development of the embodiment described above with reference to Fig. 5. The circles R1 and R2 now represent image data forming a panoramic image or a 3D image, covering, in principle, the entire sphere of perception space. Image data may also be image data obtained from a video camera connected to the device. The three-dimensional image data D12, describing, for example, an object, can be transferred to this image data and set to a desired location and in a desired orientation in a “real environment”. In response to the movement of the device 2 in the manner described with reference to FIG. 5, for example, as shown by arrow T9, both the background data and the 3D data D12 ', D12 "describing this object are changed in accordance with various positions devices, respectively, viewing directions A1 and A2, so this object can be viewed from different directions in the “real environment.” This can be used, for example, when searching for a 3D model of a sofa in a web store and when placing it in a desired position in the data images representing poku house patel, and it can be viewed from various angles to see how the sofa looks in the real environment surrounding it.

The solutions corresponding to the invention can be implemented by equipping a portable device, for example, with a multi-axis accelerometer and corresponding circuits and programs that interact with the device’s operating system and, possibly, application programs for processing and interpreting measurement results, so that data changes on the display are made according to movements measured by an accelerometer. For example, the implementation with the measurement of accelerations is based on the use of technical solutions known per se, and a specialist in the field of technology, based on instructions and specifications, is able to implement the user interface corresponding to the invention with reasonable effort.

In small-sized data processing devices, such as handheld computers, using technology known from the prior art, it is impossible to implement operating systems and applications with performance and applicability that would be close to the level achieved in larger data processing devices such as desktop or laptop computers. It is also impossible to incorporate traditional memory devices into portable devices such as a hard disk, floppy disk or CD-ROM drives (ROM on CD-ROMs). The solution to these problems may consist in creating a device consisting of two parts, and the portable part includes, in addition to the display, only part of the necessary circuits and programs. Most of the circuits and programs are located in another portable part that is available to the user, while this part and said portable part are connected by a wireless communication line. A wireless communication line can operate at infrared frequencies or at radio frequencies. In a device of this type, it is also easy to implement a system that detects the movement or position of the portable part relative to the portable part. The method of operation of the user interface corresponding to the invention can be implemented through this system.

Below, with reference to FIG. 16, another embodiment of the invention based on the detection of accelerations is described with reference to FIG. 16. The device includes a multi-axis accelerometer and the necessary circuits and programs for measurements. The device turns on, monitors the accelerations, and at step 11 detects that the acceleration that exceeds a certain threshold is probably the initial stage of the movement defined in the way the user interface works. In response to this, timers TD1 and TD2 are started at step 12. Timer TD1 sets a very short experimentally determined time interval of the order of milliseconds from detecting the start of movement, during which the actual direction and magnitude of the acceleration determines the movement that the user tells the device. Timer TD2 sets a longer time interval, after which the data change on the display stops if there are no other reasons for stopping. At the time TD1 expires, in step 13 the prevailing acceleration vector is measured. At step 14, it is analyzed whether the resulting vector is a specific vector, i.e. Does it correspond to any movement defined in the way the user interface works. If not, the procedure goes back to the beginning of 10 and step 11 to control the accelerations of the device. On the other hand, if this vector is a specific vector, then the data on the display changes at step 15, for example, it is viewed, scaled, etc. in accordance with a specific algorithm. As described above, this may be associated, for example, determining the speed of viewing data on the display based on the magnitude of the acceleration vector. At step 16, it is also monitored whether the TD2 time interval has expired. If this time interval has elapsed, the procedure proceeds to step 19, where the change in the data on the display stops, and the procedure ends at step 20. If the TD2 time has not expired, then at step 17 it is checked whether any opposite vector is detected in relation to the previously received specific vector or some other stop signal, for example, pressing the display. If the opposite vector or stop signal is detected in step 18, then in step 19 the data change on the display stops and the procedure ends. If the opposite vector or other stop signals are not detected in step 18, then the procedure goes back to control the above points.

The method of operation of the user interface corresponding to the invention can be implemented in other ways than using acceleration measurements. Any method can be applied by which changes in the location and position of the device can be measured. A solution that can be implemented provides a method in which the location and orientation of the device are detected by transmitting a pulsed constant magnetic field and measuring it with a detector in the device. Using this method, you can completely determine both the location and orientation of the device in three-dimensional space.

A possible solution is also the method on which the so-called optical mouse is based and in which the movement and speed of movement are detected by observing and analyzing any surface nearby, for example, some surface on the user. By combining this method with distance measurements, which is easy to implement, and by analyzing changes in distance and relative movement, you can detect changes in the location and orientation of the device.

The invention, of course, can be implemented in various ways. It can be provided, for example, that in order to detect (detect) a certain movement in a more reliable way, acceleration is measured several times during a short time window, and a certain movement is detected if any of the measured vectors satisfies the requirements established for such detection. Naturally, a more complex displacement analysis based on sequentially measured acceleration vectors can be applied. The movement of the device can also be detected using such a means, in which, for example, the distances between transmitters and receivers located, on the one hand, on the device, and on the other hand, on the user are measured.

The user interface corresponding to the invention can be implemented in accordance with the use and necessary features of portable devices as a suitable combination of the options presented above, when the user interface includes various operating modes in which different changes in the data on the display can correspond to the same movement of the device. As noted above, in some of the options may be provided, for example, a key located on the device, designed to select various operating modes.

The invention allows changes within the scope defined by the claims.

Claims (10)

1. The way the user interface of a portable device, such as a miniature or PDA, mobile phone, communicator or other similar means, in which the data on the display (2 ') of the device (2) changes in response to certain movements of the device in the three-dimensional space of its use, moreover, said movement of the device and corresponding changes in the data on the display are determined in such a way that the movement and the corresponding change in the data on the display have a causal effect ratio, which is similar to the causal relationship of the three-dimensional space of perception and user action (1), characterized in that the said method of operation includes an operating mode in which, in response to the inclination of the portable device, the data on the display scrolls in the direction in which portable device. 2. The method according to claim 1, characterized in that the operation mode further includes one or more of the following features: in response to the rotation (T8) of the device, a stack of data (S12) is selected from the data virtually arranged in stacks (S11-S18) according to the circle surrounding the user of the device (2), while the stack (S12) is viewed by tilting (F1) the device in the direction of the stack (S12) and away from it, respectively; in response to moving (Z) of the device (2) forward or backward perpendicular to the display (2 ′), the data (6a) on the display respectively scales with increasing or decreasing; in response to the movement (N, E, S, W) of the device (2) essentially in the direction of its display plane (2 ′), data (q) is selected from the data (D11) that are larger than the display, and in response to placing the cursor (C) on the display (2 ') and moving (N, E, S, W) of the device (2) essentially in the direction of its display plane (2'), the cursor (C) moves opposite to the movement to select data (01-09) on the display, similar to moving data on the display in real space under the cursor, preserving its position in real space. 3. The method according to claim 1, characterized in that the method of operation further includes an operation mode in which, in response to the swing (P) of the device (2), like turning the pages, the data (8a, 8b, 8c) on the display ( 2 ') are also changed in a manner corresponding to page turning. 4. The method according to claim 1, characterized in that the method of operation further includes an operation mode in which, in response to the tilt (T2) of the device (2), data (D2) moves on the display (2 ') from the tilt direction to the cursor (FROM). 5. The method according to claim 1, characterized in that said method of operation includes an operation mode in which, in response to the tilt (T6, T7) of the device (2), data (SI, C1, ..., Ci, .. ., Cn) on the display (2 ′) are viewed (M4) in the direction corresponding to the direction of inclination. 6. The method according to claim 1, characterized in that the method of operation further includes an operation mode in which, in response to the tilt (T3) of the device (2) from the user, the data (D3) on the display (2 ′) is increased (Z2) and / or in response to the tilt (T4) of the device (2) towards the user, the data (D4) on the display (2 ') is reduced (Z3). 7. The method according to claim 1, characterized in that the method of operation further includes an operation mode in which the data displayed on the display are a desired part of the panoramic image (9a), and in response to the rotation of the device (2), the displayed data (9b) are changed so that the change in the direction of observation and the displayed part of the panoramic image (9a) correspond to a rotation. 8. The method according to claim 1, characterized in that the operation method further includes an operation mode in which the data displayed on the display is a desired part of the panoramic image or three-dimensional image data (R1, R2), and three-dimensional data (D12) describing the object is placed on the image data, in response to the rotation of the device (2), both the panoramic image or three-dimensional image data (R1, R2) and the three-dimensional image data (D12 ', D12' ') are changed on the display (2') so as to match the direction of observation Ia (A1, A2), a corresponding orientation of the device (2). 9. The method according to claim 1, characterized in that in response to a stop or a significant slowdown (18) of movement, the change in data on the display is stopped. 10. The method according to claim 1, characterized in that the change in the data on the display is stopped by issuing a signal to the device in a different way than by moving the device.

Images