FI115255B - Monitor control method for a mobile terminal and a mobile terminal - Google Patents

Description

115255

A method for controlling the display of a portable terminal and a portable terminal

FIELD OF THE INVENTION The invention relates generally to the display of an electronic device such as a mobile station, and more particularly to browsing and reading a large document on a display screen.

Background Art 10 Nowadays, portable PDAs such as mobile devices and personal digital assistants (PDAs) are becoming smaller and smaller. As the size of the portable device decreases, so does the screen size of the device. On the other hand, there is a growing need for subscribers to offer services that include text and images. The screen size of the screen is usually smaller than the actual doc-15, so only part of the document is visible at a time. It is clear that reading a large document from a small screen is relatively difficult.

Different types of mechanisms have been developed to facilitate reading from a small screen. Some of these are briefly discussed below.

20 For some small handheld devices that can be used while carrying, the screen view scrolling is based on • »location (at least one of the x, y, or z coordinates) and / or device position / ·; * (at least one tilt, tilt, rotation). In such a motion control environment, the portion displayed on the display of the handheld device always depends on the movements of the hand. The inaccuracy of the hand causes unwanted effects such as shaking and jumping, which makes reading more difficult. Similar undesirable effects may also occur when a hand-held device is used in a moving vehicle.

To avoid the above disadvantages, a method has been developed in which the direction of data scrolling can be locked. However, one disadvantage of this method is that it allows data to be scrolled in one direction only, i.e., it does not take into account the direction of the text lines of the data.

•: In methods using scroll bars that allow text to move in either horizontal or vertical direction, there is no vibration. . ···. In other words, the scroll bars can be used to allow a user to change a portion of the screen view horizontally by flagging 2,115,255 horizontal scroll bars and vertically dissolving the vertical scroll bar. However, the disadvantage of such methods is that the scrolling is always either horizontal or vertical, i.e. no other direction is possible.

5 Currently, there is no method available to reduce interference from text lines when reading a document on the screen of a portable motion control assistive device due to jumping motion.

BRIEF SUMMARY OF THE INVENTION

It is an object of the invention to provide a control method for scrolling the screen on a display screen of a handheld, motion-controlled device and a handheld device for implementing the method. The on-screen view is part of larger content that includes lines of text where the lines of text are too long to be displayed on the small screen as a whole.

The control method should be adapted to closely follow the reading direction of the text lines from the beginning of the text lines to the end of the text lines. The reading direction should remain as even as possible throughout the reading process. In other words, unwanted phenomena such as text line colors should not occur during a read-only event. . running and jumping.

• »

A typical situation could be one where the user scrolls a large full size image or document by scrolling the motion control scroll; screen of a mobile station.

'25 The control method detects when the user stops browsing and': starts reading the lines of text. The need for guidance is then examined. If the result of the examination: indicates that the user is reading lines of text, the scrolling direction is directed to follow the reading direction of the lines of text. Control not visible to user, i ': Target is achieved as described in independent requirements.

; : 30 The control method consists of three main scrolls: 1) periodically examining the direction of the lines of text displayed on the screen; 2) periodically examining the movements of the mobile station and determining whether the user reads based on the movements

»* I» I

lines of text; 3) If the result of the examination in section 2 shows that the user reads: T: lines of text, the scrolling direction is forced to parallel to the line of text as long as · ···. The difference between the 35 most recent scroll direction and the text line direction is less than a predefined threshold.

3,115,255

List of figures

In the following, the invention will be described in more detail with reference to the accompanying drawings in which: FIG. 1 illustrates a document readable on a small handheld device, FIG. 2 is a flow chart illustrating the direction of text lines on the screen; FIG. 3 is a flow chart illustrating a method the idea of reading a full size image on the screen of a small portable device, Figures 5a-b illustrate an example of determining the coordinates of a full size image, and Figures 6a-b illustrate how filtering is performed.

DETAILED DESCRIPTION OF THE INVENTION

The invention may be implemented in a terminal having a small display, at least one sensor mounted behind the display, such as a magnetic pulse sensor, an accelerometer or a gyroscope, and a pattern recognition method or alternative. or another method for detecting the direction of lines of text.

. The basic idea and embodiments of the invention will be described using the prefix as a mobile station. The same basic idea is, of course, feasible for any type of terminal.

* * ': Figure 1 illustrates a typical situation in which a user scrolls a large \. · * Full-size image on a screen of 10 mobile stations. Here, a full-size image refers to an entire document or page of which only a portion can be displayed on the actual display of the terminal.

; The actual mobile display is depicted on the document page as a dotted square 16 to illustrate the dimensions of the display relative to the document. Thus, the user reads a document whose actual size is larger than the physical size of the display. The document page is subdivided into a plurality of s::: columns comprising text areas 11 to 13 and non-text areas such as:: 35 image areas 14 to 15. The text lines in column 13 are in different directions than in the other two columns 11 and 12.

4, 115255

Figure 1b is an enlarged view of the circular area of Figure 1a. As you can see, only part of the data is visible on the screen at a time, meaning that the document line does not fit on the screen from start to finish. Scrolling directions are indicated by double-headed arrows. Other scrolling directions are also possible, for example, an angular direction, e.g.

Assume that motion-controlled scrolling is enabled. To browse the document, the data is displayed relative to the position or location of the mobile station. In other words, data is scrolled vertically when the mobile station is tilted vertically. Correspondingly, the data scrolls horizontally when the mobile station is tilted horizontally.

When an interesting piece of text appears on the screen and the user wants to investigate further, that is, to read the entire document or paragraph, the portion of the on-screen view is moved to the beginning of the document or desired paragraph by tilting the mobile station appropriately. As a precursor, the movable cursor is used to indicate an interesting spot on the screen surface by tilting the mobile station. The cursor moves horizontally as the mobile station is tilted toward the vertical axis of the display. Correspondingly, the cursor moves vertically as the mobile station is tilted toward the horizontal axis of the display.

20 The view of the screen therefore depends on the movements of the hand. However, as stated above, the hand movements are inaccurate. Without precise control, unwanted vibrations occur, which makes reading uncomfortable and slow. This drawback is eliminated by the control method described in detail; next.

; : 25 The control method consists of three main scrolls: 1) on the screen ·. * ·: The direction of the displayed lines of text is examined periodically 2) the movements of the mobile station are examined periodically and based on the movements it is decided whether the user reads text lines; 3) if the result of the examination in step 2 shows that the user reads: lines of text, the scrolling direction is forced to parallel to the text line as long as: ': the difference between the last 30 scrolling directions and the text lines is less than a predetermined threshold;

'· · · * Fig. 2 is a flow chart illustrating an example of an implementation for studying the direction of text.

Here, it is assumed that the user scrolls the document on the display screen of the mobile station 35 as described in connection with Figures 1a and 1b.

5, 115255

In step 20, the first task is to roughly analyze whether there is text in the currently visible portion of the screen. If no text is detected, step 21 permits free scrolling of the data and control returns to step 20.

5 Analysis can be done in a number of ways. One alternative is to analyze the portion of the data displayed on the screen by defining the main components, i.e., analyzing whether the component type represents text or non-text. Some pattern recognition method may be used, for example, Optical Cha-10 racter Recognition (OCR). In some cases, the document type and size information, as well as the font and size, may be pre-installed in the document, which eliminates the need for pattern recognition.

The next step 22 defines the orientation of the lines of text. In Figure 1a, most of the page is text. When analyzing this page using the hah-15 montage detection method, the text lines are oriented horizontally.

However, if column 13 had been displayed, the lines of text would have been set to vertical.

In one embodiment, the orientation of the lines of text can be determined based on the center of the current screen view. This meth-20 method will be discussed in detail later.

··, Another option utilizes image analysis and pattern recognition. The direction is determined by the features of the data being analyzed.

• · »; The results of the orientation of the lines of text are stored in step · · · <»25 in step 23. The information also contains the current time information. The above * · »: analysis and assay is carried out at periodic intervals.

: The flowchart of Figure 3 is used to illustrate how the need for control is investigated and executed while a large document is being browsed; 1 ': Scrolls the terminal screen on the screen.

: "30 The most recent coordinates of a partial view of a full-size image are read from the fixed reading point (xi, yi) of the display. In this example, the fixed reading '···', is the center of the display.

• »♦ · ·

The coordinates of the center of the most recent screen view are recorded at certain times of the time, for example, every second, at step 30. The set of coordinates provides a set of motion vectors. In step 31, the motion distribution of I t is calculated as a function of time using a set of motion vectors.

6, 115255

Paragraph 32 examines the need for guidance. If the distribution is narrow, it means that movements which do not deviate significantly from the main direction of motion, i.e. the direction of scrolling, have been detected. In other words, if the scrolling direction is in the x-axis direction, the page coordinates read from the center of the screen show the direction of the next x-axis (lines of text), so the scrolling direction is not exactly linear. If no movement such as the one described above is detected, i.e., the distribution is relatively wide, free scrolling is still allowed in step 35. Otherwise, in the next step 33, the need for reading direction control is investigated further. In this case, the distribution is compared with the stored information about the text line direction (see Figure 2, step 23). Paragraph 34 analyzes whether the comparison results in successive motions roughly in the same direction at a given time. If no such motion is detected in the result, free scrolling of the data is still allowed in step 35. Otherwise, the data is filtered so that the direction of the text lines is taken into account in step 36.

Regardless of the result of the comparison, the steps described above are repeated from step 30.

Figure 4 illustrates how the determination and filtering are performed mathematically. In this example, it is assumed that the text lines on the screen are oriented horizontally by the pattern recognition method. Vector 42 represents this direction. A further assumption is that the full size ·· * coordinates of the partial view 40 of that image are read from the center of the screen 43.

'' The consecutive coordinates give a set of motion vectors. If the directions of said motion vectors are roughly parallel to the lines of text; the user is likely to read the lines of text. The curve 41 represents the set of motion vectors' 25.

'· *: The similarity between the directions of vector 42 and curve 41 is investigated.

One way to study similarity is described below.

The coordinates of the sub-view of a full-size image are controlled in a specific time window. In other words, as the document is scrolled, the center point of the full-size image 10 changes as a function of time.

* · \ The calculation unit draws samples from the curve 41 using a sampling rate FS (e.g. FS = 1 s). Each sample comprises the coordinates (xi, yi) of the Pi point of the curve at a given time. For example, the last pair of coordinates (xi, yi) 35 is stored every second at certain time intervals, for example 10 s. These screens form a temporal matrix T, where the first row of the matrix represents the x-coordinates and the second row the y-coordinates such that each column represents one pair of coordinates (xi, yi).

Parameter D represents the direction (angle) of the line of text being studied. The value of D is set to 0 radians when the line of text is 5 in the vertical direction and π / 2 radians when the line of text is in the horizontal direction. Of course, the parameter D can obtain a value of any angle between 0 and η2π radians.

The last direction (angle) between each successive point of the temporal matrix T is calculated. For example, if at t1 the coordinates 10 of point P1 are (x1, y1) and at t2 the coordinates of point P2 are (x2, y2), the vector between these two points P1 and P2 defines the direction (angle) at t1 where At is (t2-t1). , t2> t1).

Generally, the result of the computation is N-1 directions, or vectors, when the number of samples is N. The computed N-1 directions make up the vector 15 U. Within this 10 seconds, nine vectors with a direction (angle) of 0-η2π radians, the result is a directional distribution of 10 seconds. When the directional distribution is narrow, much movement has occurred in the same direction. Similarly, when the directional distribution is wide in shape, there is little movement in the same direction.

The similarity between curve 41 and vector 42 is examined by comparison. · ", The vectors described above as described below.

• ·

First, it is compared whether the difference between the mean values of the vector U and the parameter D is less than the predetermined threshold parameter V, and whether the vector; The standard deviation of U is smaller than the predetermined parameter X. In the comparison f f * 25 the following equations Y are used: Abs (Mean (U) - D) <V (1) • »

Std (U) <X (2): * * ': If the results in both comparisons are smaller, the curve 41 30 and the vector 42 are parallel. Otherwise, the curve 41 and vector 42 are in different directions.

In the first case described above, that is, when the curve 41 and the vector 42 are parallel, the scrolling direction is controlled to closely follow the reading direction of the text lines by filtering out from the scrolling motion the components that differ from the text line direction.

* · · 8 115255

The effect of filtering is illustrated by curves 44 and 45 in Figure 4. Curve 44 illustrates page motion before filtering, and vector 45 represents page motion after filtering. The filter makes sure that the full-size image partial view of the movement takes place precisely in the direction of lines of text. As a Tu-5, it is easy for the user to read lines of text, since unwanted reading effects are prevented.

One way of performing the filtration is described with reference to Figures 5a-5b and 6a-6b.

Initially, it is assumed that filtration is not turned on in Figures 5a and 5b.

Fig. 5a illustrates how the graph on the screen shows at time t1. It is seen that the center 500 of that view is at point P1 at time t1. During (t1 + At) - t1, the screen view has moved, i.e. the center of the previous screen has moved with a full size image from P1 to P2, whereby current view 500 is now view 501, as in Figure 5b.

Figures 6a and 6b illustrate a situation as described above in connection with Figures 5a-5b, assuming that filtration is on.

In Fig. 6a, the vector a, which extends from the starting point P1 to the end point P2, represents the transition of the full-size image at a given time. The vector b is a unit vector along the x-axis (i.e. | b | = 1). The starting point of vector b is 20 at P1.

In this case, only the x component of the a is parallel to the x axis, since the lines of text are parallel to the x axis.

Figure 6b illustrates said determination. First the attribute- '' · ·; Let's calculate the projection of a by b using the scalar product: 25 p = a · b. = | a || b | cos0 (3) •> *!,. where p is the product of the lengths of a and b and the cosine angle Θ is the angle between them. In this example, Θ is π / 4. Then vector x represents the x component of vector a, which is denoted as follows: ax = pb (4) 30 The origin of vector ax is at point P1 and the end point at point PS.

'···' The vector ax represents the displacement of the full-size image in the x direction during At.

'' The transitions are repeated at intervals as described above, as long as: '; ': When filtering is activated. The interval is a sampling interval of 1 / FS.

: * "; The method described above can be used in a handheld terminal having a motion control scrolling function such that the terminal is adapted to: read the coordinates of the scrolling display from a fixed point on the display screen at predetermined time intervals to obtain a plurality of motion vectors; determining a scroll direction based on a set of motion vectors; examining whether the scroll direction is roughly the same as the text line direction; and if so, the screen is forced to scroll parallel to the text line until the difference between the current scroll direction and the text line direction is less than a predetermined threshold.

The embodiment and embodiment of the present invention have been described above with a few examples. However, it is to be understood that the details of the above-described embodiments are not limiting of the invention and that numerous modifications and variations may be made by one skilled in the art without departing from the features of the invention. The embodiment described is to be considered as illustrative but not limiting. Therefore, the invention is limited only by the appended claims. Alternative embodiments as defined by the claims as well as corresponding embodiments are within the scope of the invention. For example, the control can be implemented in any scroll direction, also diagonally. Parameters such as sampling rate and number of samples as well as time slots depend on the application used. In some cases, it is also possible for the user to adjust some of the above parameters, whereby the control method behaves differently for slow and fast users.

• I>

• I

t 1 • · t

• 1 I

• · 1

«I

• «1

• · I

• f · f «lii ·

• I

• | • I · • »t I | • f »t • I <» 1 t

I I

»

»I

· »I> ·

Claims (6)

115255 A control method for scrolling data on a terminal display, wherein the scrolling direction depends on the tilt angle of the terminal, characterized in that the method comprises the steps of: reading the coordinates of the scrolling view from a fixed floating point of the display whether the scroll direction is roughly the same as the text line direction, and if so, the screen is scrolled to the text line direction, as long as the difference between the current scroll direction and the text line direction is less than a predetermined threshold. Method according to claim 1, characterized in that the method further comprises the steps of: periodically detecting whether text is displayed, determining the direction of the text lines in response to the observation and storing the result together with the observation time; . · * ·. A method according to claim 1, characterized in that the text line direction is determined using pattern recognition. * »* Method according to claim 1, characterized in that: that the direction of the text lines is determined by coordinates. ] 25 5. A hand-held terminal having a motion-controlled scroll function, characterized in that the terminal is adapted to: read the coordinates of the scrolling view from a fixed reading point of the display at predetermined intervals to form a plurality of Ini '': lectors: 1 : 30 to determine the scroll direction based on a set of motion vectors, to investigate whether the scroll direction is roughly the same as the text line direction '·'], and if so, to force screen scroll to text line direction as long as: ':': less than the current scroll direction and text line direction “': 35 as a predefined threshold. 115255 The hand-held terminal according to claim 5, characterized in that the terminal is further adapted to: periodically observe whether text is displayed, determine the direction of the lines of text in response to the observation and store the result along with the observation time, calculate motion motion distribution over time, and compare motion with. * I *> »I I I» I · tn • I I »| I * * in> »t · I I t • · •> I ·» I I * $ * Iti I · I 4 I «I II t • t I • I t | t> I I · * * * »> t» I I I tilli I i I II * 1 III> 115255