EP1535142A1 - Guidance method and device - Google Patents
Guidance method and deviceInfo
- Publication number
- EP1535142A1 EP1535142A1 EP03790973A EP03790973A EP1535142A1 EP 1535142 A1 EP1535142 A1 EP 1535142A1 EP 03790973 A EP03790973 A EP 03790973A EP 03790973 A EP03790973 A EP 03790973A EP 1535142 A1 EP1535142 A1 EP 1535142A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- scrolling
- text
- display
- motion
- text lines
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
- G06F1/1694—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
- G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
- G06F2200/163—Indexing scheme relating to constructional details of the computer
- G06F2200/1637—Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
Definitions
- the present invention generally relates to the display of electronic equipment such as mobile phones and more particularly to the browsing and reading of a large document on a display screen.
- scroll bars permitting movements of the text either horizontally or vertically do not have jitter. That is, scroll bars can be used to permit the user to change the portion of the view on the display screen hori- zontally by sliding the horizontal scroll bar and vertically by sliding the vertical scroll bar.
- a drawback with such methods is that scrolling is always either horizontal or vertical, i.e. no other direction is possible.
- An objective of the present invention is to provide a guidance method for scrolling the display on a screen of a hand-held device having motion- controlled scrolling and a hand-held device performing the method.
- the view shown on the display is a portion of larger content comprising text lines that are too long to be displayed entirely on the small display.
- the guidance method should be adapted to follow accurately the reading direction of the text lines from the beginning of the text lines to the end of the text lines.
- the reading direction should remain as uniform as possible during the entire reading event, i.e. unwanted effects such as jitter or bouncing of the text lines should not arise during the reading event.
- a typical situation may be when a user is browsing a large full-size image or a document by scrolling the view shown on a mobile phone's display using motion-control scrolling.
- the guidance method discovers when the user stops browsing and starts to read text lines. Then the need for guidance is examined. If the result of examination indicates that the user is reading the text lines, the scrolling direction is guided to follow the reading direction of the text lines. The guidance is invisible to the user.
- the guidance method comprises three main steps: 1) the direction of the text lines shown on the display screen is periodically examined; 2) the movements of the mobile phone are periodically examined and, based on the movements, it is concluded whether the user is reading the text lines; 3) if the result of examination in step 2 indicates that the user is reading the text lines, the scrolling direction is reinforced in the direction of the text line so long as the difference between the current scrolling direction and the text line direction is less than a predetermined threshold value.
- FIG. 1a-b illustrates a page of the document to be read on a display of a small portable device
- FIG. 2 is a flowchart illustrating definitions of the direction of text lines on the display
- FIG. 3 is a flowchart illustrating the method according to the invention
- FIG. 4 illustrates the inventive idea with the help of a full-size image to be read on a display of a small portable device
- FIG. 5a-b show an example of the determination of coordinates on the full- size image
- FIG. 6a-b demonstrate how filtering is carried out.
- the guidance method comprises three main steps: 1) the direction of the text lines shown on the display screen is periodically examined; 2) the movements of the mobile phone are periodically examined and, based on the movements, it is concluded whether the user is reading the text lines; 3) if the result of examination in step 2 indicates that the user is reading the text lines, the scrolling direction is reinforced in the direction of the text line so long as the difference between the current scrolling direction and the text line direction is less than a predetermined threshold value.
- FIG. 2 shows as a flowchart an example of the implementation of examination of the direction of text lines.
- the first task is to analyze roughly whether there is text in the current portion on display. If no text is detected, free scrolling of the data is allowed at stage 21 , and the control is returned to stage 20.
- One alternative is that the portion of the data that is seen on the display screen is first analyzed by defining the major components, i.e. whether the type of component represents the text or non-text.
- Some character recognition method such as an optical character recognition method OCR, can be used.
- OCR optical character recognition method
- information about the type and size of the document, and the type and size of the font may already be included in the document, in which case no character recognition is needed.
- FIG. 1a most of the page is covered with text.
- the direction of text lines is defined to be horizontal. However, if the column 13 had been shown on the display, the text lines would have been defined to be vertical.
- direction definition of the text lines can be created on the basis of the middle point of the current portion of view displayed. This method will be considered later in greater detail.
- Another alternative utilizes image analysis and character recognition. The direction is defined on the basis of features of the analyzed data.
- a flowchart in FIG. 3 is used to demonstrate how the need for guidance is examined and carried out while one is browsing a large document on the display screen of a terminal by scrolling the display.
- Coordinates of the current partial view of the full-size image are read at the fixed reading point (xi, yi) of the display.
- the fixed reading point is the middle point of the display.
- the coordinates of the current view at the middle point of the display are then saved at certain time instances e.g. every second, at stage 30.
- a set of motion vectors is obtained.
- the motion distribution as a time function is calculated at stage 31 by using the motion vector set.
- the need for guidance is examined. If the distribution is narrow, it means that movements have been found, which do not deflect much from the main movement direction or scrolling direction. In other words, if the scrolling direction is in the direction of the x-axis, the coordinates of the page, which have been read at the middle point of the display, seem to follow the general direction of the x-axis (the text line), i.e. the scrolling direction is not exactly rectilinear. If such movement as described above is not detected or the distribution is relatively broad, free scrolling is still allowed, at stage 35. Otherwise, the next stage 33 is to examine more accurately whether there is a need to guide the reading direction. In this case the distribution is compared with the saved information (see FIG.
- stage 34 it is analyzed whether the result of the comparison indicates, in a given time, successive movements which are roughly in the same direction. If the result does not indicate such movements, free scrolling of the data is still allowed at stage 35. Otherwise, the data is filtered in such a way that the direction of the text lines are taken into account at stage 36. Regardless of the result of the comparison, the steps above are repeated from stage 30.
- FIG. 4 demonstrates how the defining and filtering are carried out mathematically.
- the assumption in this example is that the direction of the text lines on the display is defined to be horizontal by the character recognition method.
- a vector 42 represents that direction.
- the additional assumption is that coordinates of the partial view 40 of the full-size image are read at the middle point 43 of the display. Based on the successive coordinates, a set of motion vectors are obtained, wherein if the directions of said motion vectors are roughly in the direction of the text line, this indicates that the user is reading the text lines.
- the curve 41 represents the set of motion vectors.
- Coordinates of the partial view of the full-size image are controlled by a cer- tain time window.
- the position of the middle point 43 on the full-size image 10 varies as time function.
- FS sample rate
- Each sample consists of the coordinates (xi, yi) of a point Pi on the curve at a given time.
- a current coordinate pair (xi,yi) is saved every second within a certain time interval, such as 10 s.
- These samples formulate a temporal matrix T, where the first row of the matrix represents the x-coordinates and the second row the y-coordinates, so that each column presents one coordinate pair (xi,yi).
- the parameter D represents the direction (angle) of the text line under examination.
- the definition value for D is 0 radians when the text line is in the vertical direction and ⁇ /2 radians when the text line is in the horizontal direction.
- parameter D can have any angle value between 0 - n2 ⁇ radians.
- the current direction (angle) between each consecutive point in temporal matrix T is calculated. For example, if at time t1 the coordinates at point P1 are (x1 , y1) and at time t2 at point P2 (x2, y1), the vector between the two points P1 and P2 defines the direction (angle) at time t1 , where ⁇ t is (t2-t1 , t2 > t1). In general the result of calculation is N-1 directions or vectors when the number of samples is N.
- the calculated N-1 directions formulate a vector U.
- the result is nine vectors, each of which has a direction (angle) between 0-n2 ⁇ radians. That is the result is the distribution of directions within 10 seconds.
- the shape of distribution of the directions is narrow, a lot of movements have been found in the same direction.
- the shape of distribution is broad, only some movements are in the same direction.
- the scroll direction is guided to follow accurately the reading direction of the text lines by filtering from the scrolling motion those components which differ from the direction of the text lines.
- FIG. 5a illustrates how the view shown on the display looks at time t1. It is seen that the middle point of the current view 500 is in point P1 at time t1. Within time (t1+ ⁇ t) - 11 the display on the screen has been moved or the middle point of the previous display has been moved on the full-size image from point P1 to point P2, when the current view 500 is now the view 501 as shown in FIG. 5b.
- FIG. 6a and 6b illustrate a situation similar to what was explained above with reference to FIG. 5a-5b, but here the filtering is in action.
- the vector a from the initial point P1 to the terminal point P2 represents the displacement of the full-size image within a given time.
- the vector b is a unit vector (i.e.
- 1) in the direction of the x-axis.
- the initial point of vector b is at point P1.
- FIG. 6b illustrates said determination.
- p is the product of the magnitudes of a and b and the cosine angle ⁇ between them.
- ⁇ is ⁇ /4.
- the calculation described above is repeated correspondingly for displacements at intervals as long as the filtering is activated.
- the time period is a sample period 1/FS.
- the above-described method can be used in a hand-held terminal having a motion controlled scrolling operation by adapting the terminal to: read coordinates of the scrolling display at a fixed reading point of the screen at predetermined time intervals, wherein a set of motion vectors is obtained; determine the scrolling direction based on the set of motion vectors; examine whether the scrolling direction is roughly the same as the direction of a text line; and if so, reinforce the displayed scrolling in the direction of the text line as long as the difference between the current scrolling direction and the text line direction is less than a predetermined threshold value.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FI20021560 | 2002-09-02 | ||
FI20021560A FI115255B (en) | 2002-09-02 | 2002-09-02 | Monitor control method for a mobile terminal and a mobile terminal |
PCT/FI2003/000598 WO2004021166A1 (en) | 2002-09-02 | 2003-08-12 | Guidance method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1535142A1 true EP1535142A1 (en) | 2005-06-01 |
Family
ID=8564507
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03790973A Withdrawn EP1535142A1 (en) | 2002-09-02 | 2003-08-12 | Guidance method and device |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP1535142A1 (en) |
AU (1) | AU2003249135A1 (en) |
FI (1) | FI115255B (en) |
WO (1) | WO2004021166A1 (en) |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2412048A (en) * | 2004-03-09 | 2005-09-14 | Jitendra Jayantilal Ranpura | Viewing an image larger than the display device |
JP2006113801A (en) * | 2004-10-14 | 2006-04-27 | Canon Inc | Apparatus and method for displaying image processing result, and program |
US7509588B2 (en) | 2005-12-30 | 2009-03-24 | Apple Inc. | Portable electronic device with interface reconfiguration mode |
US10313505B2 (en) | 2006-09-06 | 2019-06-04 | Apple Inc. | Portable multifunction device, method, and graphical user interface for configuring and displaying widgets |
US7956849B2 (en) | 2006-09-06 | 2011-06-07 | Apple Inc. | Video manager for portable multifunction device |
US7864163B2 (en) | 2006-09-06 | 2011-01-04 | Apple Inc. | Portable electronic device, method, and graphical user interface for displaying structured electronic documents |
US8842074B2 (en) | 2006-09-06 | 2014-09-23 | Apple Inc. | Portable electronic device performing similar operations for different gestures |
US8214768B2 (en) | 2007-01-05 | 2012-07-03 | Apple Inc. | Method, system, and graphical user interface for viewing multiple application windows |
US8519964B2 (en) | 2007-01-07 | 2013-08-27 | Apple Inc. | Portable multifunction device, method, and graphical user interface supporting user navigations of graphical objects on a touch screen display |
US9933937B2 (en) | 2007-06-20 | 2018-04-03 | Apple Inc. | Portable multifunction device, method, and graphical user interface for playing online videos |
US9772751B2 (en) | 2007-06-29 | 2017-09-26 | Apple Inc. | Using gestures to slide between user interfaces |
US11126321B2 (en) | 2007-09-04 | 2021-09-21 | Apple Inc. | Application menu user interface |
US8619038B2 (en) | 2007-09-04 | 2013-12-31 | Apple Inc. | Editing interface |
US9619143B2 (en) | 2008-01-06 | 2017-04-11 | Apple Inc. | Device, method, and graphical user interface for viewing application launch icons |
US8736561B2 (en) | 2010-01-06 | 2014-05-27 | Apple Inc. | Device, method, and graphical user interface with content display modes and display rotation heuristics |
US8438504B2 (en) | 2010-01-06 | 2013-05-07 | Apple Inc. | Device, method, and graphical user interface for navigating through multiple viewing areas |
US9383818B2 (en) | 2013-12-27 | 2016-07-05 | Google Technology Holdings LLC | Method and system for tilt-based actuation |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0764754A (en) * | 1993-08-24 | 1995-03-10 | Hitachi Ltd | Compact information processor |
US6624824B1 (en) * | 1996-04-30 | 2003-09-23 | Sun Microsystems, Inc. | Tilt-scrolling on the sunpad |
GB9620464D0 (en) * | 1996-10-01 | 1996-11-20 | Philips Electronics Nv | Hand held image display device |
JP3338777B2 (en) * | 1998-04-22 | 2002-10-28 | 日本電気株式会社 | Mobile terminal and screen display method thereof |
-
2002
- 2002-09-02 FI FI20021560A patent/FI115255B/en active IP Right Grant
-
2003
- 2003-08-12 WO PCT/FI2003/000598 patent/WO2004021166A1/en not_active Application Discontinuation
- 2003-08-12 EP EP03790973A patent/EP1535142A1/en not_active Withdrawn
- 2003-08-12 AU AU2003249135A patent/AU2003249135A1/en not_active Abandoned
Non-Patent Citations (1)
Title |
---|
See references of WO2004021166A1 * |
Also Published As
Publication number | Publication date |
---|---|
FI20021560A (en) | 2004-03-03 |
FI20021560A0 (en) | 2002-09-02 |
FI115255B (en) | 2005-03-31 |
WO2004021166A1 (en) | 2004-03-11 |
AU2003249135A1 (en) | 2004-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2004021166A1 (en) | Guidance method and device | |
US7129926B2 (en) | Navigation tool | |
EP1507192B1 (en) | Detection of a dwell gesture by examining parameters associated with pen motion | |
EP1328919B2 (en) | Pointer tool | |
JP4943543B2 (en) | MAP DISPLAY DEVICE, MAP DISPLAY METHOD, MAP DISPLAY PROGRAM, AND RECORDING MEDIUM | |
JP4043447B2 (en) | Written motion image classification recognition system and recognition method thereof | |
US20070070037A1 (en) | Graphic signal display apparatus and method for hand-held terminal | |
US8531410B2 (en) | Finger occlusion avoidance on touch display devices | |
US20070018966A1 (en) | Predicted object location | |
US20090002324A1 (en) | Method, Apparatus and Computer Program Product for Providing a Scrolling Mechanism for Touch Screen Devices | |
EP1887776A1 (en) | Portable terminal and user interface control method thereof based on pattern recognition and analysis of image captured by camera | |
JP5604279B2 (en) | Gesture recognition apparatus, method, program, and computer-readable medium storing the program | |
US20080030477A1 (en) | Display Motion Multiplier | |
US20040208394A1 (en) | Image display device and method for preventing image Blurring | |
JPH0772970A (en) | Apparatus and method for selection of information | |
KR100777107B1 (en) | apparatus and method for handwriting recognition using acceleration sensor | |
US20110043453A1 (en) | Finger occlusion avoidance on touch display devices | |
WO2008054185A1 (en) | Method of moving/enlarging/reducing a virtual screen by movement of display device and hand helded information equipment using the same | |
EP1993025A2 (en) | Delay judgment systems and methods | |
CN103106388A (en) | Method and system of image recognition | |
US8355599B2 (en) | Methods and devices for detecting changes in background of images using multiple binary images thereof and hough transformation | |
EP3125089B1 (en) | Terminal device, display control method, and program | |
EP3379451A1 (en) | Information processing device | |
JP2010258603A (en) | Communication terminal and method of adjusting display size | |
JP5080409B2 (en) | Information terminal equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20050324 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
DAX | Request for extension of the european patent (deleted) | ||
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: TIPOT S.A.R.L. |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: MYORIGO S.A.R.L. |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: HANNULA, MANNE Inventor name: METSAEVAINIO, JUKKA-PEKKA |
|
17Q | First examination report despatched |
Effective date: 20071017 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20100302 |