KR20010051396A - Apparatus and methods for selecting a user interface option on a portable electronic device - Google Patents

Abstract

PURPOSE: A device and method for selecting user interface option on portable electronic equipment are provided to supply a portable electronic device which facilitates entry of user data and commands. CONSTITUTION: A portable electronic device(100) facilitates entry of user data and commands by detecting motion of the device(100). A display(112) is used to depict a plurality of user interface options(500,502). The user physically moves the device(100) to highlight and enter a desired character of data or a command. For example, a virtual keypad may be displayed, and when the device(100) is moved forward, backward, left, or right relative to the plane of the display(112) the next letter or number in that direction(or the opposite direction) may be highlighted. Then, by moving the device(100) downward the highlighted letter or number may be entered as if it were typed on a real, dedicated keypad.

Description

Apparatus and method for selecting user interface options on portable electronic devices {APPARATUS AND METHODS FOR SELECTING A USER INTERFACE OPTION ON A PORTABLE ELECTRONIC DEVICE}

BACKGROUND OF THE INVENTION Field of the Invention The present invention generally relates to methods and apparatus for selecting a user interface on a portable electronic device. In particular, the present invention relates to data through a graphical user interface associated with a device by detecting movement of the portable electronic device concurrently with a given graphical display. And to select menu options.

Advances in integrated circuit technology have allowed the size of portable electronic devices such as cellular telephones to be significantly reduced. These reduced size devices are often preferred to users because they are lighter and more portable. However, some components of a typical portable electronic device cannot be further reduced in size without significantly compromising the usefulness of the device. One such component is a keypad. A typical cellular telephone keypad includes dialing numbers 0-9 as well as several other function keys such as "clear", "send", "end" and the like. These keypads add to the cost, mechanical design complexity, size and weight of cellular telephones. The minimum size of the key is defined by the size of the human finger. In other words, if the keys are too small, many users will have difficulty dialing the phone.

While the proposed methods include removing the keypad using techniques such as voice recognition and touch screens, these techniques have significant limitations and drawbacks. Speech recognition methods may require extensive training and often do not work well, so the user may need to repeat some commands. This repetition can be time consuming and frustrating for the user. In addition, speech recognition algorithms require significant computational resources, adding to the cost of the electronic device and / or delaying the responsiveness of the recognition process. In addition, the voice recognition method does not guarantee privacy. For example, when a user dials a telephone number in a public place, the telephone number and / or name of the called party may be heard by a nearby person. Touch screen technology solves the problems of privacy, accuracy, and computational resources. However, touch screens are expensive and often degrade the visual clarity of the underlying display. The touch screen is also size limited likewise if the user is intended to use a finger for selection, or the touch screen requires the use of stylus, which is liable to be lost.

1 is a block diagram illustrating a portable electronic device.

2 is a flow diagram of a program that may be executed by the portable electronic device of FIG. 1 to select alphanumeric characters through a user interface.

3 is a line diagram of an exemplary user interface that may be executed by the portable electronic device of FIG. 1 to display alphanumeric options.

4 is a flow diagram of a program that may be executed by the portable electronic device of FIG. 1 to select a menu item via a user interface.

5 is a line diagram of an exemplary user interface that may be executed by the portable electronic device of FIG. 1 to display menu options.

6 is a flowchart of a program that may be executed by the portable electronic device of FIG. 1 to execute a software routine in response to the detection of a defined display movement.

<Explanation of symbols for main parts of drawing>

100: portable electronic device

101: power

102: controller

104: data memory

106: program memory

109: address / data bus

110: motion detector

111: A / D Converter

112: display

114: input / output circuit

116: antenna

These and other features and advantages of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments with reference to the accompanying drawings.

The portable electronic device displays a plurality of user interface options on the display and allows the user to select one or more of the options by physically moving the device in a particular direction. For example, a virtual keypad can be displayed and when the device is moved forward, backward, left or right relative to the plane of the display, the next character in that direction (or the opposite direction) can be highlighted. . Then, by moving the device downward (ie, perpendicular to the plane of the display), the highlighted characters can be entered as if they were typed on an actual dedicated keypad.

The portable electronic device 100 is shown in FIG. 1. In a preferred embodiment, portable electronic device 100 is a wireless communication device such as a handheld cellular telephone or pager. However, the portable electronic device 100 may be a multipurpose computing device such as a personal digital assistant or an application specific device such as an electronic book or an electronic map. The portable electronic device is connected to the power source 101. Preferably, the power source is a battery. However, those skilled in the art will readily appreciate that other power sources such as, for example, a DC converted output of an AC transformer or a solar panel can be used as the power source 101.

Controller 102 in portable electronic device 100 may include data memory 104, such as random access memory, program memory 106, which may be in the form of read-only memory (ROM), and microprocessor 108-both May be interconnected by an address / data bus 109. In one embodiment, program memory 106 electronically stores a computer program that executes all or part of the methods described below with respect to FIGS. 2, 4, 6. Preferably, the program is executed by the microprocessor 108. Program memory 106 may be loaded from a fixed memory device, such as a hard drive, or program memory 106 may be preloaded by firmware, as is well known to those skilled in the art. Some of the steps described in the following method may be performed manually or without using the portable electronic device 100.

Preferably, motion detector 110 and display 112 via A / D converter 111 are electronically connected to controller 102 via conventional input / output (I / O) circuit 114. The motion detector 110 is also mechanically coupled to the display 112 such that the movement of the motion detector 110 corresponds to the movement of the display 112.

The motion detector 110 preferably consists of two accelerometers, each of which detects a change in movement (ie, acceleration). The first accelerometer is disposed parallel to the face of the display 112 to detect movement of the device 100 parallel to the face of the display 112 (ie, along the x and y axes). The first accelerometer outputs two voltages approximately centered around the static DC voltage point, one for the x-axis and one for the y-axis. As device 100 is accelerated (or moved) in the positive or negative direction along the x-axis, y-axis, or a combination thereof, the output voltage increases or decreases from a static DC voltage point. For example, if the x-axis output voltage increases from a static DC voltage point as the device 100 moves, there is a movement in the X direction and the acceleration in the X direction is positive. If the x-axis output voltage decreases from the static DC voltage point as the device 100 moves, there is a movement in the X direction and the acceleration in the X direction is negative. If there is no movement of the device 100 in the X direction, the x-axis output voltage does not change and there is no acceleration in the X direction. This also applies to the y-axis output voltage. The second accelerometer is disposed perpendicular to the face of the display 112 to detect movement of the device 100 perpendicular to the face of the display 112 (ie, along the z axis). The second accelerometer operates similarly to the first accelerometer except that one output voltage tracks the movement of the device in the Z direction. Accelerometers may be an accelerometer commonly available from several sources, an analog device biaxial MEMS (micromachined) using part number ADLX202, or other suitable commercially available accelerometers.

The controller 102 receives a motion detection signal from the motion detector 110. In response to the particular motion detection signal, controller 102 causes display 112 to show various predefined graphical representations. By moving the electronic device 100 in a specific direction simultaneously with the particular display being shown, the user of the portable electronic device 100 may input data and commands to the controller 102.

I / O circuitry 114 may also include a transmitter or receiver (not shown) for electronically connecting controller 102 to antenna 116. Antenna 116 may be used to transmit and / or receive information related to user interface options and other information used in the processes described below.

In the illustrated embodiment of FIG. 1, the device 100 does not have a dedicated keypad, such as a cellular telephone keypad with alphanumeric keys 0-9 and function keys "send", "end", "clear", and the like, and thus, such a dedicated. Avoid weight, mechanical design complexity, size and weight associated with keypads.

A flowchart of a program 200 that can be executed by the portable electronic device 100 is shown in FIG. 2. Preferably, the programmed steps are executed by the controller 102. Generally, program 200 generates a series of graphical representations representing selected (but not yet entered) alphanumeric characters in response to movement of display 112. Program 200 also allows selected characters to be input in response to movement of display 112. Alphanumeric characters include English letters, numbers, Roman numerals, Chinese characters, Kanji, Kana, and / or other human-recognized language symbols.

Once the program 200 is initiated, the controller 102 initializes the " current character " variable to a default value at step 202. For example, a character located near the center of the virtual keypad can be selected as the default character to minimize the average "distance" to the desired character. Alternatively, the default character may be the last character entered. Then, in step 204, controller 102 shows a plurality of characters 300 (e.g., virtual keypad) where current character 302 is visually identified (e.g., highlighted, see Figure 3). Generate a graphical user interface on display 112.

The program 200 then enters a motion detection loop (steps 206-214). In this example, the motion detection loop is in accordance with the face of the display 112 in the direction requiring the vectors to the left (step 206), the right (step 208), the front (step 210) and the rear (step 212) to be required. Find the movement perpendicular to the face of the display 112 in the direction that requires the movement (eg, acceleration), and the downward vector (step 214). For convenience and clarity of explanation, it is assumed that the face of the display 112 extends coplanar with the face defined by the FIG. 3 drawing paper in which the display 112 of the device 100 is shown. A change in motion or speed from a stationary state means acceleration. The movement is preferably generated by the user holding the device 100 and then moving his or her hand, or the movement may be generated by the user tapping the side of the device 100. In general, some movement always occurs. Some movements are below the normal detection threshold and, therefore, cannot be considered by the program 200. Other movements may be detected, such as tremor and / or slight hand movements while unconscious, but it is desirable to be ignored to avoid changing the display without intentional hand movements by the user. In other extremes, excessive acceleration can be ignored. For example, dropping portable electronic device 100 on a hard surface creates an acceleration that is considered too large to be due to intentional hand gestures.

While this embodiment only finds movement in five directions, it will be readily apparent to one skilled in the art that detection of movement in several different directions is possible. For example, a motion (ie, diagonal movement) that includes both a significant forward vector and a significant right vector can be detected. Once detected, the diagonal movement may generate a change in the display 112 (eg, the diagonal movement of the cursor), or the diagonal movement may be ignored depending on the current indication on the display.

Once a motion with a magnitude greater than the minimum threshold and less than the maximum threshold is detected in one of the predefined directions, the corresponding steps (steps 216-224, 400) are taken to enter a new mode of operation and on display 112. Create a new representation in and / or enter the current character 302. In this embodiment, if movement to the left is detected (step 206), the current character 302 (or a button or icon, etc.) is moved to the left if possible (step 216). If there are no characters to the left of the current character 302, no change is made to the current character 302. Likewise, if a movement to the right is detected (step 208), the current character 302 is moved to the right if possible (step 210). If forward movement is detected (step 210), the current character 302 is moved up (step 220) if possible. Alternatively, in this embodiment, the button 303 of the menu may be selected. If backward movement is detected (step 212), the current character 302 is moved down if possible (step 222). Once the current character 302 is updated, the program 200 regenerates the presentation of the graphical user interface on the display 112 to show the highlighted new current character 302 (step 204). In another embodiment, the direction of the current character 302 moves opposite the detected direction of the display 112. In this way, the device 100 moves to the user but the appearance of the cursor position is shown in a stationary state.

The number of character positions at which the current character 302 is moving may vary. In one embodiment, the current character 302 is shifted by one position for each detected effective movement, regardless of the magnitude of the movement. In another embodiment, the number of positions where the current character 302 is moving may be proportional to the magnitude of the detected movement. For example, the range of motion between the minimum and maximum thresholds can be divided into three ranges (eg, low, medium, high). The detected motion can be classified by comparing the magnitude of the motion with thresholds of certain ranges. If a low range of motion is detected, the current character 302 can move one position (eg, from "1" to "2"). If a medium range of motion is detected, the current character 302 can move two positions (eg, from "1" to "3"). If a high range of motion is detected, the current character 302 can move three positions (eg, from "1" to "4"). Those skilled in the art will readily appreciate that various other associations between the detected position and cursor movement are possible.

Preferably, when the character that the user wishes to input is highlighted, the user creates a downward movement perpendicular to the face of the display 112. Such downward movement is preferred because it is the movement that the user associates with the pressing of a conventional key or button, and is therefore in part intuitive to the user. The program 200 detects the downward movement (step 214) and determines whether the text 302 or the menu button 303 is currently highlighted (step 215). If the character 302 is currently highlighted, the program 200 preferably enters the current character 302 (step 224). If the menu button 303 is highlighted, the program 200 preferably enters the menu mode in step 400 (see FIGS. 4 and 5). Of course, buttons other than the menu button may be used. Inputting alphanumeric characters preferably includes storing the characters in the memory 104 and displaying the characters in the alphanumeric character input area 304 on the display 112 of the portable electronic device 100. In some embodiments, entering a numeric character also includes dialing the entered number into the telephone system. Once the character is entered, the program 200 is preferably repeated (step 202).

In another preferred embodiment, the user “inputs” the current character or selects a virtual button using a predefined gesture movement (eg, a double tap on the portable electronic device 100 twice). This embodiment has the advantage of requiring one less accelerometer if the predefined gestures do not use z-axis acceleration, and avoiding the use of z-axis acceleration reduces the cost, complexity and thickness of the portable electronic device 100. do.

A flowchart of a program 400 that can be executed by the portable electronic device 100 to select menu items 504 is shown in FIG. 4. Preferably, the programmed steps are executed by the controller 102. Generally, program 400 generates a series of graphical representations representing selected (but not yet entered) menu items 504 in response to movement of display 112. The program 400 also allows the selected menu item 504 to be entered in response to the movement of the display 112. Menu items include text, icons, and / or other types of graphic symbols or symbol strings. Preferably, selection of a menu item executes a software routine associated with that menu item. For example, the menu item may cause the device 100 to enter or exit the alpha-entry mode described above.

Once the program 400 is initiated, the controller 102 initializes the " current menu item " variable to a default value at step 402. For example, the title of the first drop-down menu 502 in the menu bar 500 may be selected (see FIG. 5). Then, at step 404, the controller 102 generates a graphical user interface on the display 112 showing the menu bar 500, the drop-down menu 502, and the selected menu item 504. The program 200 then enters a motion detection loop (steps 406-414). In this example, the motion detection loop is in accordance with the face of the display 112 in the direction requiring the vectors to the left (step 406), right (step 408), front (step 410) and back (step 412). Find the movement, and the movement perpendicular to the face of the display 112.

As mentioned above, the magnitude and / or duration of certain movements may be greater or less than a predefined threshold. Preferably, these movements are ignored to avoid changing the display on the display 112 without intentional hand gestures by the user. Again, this embodiment finds only movement in five directions. However, one of ordinary skill in the art will readily appreciate that detection of movement in different directions is possible.

Once a motion with a magnitude greater than the minimum threshold and less than the maximum threshold is detected in one of the predefined directions, a corresponding step (steps 416-424) is taken to create a new indication on the display 112 and / or Or executes a software routine associated with the current menu selection 504. In this embodiment, if a movement to the left is detected (step 406), the previous menu 502 is shown if possible (step 416). If there is no menu on the left side of the current menu, nothing happens. Likewise, if a movement to the right is detected (step 408), the next menu 502 is shown if possible (step 410). If forward movement is detected (step 410), the current menu item 504 is moved up (step 420) if possible. If backward movement is detected (step 412), the current menu item 504 is moved down (step 422) if possible. Once the current menu item 504 is updated, the program 400 regenerates the presentation of the graphical user interface on the display 112 to show the highlighted new menu item 504 (step 404). In another embodiment, the direction in which the current menu item 504 moves is opposite the detected direction of the display 112. The number of positions moved in the menu 504 may vary. In one embodiment the number of positions is shifted once for each movement regardless of the magnitude of the movement. In another embodiment, the number of moving positions as described above may be proportional to the magnitude of the detected movement.

Preferably, when the menu item 504 desired by the user is highlighted, the user creates a downward movement perpendicular to the face of the display 112. Program 400 detects the downward motion (step 414) and executes a software routine associated with the current menu item 504 (step 424). Preferably, program 400 is repeated (step 402). In another embodiment, the user may use a predefined gesture instead of downward movement to preserve the accelerometer as described above.

A flowchart of a program that can be executed by the portable electronic device of FIG. 1 to execute a software routine in response to a direction of a predefined display movement (ie gesture) is shown in FIG. 6. Preferably, the programmed steps are executed by the controller 102.

Once the program 600 is initiated, the controller 102 waits for a signal from the motion detector 110 that a motion associated with the display 112 has been detected (step 602). If a movement is detected, the controller 102 executes a correlation software routine 604. The correlation software routine may include a detected movement, and optionally a series of previously detected movements stored in the data memory 104 in the sense of one or more movement patterns (e.g., no) stored in the program memory 106. A predefined gesture, such as shaking from side to side or shaking back and forth with a meaning of "yes". The controller 102 determines whether the correlation software has found a match with the predefined movement (step 606). If no match is found, the controller 102 adds the current movement to the series of movements stored in the data memory 104 for use in subsequent correlation steps (step 608). If a match is found at step 606, controller 102 may erase them so that the series of movements stored in data memory 104 are not reused (step 610).

The controller 102 then executes a display generation routine associated with the matched predetermined movement and / or toggles (ie, arm or disarm) the input mode (step 612). Preferably, the new display generated depends not only on the predefined movements but also on the current display. For example, if the current display asks the user if he wants to dial the displayed number, and if the movement is determined to be yes, such as a forward and backward motion, the new display may be an indication that the number is dialing. By activating and deactivating the input mode, the user can avoid undesired movements being interpreted as user input. For example, a predefined gesture such as tapping the display 112 twice (ie tapping twice within a predefined time) causes the device 100 to enter alpha-entry mode. can do.

A method and apparatus are provided for inputting user data and commands to a portable electronic device by detecting movement of the device. Although the foregoing description has focused on displaying displays with discrete cursor positions, such as menu items and / or keys on a virtual keyboard, those of ordinary skill in the art will appreciate that the resolution of cursor movement is as small as the display component that can change at least. It will be easy to see that it can be small. For example, a user may use the above to freely move an arrow on a display of a map or move a "windowed display" on a larger virtual display. For example, the user can see a map larger than the physical display 112 by scrolling in the direction of the detected movement or by scrolling in the opposite direction of the detected movement. Systems implementing the selection methods described herein can enjoy a rich graphical user interface without weighting the cost, complexity, size, and weight of a typical user input device.

The foregoing description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.

Claims (23)

A method for selecting a user interface option for inputting alphanumeric data 300 in a user interface of a portable electronic device 100 having a display 112, a motion detector 110 and a memory 104, Generating (204) a representation having a first alphanumeric character on a display of the portable electronic device; Detecting (206, 208, 210, 212, 214) motion associated with a display of the portable electronic device; And Storing the first alphanumeric character in a memory of the portable electronic device in response to the detection of the movement (224). Method comprising a. The method of claim 1, Generating a display on the display 112 of the portable electronic device 100 (204) includes generating a first display of the first alphanumeric character and the second alphanumeric character on the display, Detecting (206, 208, 210, 212, 214) movement associated with the display of the portable electronic device includes detecting a first movement associated with the display in a first direction. Characterized in that the method. The method of claim 2, The step 204 of generating a display on the display 112 of the portable electronic device 100 is a second display on the display—the second display is responsive to the detection of the first movement in the first direction. To visually identify the first alphanumeric character. Detecting (206, 208, 210, 212, 214) motion associated with the display of the portable electronic device further comprises detecting a second motion associated with the display in a second direction, Storing (224) the first alphanumeric character in the memory 104 of the portable electronic device may include storing the first alphanumeric character in the memory in response to the detection of the second movement in the second direction. Containing Characterized in that the method. The method of claim 3, Detecting (206, 208, 210, 212, 214) a first movement associated with the display comprises comparing a magnitude associated with the first movement with a predefined threshold, Detecting (206, 208, 210, 212, 214) a second movement associated with the display of the portable electronic device includes comparing the magnitude associated with the second movement with a predefined threshold. Characterized in that the method. The method of claim 1, further comprising: displaying the first alphanumeric character in an alphanumeric input area on a display 112 of the portable electronic device 100 in response to detecting the movement (206, 208, 210, 212, 214). 224, further comprising. 2. The method of claim 1, further comprising dialing the first alphanumeric character in response to detecting the movement. The method of claim 1, wherein the step 204 of generating an indication comprises a character selected from the group of characters consisting of letters, numbers, Roman numerals, Chinese characters, Kanji characters, Kana characters, and punctuation characters. Generating the first alphanumeric character. 1. A method for selecting a user interface option for selecting a menu item 504 in a user interface of a portable electronic device 100 having a display 112 and a motion detector 110. Generating (404) a representation with a first menu item on a display of the portable electronic device; Detecting (406, 408, 410, 412, 414) motion associated with a display of the portable electronic device; And Executing a software routine associated with the first menu item in response to detecting the movement (424). Method comprising a. The method of claim 8, Generating a display 404 on the display 112 of the portable electronic device 100 includes generating a first display of the first menu item and the second menu item on the display, Detecting 406, 408, 410, 412, 414 movement associated with the display of the portable electronic device includes detecting a first movement associated with the display in a first direction. Characterized in that the method. The method of claim 9, The step 404 of generating a display on the display 112 of the portable electronic device 100 is a second display on the display—the second display is responsive to the detection of the first movement in the first direction. To visually identify the first menu item. Detecting (406, 408, 410, 412, 414) motion associated with the display of the portable electronic device further comprises detecting a second motion associated with the display in a second direction, Executing 424 a software routine associated with the first menu item includes executing a software routine associated with the first menu item in response to detecting the second movement in the second direction. Characterized in that the method. The method of claim 9, Detecting (406, 408, 410, 412, 414) a first movement associated with the display (112) comprises comparing a magnitude associated with the first movement with a predefined threshold. . 9. The method of claim 8, wherein generating (404) generating the display comprises generating the first menu item as an icon. In the portable electronic device 100, Display 112; A motion detector (110) associated with the display, the motion detector configured to generate a motion detection signal in response to movement of the display; And A controller 102 functionally coupled to the display and the motion detector, the controller causing the display to generate user interface indications 300, 500 indicating at least one input option, the controller detecting the motion detection signal Select the at least one input option in response to Portable electronic device comprising a. The method of claim 13, The motion detector 110 is configured to generate a first motion detection signal in response to a first motion of the display 112 and a second motion detection signal in response to a second motion of the display, The controller 102 causes the display to generate a first user interface display 300, 500 indicating a plurality of user input options in response to receiving the first motion detection signal, the display generating the second And in response to receiving the motion detection signal generate a second user interface display indicative of the selected user input option. Portable electronic device, characterized in that. The portable electronic device of claim 13, wherein the portable electronic device does not have a dedicated keypad. The portable electronic device of claim 13, wherein the controller is further configured to dial a number. The portable electronic device of claim 13, wherein the portable electronic device comprises a wireless communication device. The portable electronic device of claim 13, wherein the portable electronic device comprises one of a cellular telephone, a pager, and a personal digital assistant. A method for selecting a user interface option for invoking a software routine in a user interface of a portable electronic device 100 having a display 112, a motion detector 110 and a memory 104, Detecting (602) a movement associated with the display; Executing (604) a correlation software routine in response to detecting movement of the display; Determining (606) if a movement associated with the display is correlated with a predefined movement; And Executing the software routine in response to determining that a movement associated with the display is correlated with the predefined movement (612). Method comprising a. 20. The method of claim 19, wherein executing (612) the software routine comprises executing a display generation routine. 21. The method of claim 20, wherein executing (612) the display generating routine comprises generating a windowed display associated with a larger virtual display. 20. The method of claim 19, wherein executing (612) the software routine comprises executing an input mode arming / disarming software routine. The method of claim 19, Storing (608) the movement associated with the display in the memory (104) if the movement associated with the display (112) is not correlated (606) with a predefined movement; And Erasing previous movements from the memory if the movement associated with the display is correlated with a predefined movement (610) Method further comprising a.