WO2007096688A1

WO2007096688A1 - A display and actuator device

Info

Publication number: WO2007096688A1
Application number: PCT/IB2006/000558
Authority: WO
Inventors: Alon Lotan; Nony Kamm
Original assignee: Alon Lotan; Nony Kamm
Priority date: 2006-02-23
Filing date: 2006-02-23
Publication date: 2007-08-30

Abstract

A handheld portable electronic device arranged to display actuator function representations on a display, the representations being displayed proximal to actuators which are arranged in a rotationally symmetric manner around the display. The device associates an actuator's operation with a function which corresponds to a proximally displayed actuator function representation. The device is operable in a plurality of modes corresponding to a plurality of detected orientations of use.

Description

A DISPLAY AND ACTUATOR DEVICE

The present invention relates to a display and actuator device operable in a plurality of modes corresponding to a plurality of detected orientations of use.

A significant problem with previous display and actuator devices suitable for image and data display and manipulation, for example watches, calculators, mobile phones and personal digital assistants (PDA's), is that they are only efficiently operable by a user when the device is suitably aligned with respect to the user. Typically, a device only has a single orientation of use, i.e. an orientation of the device with respect to the user for optimal operation of the device.

As an example, a standard mobile phone, comprising a keypad attached to a display, can only be optimally and efficiently operable in a single orientation with respect to the user. The keypad is required to be orientated so that the numbers on each button are the correct way up and the row of buttons "1, 2 and 3" is perpendicular to the user and located at the top of the keypad. Likewise, the display should be aligned with a user so images displayed appear the right way up. Typically the keypad is rigidly attached to the display so once the keypad is appropriately orientated so too is the display and vice versa.

Rotation of the mobile phone about a display axis, perpendicular to the planer surface of the display, for example by 90°, impedes a user's operation of the phone since not only is a displayed image askew by 90 but also the numbers on the keypad are askew. Furthermore, the relative position of each of the keys, i.e. which key is now above, below and to each side a key in the phone's new orientation, has altered. A user who was accustomed to the standard three column four row numeric arrangement of the numbers, i.e. 1, 2 and 3 on a top first row and *, 0 and # on a bottom fourth row, and who would intuitively know where a desired number's relative location was with regards to other numbers, would now be presented with a four row and three column arrangement of numbers with 3, 6, 9 and # on a top first row. The user would have to adjust to the new orientation of the keyboard and the askew numbers on the keypad. The user would have to himself re-calculate the relative positions of each of the keys and re-associate the askew numbers with the number it represents. Also the user would have to contend with viewing the displayed image and keypad numbers which are both askew by 90°. Thus such a rotated device would be extremely difficult to operate.

US 2002/0140675 discloses an example of a prior art device that attempts to overcome some of these issues. The device 100, as shown in Figure IA and IB comprises a display 101 and keys 102. The keys 102 are arranged on the device along one side of the display 103. Key icons 104 are displayed along the side of the display 103. Each icon 106 is displayed near a respective key 105. Each key icon 106 represents a function and this function is associated with the respective key 105 located near the key icon 106. Operation of the key 105 effects the respective function, e.g. pan displayed image 108 to the right. The device is operable in a portrait display mode, as shown in Figure IA, and a landscape display mode, as shown in IB. In the landscape display mode, the displayed image 108 is rotated by 90° and the key icons 104 are each individually and separately rotated by 90° such that they each remain arranged along the same side of the display 103 near their same respective keys 102. The key association remains the same with each key performing the same assigned function irrespective of the display mode of the device. In this manner, the displayed image 108 and key icons 104 are not askew when the device is operated a landscape display mode but are suitably orientated with a user for both portrait and landscape modes of display facilitating operation of the device in each display mode.

The problem with the device of US 2002/0140675 is that the relative position of the keys, with regards to a user direction, is altered when the display mode of the device is altered and more particularly the relative position of a key which effects a particular function changes. When the device is in a landscape display mode, i.e. rotated by 90°, what was, in the portrait mode, the top and bottom of the device now defines the sides of the device and vice versa. Keys formerly arranged along a bottom of a device in one display mode would be arranged along a side of the device in the other display mode. Thus the relative position of a key required to be operated to effect a certain function in one display mode is different to the relative position of a key required to be operated to effect the same function in another mode.

As an example, in the portrait mode as shown in Figure IA, the key 107 for panning the image up, is on the bottom left of the device. However, in the landscape mode as shown in Figure IB, there is no longer a key in the bottom left of the device and in order to pan the image up a user must operate key 107 which is now located on the bottom right of the device. Although the display and key icons would no longer be askew when the device was rotated by 90°, a user, who was accustomed to a certain portrait mode of key positions and would intuitively know the relative location of a key to carry our a desired function, would now have to adjust to the new orientation of the keys. The user would have to re-calculate and re-accustom himself to the relative positions of each of the keys. A further problem with the device of US 2002/0140675 is that the when changing between modes, the entire displayed image, comprising the image 107 and key icons 104, is not simply rotated 90° and displayed but separate parts of the entire displayed image are selectively rotated, i.e. the displayed image 108 is rotated about its centre and the key icons are each individually rotated about their respective centres. The entire displayed image effectively needs to be reorganised, recalculated and redrawn.

According to a first aspect of the present invention there is provided a display and actuator device comprising: a display in electrical communication with a processor, a first actuator, in electrical communication with the processor, arranged adjacent to a first side of the display, a second actuator, in electrical communication with the processor, arranged adjacent to a second side of the display and a sensor arranged to provide an output to the processor dependent on an orientation of the display; wherein the processor is arranged to be operable in a first state of operation where a first actuator function representation is displayed proximal to the first actuator and the first actuator function representation is associated with the first actuator such that operation of the first actuator effects the first function; the processor is further arranged to be operable in a second state of operation where the first actuator function representation is displayed proximal to the second actuator and the first actuator function representation is associated with the second actuator such that operation of the second actuator effects the first function and the processor changes between states of operation in dependence upon the output from the sensor. According to a second aspect of the present invention there is provided a method of altering the state of operation of a display and actuator device comprising the steps of: in a first state of operation, displaying a first actuator function representation, representing a first function, proximal to a first actuator; associating the first function with the first actuator such that operation of the first actuator effects the first function; altering the state of operation of the display and actuator device to a second state in dependence on an output from a sensor, the output being dependent on an orientation of the device; in the second state of operation, displaying the first actuator function representation proximal to a second actuator and associating the first function with the second actuator such that operation of the second actuator effects the first function.

According to a third aspect of the present invention there is provided an interface for a device as described in the first and second aspects above, in which an actuator function representation is displayed proximal to an actuator, the actuator being located adjacent to a display screen.

Devices according to at least some embodiments of the present invention provide a display and actuator device operable in at least a first and second mode of operation, corresponding to at least a first and second orientation of use of the device, which are capable of automatically switching between modes. In a first mode of operation, a function, represented by an associated actuator function representation, is assigned to a first actuator and the actuator function representation is displayed proximal to the first actuator. In a second mode of operation the actuator function representation is displayed proximal to a second actuator and the associated function is re-assigned to the second actuator. The operational mode of the device is dependent on an orientation sensor which detects the orientation of the device.

The present invention may be carried into practice in a number of ways, and several specific embodiments will now be described, by way of example, with reference to the accompanying drawings, in which:

Figures IA and IB show a prior art device operating in a portrait display mode and a landscape display mode respectively;

Figures 2A and 2B show schematic diagrams of a display and actuator device according to a first embodiment of the present invention;

Figure 3 shows a schematic diagram of the main elements in a display and actuator device according to the first embodiment of the present invention;

Figure 4 shows a schematic flow chart diagram indicating the operations of a processor suitable for use with a display and actuator device according to a second embodiment of the present invention;

Figures 5A and 5B show schematic diagrams of a display and actuator device according to a third embodiment of the present invention;

Figures 6A, 6B and 6C show schematic diagrams of alternative device shape and key arrangements suitable for use with embodiments of the present invention; Figures 7A and 7B show an actuator arrangement suitable for use with embodiments of the present invention;

Figure 8 shows an alternative actuator arrangement suitable for use with embodiments of the present invention;

Figure 9 shows a further alternative actuator arrangement suitable for use with embodiments of the present invention;

Figure 10 shows exemplary display layouts suitable for use with devices according to the present invention;

Figures 2A and 2B show a schematic diagram of a display and actuator device 200 according to a first embodiment of the present invention in a first and second mode of operation respectively.

The device 200 comprises a processor (not shown) electrically connected to a display 201 (for example a 3" LCD HLM Series graphic display as manufactured by Hosiden), and at least two actuators 202, 203, for example buttons. The actuators could also be keys, switches, scroll wheels, touch sensitive sections of a screen or micro switches (for example series 340 micro switches as manufactured by Bighead). A first button 202 is located adjacent to a side of the display 204. A second button 203 is located adjacent to another side of the display 205. The buttons are located proximal to the display in a rotationally symmetric arrangement. Preferably, the relative position of the first button along its adjacent side is equal to the relative position of the second button along its adjacent side, i.e. the ratio of distance 212 / distance 214 is equal to the ratio of the distance 213 / distance 215. The processor (for example a 3x16 bit analogue to digital convector MSP430 series microprocessor as manufactured by Texas Instruments) is arranged to cause the display of an actuator function representation 206, for example a button icon. The actuator function representation could be any representation for example a word, letter, image, icon or symbol which represent a particular function which is carried out when an associated actuator is actuated. The button icon is representative of a function, for example the input of a character "A". A device direction 207 corresponds to an initial orientation of the device which gives a reference direction for subsequent rotations of the device. A display direction 208 corresponds to a direction of a displayed display or image 214 according to the displayed image's frame of reference. A user direction 209 corresponds to a direction of a user's operation of the device. For optimum operation of prior art devices a device direction and display direction must be aligned with a user direction. As will be discussed below, embodiments according to the present invention effectively enable the rotation the device direction and display direction so that they are aligned with a user direction.

A first mode of operation is shown in Figure 2A. The device is initially orientated so that the device direction 207 is aligned with a user direction 209.

The processor is arranged such that the display direction 208 is also aligned with the device direction and user direction. Button icon 206 is displayed on the display 201 proximal to the first button 202. Furthermore the processor is arranged so that the first button 202 is assigned the function associated with the button icon 206. Accordingly, actuation of the first button 202 effects the function which the button icon 206 represents, i.e. inputting the character A into the device. In a second mode of operation as shown in Figure 2B, the device direction is perpendicular to the user direction, i.e. the device is rotated by a 90° angle clock-wise relative to the user direction 209. When changing between modes, the processor carries out the following steps. The processor displays the button icon 206 proximal to the second button 203, e.g. it rotates the entire display by a pre-determined angle, for example 90° counter clockwise, thus rotating the display direction 90° counter clockwise so that it is aligned with the user direction 209. Furthermore, the processor reassigns the function which the button icon 206 represents from the first button 202 to the second button 203. Accordingly, actuation of the second button 203 effects the function which the button icon 206 represents, i.e. inputting the character "A" into the device. Since the same function is effected by the same relatively positioned button, but not the same actual button, to all intents and purposes the device direction 207 remains constant and therefore still aligned with the user direction 209. Thus, in spite of the device being rotated by 90° with regards to the user direction 209, causing device direction 207 and display direction 208 to be misaligned to the user direction by 90°, when the device changes operation mode, the display direction 207 and user direction 209 are brought into alignment with each other.

Preferably, the device includes two further buttons 210 and 211, which are located adjacent to the other two sides of the display respectively such that upon a further clockwise rotation of the device through 90° the button icon 206 would be displayed proximal to button 211 and the function represented by the button icon 206 would be re-assigned to button 211. Similarly, upon a further clockwise rotation of the device through 90° the button icon 206 would be displayed proximal to button 210 and the function represented by the button icon 206 would be re-assigned to button 210. As the reader will appreciate similar processes are carried out for anti-clockwise 90° rotations of the device and rotations of multiples of 90° rotations.

The change between operational modes of the device is effected by a sensor located on the device which can automatically detect the rotation of the device. Upon a detection of rotation of the device by a predetermined angle, the sensor outputs a signal to the processor causing the processor to change the state of operation of the device. Gravity based sensors, for example incorporating a mercury switch, suitably arranged in the device could detect rotation of the device about a horizontal axis. Magnetic based sensors, for example incorporating a compass, suitably arranged in the device could detect rotation of the device about a vertical axis. Alternatively, the operational modes can be changed by a user's command, for example actuation of an actuator. Alternatively, the device can change mode upon operation of a button.

Preferably the device is in the form of a handheld portable electronic device. Such a device improves user interaction with the device by adjusting to the direction of interaction.

Figure 3 shows a schematic diagram of interconnections of elements in a display and actuator device according to a second embodiment of the present invention. A processor 301 receives inputs from a direction sensor 305 and keys 304. The processor controls the display of the displayed image 302 and key icons 303 as well as what function is effected upon the activation of a key.

The sensor provides an output relating to a relative orientation of the device. The processor uses this output to determine the appropriate display orientation of the displayed image and key icons as well as which function is to be associated a key. Preferably, the display is in a rotationally symmetric shape such as a regular n-sided polygon and the device changes its operational mode upon detection of a predetermined angle, such as a multiple of an angle of 360/n°. When the device has been rotated through such an angle, e.g. clockwise, the entire displayed display, comprising a displayed image, information and key icons, is rotated anti-clockwise by the same angle. If the display is a non-rotationally symmetric shape, e.g. rectangular, the processor can rotate and re-scale the entire displayed image to the appropriate dimensions suitable for the new display mode.

Figure 4 shows a schematic flow chart diagram indicating the operational steps carried out by a processor according to the second embodiment of the present invention. At step 401 the processor determines the key layout of the device and assigns a function to each key. At step 402 the processor displays a key icon, which represents the function assigned to a key, proximal to each key. At step 403 the processor displays an image or information. The orientation of the displayed key icons and the displayed image or information corresponds to an initial orientation such that a user using the device orientated in the initial orientation would view the icons and images or information the correct way up.

At step 404 the processor receives an output from an orientation sensor which indicates a detected orientation of the device. At step 405 the processor determines whether the orientation is different to a reference orientation. If the detected orientation differs to the initial orientation, the processor re-assigns the key assignments at step 401 and causes the orientation of displayed key icons and the displayed image or information to be aligned with the detected orientation such that a user using the device orientated in the detected orientation would view the icons and images or information displayed on the display the correct way up at steps 402 and 403. If the detected orientation corresponds to the initial orientation, the processor maintains the key assignments and the orientation of the displayed key icons and images or information at step 406.

At step 407, the processor determines whether a key was pressed. If a key is pressed or actuated, the processor effects the function assigned to the actuated key. This function corresponds to the key icon displayed proximal to the pressed key. If the key's function changes the function set of the keys, for example a menu key is pressed which brings up a list of menu options for each key, the processor determines the appropriate key layout and new assignments of new functions and repeats step 401.

Figure 5 A shows a device 500 according to a further embodiment of the present invention where each side of the display 503, 506, 509 and 512 has two adjacent buttons, each button having an associated button icon in an initial operational mode.

In a first mode of operation, the device is aligned so that a user direction or interaction orientation 523 is parallel to a device direction 513. The processor displays button icon 514, 515, 516, 517, 518, 519, 520 and 521, each representing a function, proximal to button 501, 502, 504, 505, 507, 508, 510 and 511 respectively. The processor associates each button with the function that the respective proximal button icon represents, i.e. button icon 514 is displayed proximal to button 501 and the button 501 is associated with the function represented by button icon 514 such that operation of button 501 effects the function represented by button icon 514. In a second mode of operation as shown in Figure 5B, the device is aligned so that a user direction 523 is perpendicular to a device direction 513, i.e. the device 500 is rotated 90° clockwise with respect to the user direction 523. The processor rotates the entire display, including all the button icons, i.e. it displays the entire display rotated by 90° anti-clockwise with respect to the user direction. Accordingly, the button icons are no longer proximal to their respective associated buttons, i.e. button 501 is no longer proximal to button icon 514, but is now proximal to button icon 516. The processor re-assigns the associations of the buttons such that each button is associated with the function that the button icon proximal to said each button represents, i.e. button 501 is associated with the function represented by button icon 516. A similar process takes place for further rotations of the device through multiples of 90° clockwise or anti-clockwise.

Thus in spite of the device being rotated through 90° and misaligned with a user direction 523, the relative position of a button which effects a given function remains constant. Accordingly, from a user's perspective, the position of a button relative to the user that effects a given function is constant in spite of rotations of multiples of 90°, i.e. to input an "E" the user would always press the bottom left button irrespective of the mode of operation of the device. When the device is rotated of 90, 180 or 270 degrees, the same particular button icon 'E' will stay in the same relative location, in proximity to a button positioned in the same relative location. This allows the continuity of interaction from different directions, which allows the rotation of the device while working, according to a user's working needs

The display can be any shape for example rectangular or polygonal. However, preferably the display shaped in a rotationally symmetric shape such as a regular polygon of n sides, for example a triangle, pentagon or hexagon as shown in Figures 6A, 6B and 6C respectively. Keys are preferably arranged around the display in a rotationally symmetric arrangement such that, for each actuator located adjacent to a side of the display, there is an equivalently positioned actuator located adjacent to each of the other sides of the display.

Figures 7A and 7B shows a further embodiment of the present invention. The embodiment provides a portable electronic device 700, including: keys for input wherein the keys are associated to a display; display means, e.g. an LCD display, also displaying images associated to the keys; direction detection means, such as a gravity sensor, for detecting changes in the position, displacement or rotation of the device; processing means; also for displaying visual information that represents the current and changing applications of each key, under the adjacent key; wherein the direction for input and display, and therefore the direction for interaction with the device is automatically changed when the direction of the device is changed, to face a predetermined direction such as the ground or North.

The device 700 displays key icons 704 and images/information 703 and is housed in a hand held unit to be carried by a user. The displayed icons are interactive with the user through touch sensitive keys 702 and are responsive to user selective actuation and movement. The direction for input and display, and therefore the direction for interaction with the device, can be automatically changed when the direction of the device is changed as detected by the direction detector. Both the display and the soft key functions are controlled by the processing unit. The arrangement of re-assignable keys and the presented operating program allows for a continuity of interaction from different directions, which allows the rotation of the device while working, according to working needs.

The keys 702 are transparent and are positioned over the display screen

701. The key icons 704 are displayed on the screen in proximity to the keys providing a clear indication of the keys changing functions.

As shown in figure 7B, a frame of micro-switches 705 is located around the display screen 701. The transparent keys 702 are placed over the screen 701. Preferably the sheet is rigid and manufactured to allow parts of the sheet, located above the keys, to be pressed separately. Each separated part 'P' of the transparent sheet is constructed with the ability to move down within a given range, thus activating a micro-switch located at its base of connection. Pressing the different parts 'P' of the sheet results in activating different micro- switches on the micro-switches' frame 705. This way, the transparent sheet is functioning as multiple keys for input to the processing unit. Moreover, each separated part 'P' of the transparent sheet is a 'soft key'- its function is programmable and is indicated by a key icon 704 which is displayed under it or next to it. The transparency of the material allows the displayed indication to be clearly visible under the key. Positioning the keys over the screen whilst still enabling their respective key icon to be viewed beneath the key allows direct interaction with an operating program. This leads to a simpler, more durable solution for creating a touch screen. Also, the keys provide protection to the screen.

An alternative transparent sheet and key mechanism is shown in figure 8. The frame of micro-switches is replaced by a frame with a touch screen mechanism 801, so the movement of each key 802 activates an input command. Additionally, a middle part 803 integrally connected to the keys 802 can be pressed separately, to also function as a key. The transparent sheet is also used to protect the display screen, being made out of durable and cheaper material than the screen. Preferably, the transparent sheet is attached to the device, by snap fitting, enabling the sheets to be detached and replaced a multiple number of times, by the user.

A further alternative transparent sheet and key mechanism is shown in figure 9. The transparent sheet 901 is structured as a frame around and over the display screen 902 including only the edge parts of the display 903 and not including a middle part 904 of the display. The transparent sheet may have an optical, lens-like shape for enlarging the displayed image in order to improve visibility of the display screen.

The particular function which each button icon represents can be preprogrammed and can vary depending upon the use of the device. As an example, if the device were to be used as a calculator the function can correspond to standard functions associated with calculators such as inputting numerical digits, adding, subtracting, dividing and multiplying inputted numbers or calculating a resulting computation. Alternatively, the function can relate to menu items. Figure 10 shows some exemplary display layouts corresponding to possible operating programs for devices according to the present invention.

Claims

1. A display and actuator device comprising: a display in electrical communication with a processor, a first actuator, in electrical communication with the processor, arranged adjacent to a first side of the display, a second actuator, in electrical communication with the processor, arranged adjacent to a second side of the display and a sensor arranged to provide an output to the processor dependent on an orientation of the display; wherein the processor is arranged to be operable in a first state of operation where a first actuator function representation is displayed proximal to the first actuator and the first actuator function representation is associated with the first actuator such that operation of the first actuator effects the first function; the processor is further arranged to be operable in a second state of operation where the first actuator function representation is displayed proximal to the second actuator and the first actuator function representation is associated with the second actuator such that operation of the second actuator effects the first function and the processor changes between states of operation in dependence upon the output from the sensor.

2. A display and actuator device as claimed in claim 1 wherein the second actuator is positioned at a same relative position along the second side of the display as a relative position of the first actuator along the first side of the display.

3. A display and actuator device as claimed in claim 1 or 2 wherein the display is shaped in a rotationally symmetric shape of a regular polygon of n sides, for example a triangle, a square or a pentagon.

4. A display and actuator device as claimed in claim 3 in which the device is operable in n states.

5. A display and actuator device as claimed in claim 4 in which the device changes state upon a sensor output indicating a rotation of the device of an angle of a multiple of 360/n °.

6. A display and actuator device as claimed in any of the previous claims further comprising a plurality of actuators arranged adjacent to each side of the display.

7. A display and actuator device as claimed in claim 6 wherein each one of the plurality of actuators is associated with an actuator function representation, said actuator function representation being displayed proximal to the each one of the plurality of actuators.

8. A display and actuator device as claimed in any of the previous claims wherein the actuators are rotationally symmetrically arranged around the display.

9. A display and actuator device as claimed in any of the previous claims wherein the sensor is a gravity based sensor.

10 A display and actuator device as claimed in any of the previous claims wherein the sensor is a magnetic based sensor.

11. A display and actuator device as claimed in any previous claim wherein the actuators are located around an outer perimeter of the display.

12. A display and actuator device as claimed in any previous claim wherein the actuators are located on the display.

13 A display and actuator device as claimed in claim 12 wherein the actuators are transparent.

14. A display and actuator device as claimed in claim 13 wherein in the first state the first actuator is located above the first actuator function representation.

15 A display and actuator device as claimed in any previous claim wherein the display is a touch sensitive screen.

16 A display and actuator device as claimed in claim 15 wherein the actuators comprises a section of the touch sensitive screen adjacent to the side of the screen.

17. A display and actuator device as claimed in any previous claim wherein the actuators comprises one of a button, switch or scroll wheel.

18. A display and actuator device as claimed in any previous claim wherein the processor is arranged to further display at least one of information, data or images.

19. A display and actuator device as claimed in claim 18 wherein the processor is arranged to rotate the display of at least one of: information, data or images by a predetermined angle.

20. A method of altering the state of operation of a display and actuator device comprising the steps of: in a first state of operation, displaying a first actuator function representation, representing a first function, proximal to a first actuator; associating the first function with the first actuator such that operation of the first actuator effects the first function; altering the state of operation of the display and actuator device to a second state in dependence on an output from a sensor, the output being dependent on an orientation of the device; in the second state of operation, displaying the first actuator function representation proximal to a second actuator and associating the first function with the second actuator such that operation of the second actuator effects the first function.

21. A method of altering the state of operation of a display and actuator device wherein the step of displaying the first actuator function representation proximal to a second actuator comprising rotating a displayed image by a predetermined angle.

22. A computer program adapted to carry out a method as claimed in claims 20 and 21.

23. An interface for a device according to any of claims 1 to 17 in which an actuator function representation is displayed proximal to an actuator, the actuator being located adjacent to a display screen.