Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
  • G06F1/16—Constructional details or arrangements
  • G06F1/1613—Constructional details or arrangements for portable computers
  • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
  • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
  • G06F1/1694—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
  • G06F1/16—Constructional details or arrangements
  • G06F1/1613—Constructional details or arrangements for portable computers
  • G06F1/1626—Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q10/00—Administration; Management
  • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
  • G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q30/00—Commerce
  • G06Q30/06—Buying, selling or leasing transactions
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
  • G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
  • G06F2200/161—Indexing scheme relating to constructional details of the monitor
  • G06F2200/1614—Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
  • G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
  • G06F2200/163—Indexing scheme relating to constructional details of the computer
  • G06F2200/1636—Sensing arrangement for detection of a tap gesture on the housing
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
  • G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
  • G06F2200/163—Indexing scheme relating to constructional details of the computer
  • G06F2200/1637—Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M1/00—Substation equipment, e.g. for use by subscribers
  • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
  • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
  • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M1/00—Substation equipment, e.g. for use by subscribers
  • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
  • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
  • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
  • H04M1/72445—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality for supporting Internet browser applications
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04M—TELEPHONIC COMMUNICATION
  • H04M2250/00—Details of telephonic subscriber devices
  • H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion

Definitions

• the present invention relates, in general, to the field of mobile computing and communication hand-held devices utilizing an information display, specifically to view navigation and scrolling of a stored virtual display or a magnified image of the display in response to changes of the orientation at which the device is held in one hand.
• Web Clipping is an approach taken by Palm Inc. to allow their commercially available Palm Series to browse the Internet.
• Web Clipping applications were developed for popular web sites, which respond to user's queries by clipping minimal information from the accessed sites in a typically textual form.
• the disadvantage of Web Clipping is that limited information is brought to the user, and not all web sites have the application to create the web clipping for the user. In many cases, even after clipping, the amount of available information is much more than can fit in one display view.
• optical magnification of a small display has been adopted for virtual reality helmets and other applications where the display is seated at a fixed distance from the user's eye.
• U.S. Patent No. 5,739,955 discloses such a virtual reality helmet with binocular magnifying optics.
• Optical magnifiers may appeal to persons with impaired vision who cannot view detailed information in a small handheld display.
• the main problem with optical magnification when used with a hand-held device is the difficulty of use.
• Such a hand-held device and its associated optics must be placed at a relatively fixed place in front of the user's eyes so that the magnified display can stay in focus.
• U.S. Patent No. 3,976,995 (Reissue Patent. 32,365) teaches the use of a processing display which moves the message across the display in a continuous fashion so that the display needs to be only large enough to present a relatively small portion of the total message. While this approach may be useful to display simple sentences, it is not practical when displaying complex graphic information. And even with simple character displays, the user needs to wait patiently while the message is scrolling around.
• U.S. Patent 5,311 ,203 discloses a hand-held viewing apparatus that determines the three dimensional direction in which it is pointing and automatically presents information to match the features visible in its field of view.
• This device is intended to observe, identify and locate stars or stellar constellations in an observed portion of the night sky. Once the exact direction of the device is measured, the device correlates the viewed objects and information stored in its database and displays identifying annotations near the corresponding objects. Since the device must correlate exactly between the database information with the observed objects, an exact spatial angle relative to earth is required, thus making the device prohibitively complex and expensive.
• the present invention seeks to provide a user friendly and convenient navigation of displayed information in a hand-held device, so that a large amount of data can be viewed in the relatively small size of the device's display.
• the present invention allows the operator to perform such navigation of the display view using the same hand that holds the device.
• a hand-held device in accordance with the present invention has a housing with a display, memory means to store a virtual display, processing means, and an orientation sensor responsive to changes in the spatial orientation at which the device is held.
• the display can be set at a navigation and scrolling mode or at a fixed mode. When the view navigation mode is set, the display scrolls the stored virtual display under the direction of the processing means in response to the changes in the orientation of the device measured by the orientation sensor.
• the display When set to the fixed mode, the display remains fixed and no longer follows the changes in orientation.
• the display provides a clear visual indication during the view navigation mode in order to alert the operator that it will scroll in response to changes in the orientation of the device.
• the device further comprises a set of two ergonomic switches placed along both sides of the housing so that the operator must press both switches during the view navigation mode.
• Such arrangement ensures convenient activation of the view navigation mode, as it is natural for the holding hand to press both switches at the same time. Also, the requirement that both switches must be pressed in order to activate the view navigation mode better protects against unintentional change of the displayed view.
• said ergonomic switches are replaced with means to detect a tap by the operator's finger at the bottom of the device as an instruction to set said view navigation mode after a short time delay. Once the view navigation mode is set, it remains in this mode for a preset period of time or as long as the operator substantially changes the orientation of the device.
• preset hand gestures by the operator communicate special commands to the hand-held device. For example, a quick upward jerk of the hand-held device without substantial change in orientation might be used to set the device to the view navigation mode.
• a built-in voice recognition means accepts a spoken word command from the operator to enter said view navigation mode.
• the present invention is particularly suitable for a display magnification application. Once set to magnification mode, the current view on the display can be arbitrarily magnified even by a large factor, and then using the view navigation mode to navigate the enlarged view on the display.
• FIG. 1A, FIG. 1B, and FIG. 1C show three subsequent views of a hand-held device incorporating the present invention when it is rolled from left to right while navigating the virtual display of FIG. 1 D.
• FIG. 2 indicates the relative axes of roll and pitch along which the hand-held device is rotated in order to navigate the view in accordance with the present invention.
• FIG. 3 illustrates the use of the present invention as a magnifier for the display of a hand-held device in accordance with the present invention.
• FIG. 4 shows an ergonomic embodiment of the present invention that uses two side switches to activate the view navigation mode.
• FIG. 5 is a block diagram of the embodiment in FIG. 4.
• FIG. 6 outlines the software flow diagram for the embodiment of the invention of FIG 5.
• FIG. 7 is a timing chart showing an example of the time dependent response curve of the system to changes in orientation along one axis.
• FIG. 8 is a block diagram of another embodiment of the present invention that uses a finger tap of the operator to set the hand-held device into view navigation mode.
• FIG. 9 is a perspective view of the hand-held device of FIG. 8 showing a cutaway view of the finger tap sensor.
• FIG. 10 is the block diagram of the embodiment of the present invention in the form of an add-on upgrade to an existing hand-held device.
• FIG. 1 1 shows yet another embodiment of the present invention that uses different sensors to track the orientation, and uses a speech recognition module to accept spoken commands to enter and exit view navigation mode.
• FIG. 12 is a timing chart illustrating how the view navigation mode is entered and exited in accordance with the present invention.
• FIG. 13A outlines the software flow diagram for the embodiment of FIG. 12C that remains in view navigation mode only for a fixed time period and then exits to the fixed mode.
• FIG. 13B outlines the software flow diagram for the embodiment of FIG. 12D that remains in view navigation mode as long as the operator continues to change the orientation of the device.
• FIG. 14 illustrates a further embodiment of the present invention that uses an additional Z-axis accelerometer to identify the operator's vertical hand gestures to enter the view navigation mode and for other commands.
• FIG. 15 illustrates the placement of the three accelerometers of the embodiment of FIG. 14 on the main PCB of the hand-held device.
• FIG. 16 is a timing chart illustrating the use of the embodiment of the present invention of FIG. 14 with three accelerometers.
• FIG. 17 outlines the software flow diagram for the embodiment of FIG. 14 that uses a vertical movement gesture of the operator to enter and exit the view navigation mode.
• This invention allows hand-held communication or computing devices with a relatively small display to conveniently navigate a large stored virtual display with one hand.
• Such devices may include mobile computers like the commercially available PALM PILOT, Cassiopea, PSION, Newton, and other palmtop computers. It may also include various PDA devices, mobile. hand-held terminals, advanced pagers, and a variety of mobile telephones with expanded information displays.
• the hand-held device in accordance with the present invention employs two operational modes, which throughout this document are referred to as view navigation mode and fixed mode.
• view navigation mode When set to the view navigation mode, the display view is automatically scrolled to follow the rotational movements of the holding hand.
• fixed mode When set back to the fixed mode, the display view becomes stationary and no longer follows said movements of the hand.
• FIG.1 shows an overview of the operation of a hand-held device 10 built in accordance with the present invention when it is set to the view navigation mode.
• the device has a flat display 12 that is typically made of a LCD with optional back lighting, and a plurality of operational keys.
• the display 12 is too small to show the entire virtual display 30 that is stored in the hand-held device and is shown in FIG. 1 D.
• the virtual display 30 shows a combination of easily identified graphical objects like a space station 26, a space shuttle 24, and an astronaut with the American flag 22, in addition to a character message 28.
• FIG. 1 A the navigation process is started when the operator's hand 20 rolls the device 10 to the left so that the display 12 shows the left portion of the stored virtual display 30.
• FIG. 1B shows how the view in the display 12 scrolls to the left, as the space shuttle picture 24 comes into view.
• FIG. 1 C further shows how the right portion of the virtual display 30 including the American flag 22 is viewed when the operator's hand 20 continues to roll to the right.
• FIG. 2 indicates the relative axes of orientation along which the hand-held device 10 is rotated in order to navigate the display 12 in accordance with the present invention.
• I will refer to axis 32 as the Y-axis or, with influence from aviation and naval terms, the roll axis.
• I will refer to axis 36 as the X-axis or the pitch axis.
• changes in the height at which the device 10 is held, and which are measured along the Z-axis 40 are used to switch the view navigation mode on. Referring back to the process shown in FIG. 1 , it should be clear even though not shown in this drawing, that the display can be navigated vertically as the device 10 is tilted back and forth along the pitch axis.
• the control software of the present invention smoothes the hand movements to provide convenient navigation even when using relatively coarse orientation sensors.
• FIG. 3 illustrates how the present invention is well adapted to provide display magnification and navigation of said magnified display.
• FIG. 3A shows a hand-held device that displays some information 50, which may not be viewable by a user who suffers from impaired vision.
• the display 12 shows a magnified portion 52 of the original information 50 as shown in FIG. 3B.
• the device 10 can now navigate the magnified view in accordance with the present invention.
• the display preferably includes some visual indication 54 to the operator, to alert that the display will be scrolled as the operator changes the device orientation.
• the indication shown in the drawing depicts four rotated 'L' shaped 54 markers at the four corners of the display 12.
• the visual indication 54 greatly benefits those embodiments of the present invention that do not use an actual switch for entering and leaving the view navigation mode. It should be noted that for a magnification application for persons with impaired vision, the display should employ an active TFT LCD or some other display technologies that exhibit sharp contrast features.
• FIG. 4 shows an ergonomic embodiment of the present invention that uses two side switches 62 and 64 to activate the view navigation mode.
• the drawing illustrates mechanical switches that utilize springs 66 to provide tactile response to the operator. Other tactile switches like the sealed membrane type and capacitance strip switches may be used.
• the hand-held device 10 is set to the view navigation mode only when both switches are pressed by the operator, and it reverts back to fixed mode when at least one of the switches is disengaged. Alternatively, the switches can be used to signal a command to enter the view navigation mode and allow the program to terminate the mode in accordance with the description below. While the present invention will work with one switch only, experimentation showed that this arrangement of two side switches is more ergonomic and provides better intuitive control to the operator. The use of two switches on both sides of the hand-held device seems to equally benefit both right-handed and left-handed persons.
• FIG. 5 is a block diagram of the embodiment in FIG. 4, where the view navigation circuitry of the present invention is fully integrated into the main circuitry of the hand-held device.
• All smart hand-held devices with displays 12 typically employ at least one micro-controller 100 or a micro-processor, memory 102 to store program and display data, and a display controller 104.
• Orientation sensor circuitry 80 includes X-axis 82 and Y-axis 84 accelerometers that are respectively mounted inside the hand-held device to align with axes 36 and 32 of FIG. 2.
• the view navigation according to the present invention provides a close loop between the user's movements and the actual navigation, there is no need for an exact alignment between the sensor and the device axes. In fact, it is enough that the two sensors will be generally perpendicular to each other. Also, any misalignment among the sensor axes of sensitivity and the device's X and Y axes can be corrected by the program.
• Such accelerometers are preferably implemented by a surface micro-machining technique that builds electromechanical structures in silicon. The small size of micro-machined structures makes it economical to place the circuitry and sensors on the same die. For example, a commercially available 1 micrometer CMOS process known as "iMEMS" from Analog Devices Inc.
• ADXL202 accelerometer which incorporates two axes and circuitry to digitally generate a duty cycle modulator (DCM) output.
• DCM duty cycle modulator
• Such DCM or other analog to digital conversion 86 and 88 are used to interface the orientation data to the micro-controller 100.
• the accelerometers provide tilt angle information depending on their inclination relative to earth gravity. When the operator is moving, there are also small acceleration artifacts picked up by the accelerometers. The program can filter out such acceleration artifacts, or the operator may re-navigate the view to correct undesired view movements.
• the right switch 62 and the left switch 64 are connected in series to the VCC potential 70 (or a ground potential if reverse logic is used) so that when both are pressed, they connect an activation signal to the micro-controller 100 via line 72.
• This signal instructs the program of the micro-controller to set to the view navigation mode. If the hand-held device includes a beeper 94, the program can provide a beep to alert the operator that the view navigation mode is set.
• the micro-controller instructs the display controller 104 to provide a visual indication 54 as shown in FIG. 3B to the operator that the hand-held device is in the view navigation mode.
• the micro-controller 100 translates the changes in pitch and roll orientation as communicated through lines 90 and 92 to navigation commands that scroll a virtual display which is stored in memory 102. For example, when the Y-axis accelerometer indicates that the operator has rotated the hand-held device 10 like in FIG. 1 B to the right, the micro-controller 100 controls the virtual display in the memory 102 and the display controller 104 to scroll the view in the display 12 to the right.
• FIG. 6 outlines the software flow diagram for the embodiment of the invention of FIG 5.
• the flow from start 120 to end 134 is performed several times a second in a standard polling process of the micro-controller 100.
• the initialization step at block 122 the current boundary of the display view is marked in comparison to the stored virtual display.
• the status of both navigation switches 62 and 64 are checked at block 124. If both switches are pressed, the system is set to view navigation mode in block 126 providing the visual indication 54 to alert the operator that changes in orientation of the hand-held device will navigate the display.
• the pitch and roll data are acquired, stored and compared to the previous reading. If a change in orientation is detected at block 130, the program computes the new boundary for the view at block 132.
• the program can be set with different response curves for computing the new boundary in response to changes in orientation at block 132.
• Fine or coarse modes of response can be set by the operator or can be changed dynamically during the time the system is in view navigation mode. With fine response, the display view navigates the virtual display at a relatively slow rate in response to the orientation changes. With coarse response, the display view changes rapidly in response to the orientation changes.
• FIG. 7 is a timing chart showing an example of the time dependent response curve of the system to changes in orientation along one axis. Similar relations are employed along the other orientation axis, although response figures may be set with different biases to each axis.
• FIG. 7A shows the relative response curve that is setup onto the program and may be modified by the operator. The relative response is obtained by dividing the amount of change in view navigation, which is a normalized figure proportional to what percentage of the virtual display has been scrolled, to the change of orientation that caused it. Thus, the operator can achieve fine and slow navigation when the relative response figure is low, and coarse and fast navigation when the relative response figure is high.
• FIG. 7B illustrates the reading from the orientation sensor on one axis as represented by the signal on line 90 or 92 of FIG. 5.
• FIG. 7C shows the time corresponding changes in orientation along the monitored axis as measured by the micro-controller 100 at block 128 of FIG. 6.
• FIG. 7D illustrates the resulting navigation of the view along the monitored axis as computed by the micro-controller 100 at block 132 in accordance with one mode of operation.
• FIG. 7E illustrates the resulting navigation of the view along the monitored axis as computed by the micro-controller 100 in response to the same stimuli but in accordance with an alternate mode of operation.
• FIG. 7A indicates that the device was switched to the view navigation mode at time t1 140, and switched back to the fixed mode at time t4. During the fixed mode periods 144 and 146 of the drawing, the view remain fixed on the display during the corresponding periods 148 and 150 in FIG.
• a filtering algorithm cleans the orientation data from jerks and other orientation "noises". For example, while unintentional noise 160 and 162 in FIG. 7B results in corresponding orientation change pulses 164 and 166 in FIG. 7C, the program actually ignores them and does not navigate the view as shown in FIG. 7D and FIG. 7E.
• FIG. 7D illustrates the view navigation response when using a mode of operation that changes the navigated view only during a period of orientation changes.
• a mode of operation that changes the navigated view only during a period of orientation changes.
• an orientation change is detected and the view is slowly navigated in response.
• the view navigates at an accelerated rate 176.
• a relatively slow orientation change 182 results in a rapid navigation 184 of the view. In this operation mode the view remains stationary in the absence of orientation changes.
• FIG. 7E illustrates the view navigation response when an alternate continuous mode of operation keeps the view navigating at the rate and direction which was established during the last valid orientation change.
• an orientation change is detected and the view is slowly navigated 186 in response.
• the view continues to navigate in the same direction until a change in orientation occurs again in period 174.
• a new navigation rate 188 in the same direction but at a rate responsive to the reduced orientation rate change in period 174 is multiplied by the increase relative response at 154.
• This navigation continues until period 180, which occurs during the coarse navigation.
• a relatively slow orientation change 182 results in a rapid navigation 190 of the view.
• the program employs a minimum response threshold to allow the navigation to stop when the operator slightly reverses the direction of orientation.
• Other modes of operation can be established as variants of those shown in FIG. 7D and FIG. 7E.
• Such a response curve with at least two settings of fine and coarse navigation allows exact view navigation.
• Other response curves may be a fixed value, or may toggle from fine to coarse navigation at each subsequent entry to view navigation mode.
• Another solution is to eliminate the switches altogether and to monitor the pitch and roll changes for user specific gestures that will activate the view navigation mode.
• the program will keep an ongoing track of the roll and pitch orientation in storage and will analyze it continuously to see if the operator's gestures have been detected. It seems that for a more reliable activation of the view navigation mode, an additional sensor is required.
• FIG. 8 is another embodiment of the present invention that substitutes the switches 62 and 64 with an activation detector 200 that responds to a finger tap of the operator to set the hand-held device into view navigation mode.
• the finger tap detector includes a sound/vibration transducer 202 to sense the finger tap and to output a voltage that represents the sensed tap vibrations.
• the output of the sensor 202 is connected by wire 203 to the input of amplifier 204.
• the amplifier 204 is set up with a certain threshold to insure that only signals above the threshold are amplified.
• the amplified output of amplifier 204 is filtered by a low pass filter 206, whose output is connected to the analog-to-digital converter 208 to provide digital data to the microcontroller 100 via connection 212.
• the micro-controller uses the orientation change information from the orientation sensor 80 to navigate the display 12 in a similar way to the discussion of FIG.5.
• FIG. 9 is a perspective view of the hand-held device 10 in accordance with the embodiment of FIG. 8 showing a cutaway view of the finger tap sensor 202.
• the finger tap sensor 202 is attached to the bottom of the housing of the device 10, or it may be placed in the area just beneath the top side of the display 12 where the operator's finger is likely to tap while the device is held by one hand.
• the sensor 202 is connected by wire 203 to the circuitry described above. Proper amplification can insure that the device will detect all relevant finger taps at the bottom.
• Such finger taps produce vibrations and sound patterns that are significantly different from the sound that may be created by stylus strikes on the screen or the activation of a key 13.
• the micro-controller can distinguish a finger tap sound from other noises such as those created when the device is placed on the table.
• the finger tap sensor 202 can be surface mounted to the bottom of the PCB assembly 450 inside the device 10.
• FIG. 10 is the block diagram of the embodiment of the present invention in the form of an add-on upgrade 300 that is attached to an existing handheld device 10.
• the add-on upgrade 300 is built on a miniature PCB that is connected to the mobile device 10 externally or internally, depending on the expansion capability of said hand-held device.
• the add-on upgrade 300 comprises the sensors described in other embodiments of the present invention, as well as a standard micro-controller 302.
• a micro-controller typically includes its own stored program and data memory as well as a communication channel 304 like UART, SPC, and I2C.
• the microcontroller 302 receives the orientation changes information in lines 90 and 92 as well as the signal to enter or exit the view navigation mode 212. Processing the orientation change data and navigation entry command as explained above, the micro-controller computes the desired navigation changes for the device's display and provides commands to the hand-held device via the communication channel 304.
• the commands which might be in the form of a serial protocol, interface to the handheld device via the application interface port 310.
• the advantages of using the finger tap detector to command the entry to the view navigation mode is that it is less costly to manufacture than the side switches of FIG. 4. It is also easier to add a view navigation system with a finger tap sensor to an existing hand-held device.
• FIG. 1 1 shows yet another embodiment of the present invention that uses different sensors.
• the orientation sensor assembly 380 that includes a magnetic direction sensor 364 and a tilt sensor 368 replaces the accelerometer based orientation sensor 80 of FIG. 8.
• Various devices are commercially available for measuring the magnetic direction of the device. For example, magneto-resistors placed along axes 32 and 36 will produce orientation indications as the device is moved relative to the magnetic field of the earth.
• Tilt sensors may be comprised of a potentiometer and a small weight coupled to the potentiometer that is free to rotate on a fixed axis around which the tilt is measured.
• Other types of commercially available inclinometers are made of small, liquid-filled variable resistors that change their resistance based on their inclination.
• orientation sensors like laser gyros, may be used instead of the accelerometers 82 and 84.
• Yet another embodiment may employ several sets of magneto-resistors to measure orientation changes in two degrees of freedom without the use of an inclinometer or the accelerometers.
• the navigation command unit uses a speech recognition module 400 instead of the finger tap sensor 200 of FIG. 8 to command the micro-controller 100 to enter into view navigation mode.
• the speech recognition module may already be a part of the hand-held device 10 regardless of the view navigation feature, and as such can be used to allow the operator to enter the view navigation mode verbally, while still holding the device in one hand.
• FIG. 11 omits the display 12, the display controller 104 and the memory 102 which are still connected to the microcontroller 100 as shown in FIG. 8 and FIG. 5.
• the magnetic direction sensor 364 and tilt sensor 368 can replace the accelerometers 82 and 84 in FIG. 5.
• the speech recognition circuitry 400 may be used instead of the finger tap detector 200 of FIG. 8, or the accelerometers 82 and 84 of the embodiment of FIG. 8 can replace the orientation sensor of FIG. 1 1. Selection of the sensor should be made with a consideration of the overall structure of the hand-held device. In most applications, the devices that do not require moving parts and may be integrated into the silicon die of the main unit will most likely prove the most cost efficient.
• FIG. 12 is a timing chart illustrating how the view navigation mode is entered and exited in the embodiments of the present invention that use the finger tap sensor of FIG. 5.
• the signal received from the sensor 202 on line 203 is shown in FIG. 12A.
• FIG. 12B shows the signal that represents the change of orientation along one axis.
• the signal 240 representing a finger tap is detected between time t1 and t2.
• a sufficiently large time delay of t3-t2 is introduced by the micro-controller to eliminate any wrong orientation change readings that result from the finger tap motion.
• the orientation change signals 242 and 244 are likely an artifact of the tapping by the operator.
• the micro-controller sets the device to the view navigation mode and alerts the operator with the visual indication 54.
• FIG. 12D are logic signals indicating with logic "high" that the device is set to the view navigation mode.
• FIG. 12C illustrates a navigation exit method by which the device remains at the view navigation mode 254 for a preset time of t4-t3.
• the view navigation mode terminates to fix the displayed view. If more navigation is required, the operator repeats the process by tapping again at the bottom of the device.
• FIG. 12D illustrates another navigation exit method by which the device remains at the view navigation mode 256 as long as the operator changes the orientation of the device.
• the time period 246 the operator rotates the device in one direction, and during time period 248, the operator changes the direction of rotation.
• the time period 250 which is equal to t6-t5
• the micro-controller terminates the view navigation mode 246 at time t6.
• the display will not navigate in response to renewed orientation changes 252.
• the timing diagram of FIG. 12 can be used for other embodiments of the present invention.
• FIG. 13A outlines the software flow diagram for the embodiment of FIG. 12C that remains in view navigation mode only for a fixed time period and then exits to the fixed mode.
• the process starts at block 260 when the micro-controller 100 identifies the finger tapping or the voice command to enter the view navigation mode.
• the current boundary of the display view is marked in comparison to the stored virtual display.
• a delay corresponding to the time length t3- t2 of FIG. 12 is introduced at block 262 to allow the device some time to stabilize after the finger tap.
• the system is set to view navigation mode in block 268 to start the view navigation. It also provides the visual indication 54 to alert the operator that changes in orientation of the hand-held device will navigate the display.
• a variation of the process may activate the navigation indication 54 before block 264 but still enter the view navigation mode after the delay in block 264.
• a process timer is activated at block 268 to limit the stay at view navigation mode to a period of time equal to t4-t3.
• Block 270 monitors the process timer. If the process timer has not expired, the process continues to block 272 to acquire the pitch and roll orientation data, or in other embodiments of the present invention, the azimuth and inclination data. The process at block 272 also stores the orientation data for the next iteration, and compares the new and old orientation data. If a change in orientation is detected at block 274, the program computes the new boundary for the view at block 276.
• the micro-controller may interrupt the process to perform other tasks. Also, at the end of block 276 the micro-controller may allocate time through its operating system to other tasks, or even just wait for a preset time, to limit the number of iterations of the view navigation process in blocks 270 to 276 that are performed each second.
• a proper value of iterations per second depends on the selection of a coarse or fine view navigation mode as well as the response time of the orientation sensors, and it can be in the range of 5 to 30 for most practical applications.
• the advantage of a higher number of iterations per second is that it allows the navigation to match more closely the rotational movements of the hand, at the cost of an increased overhead on the micro-controller operating system.
• FIG. 13B outlines the software flow diagram for the embodiment of FIG. 12D that remains in view navigation mode as long as the operator continues to change the orientation of the device.
• the view navigation mode terminates.
• the process starts in blocks 260, 262 and 264 as described in FIG. 13A.
• the process provides the visual indication 54 to alert the operator that changes in orientation of the hand-held device will navigate the display.
• the acquisition of orientation data in block 284 is similar to the discussion for block 272 of FIG. 13A. If no change is detected in the orientation data at block 286, the process activates the no-action timer of block 290 to t6-t5. It then continues to acquire orientation data in block 292. If no change in the orientation data is detected at block 294, the process checks if the no- action timer has expired. If the timer expired, the view navigation mode ends in block 278 and 280 as described in FIG 13A.
• Block 286 If a change in orientation is detected at block 286, the program computes the new boundary for the view at block 288 and refreshes the display with the new view. It also saves the new current orientation as the basis for comparison in the next iteration of the process. Similarly, if block 294 detects a change in orientation data, it proceeds to block 288. Block 288 also deactivates the no-action timer of block 290 since a rotational action has been just detected.
• FIG. 14 illustrates a further embodiment of the present invention that uses an additional Z-axis accelerometer to identify the operator's vertical hand gestures to enter the view navigation mode and for other commands.
• the orientation and movement sub-section 420 includes a Z-axis accelerometer 432 set to sense movement at the vertical axis 40 of FIG. 2, in addition to the orientation sensors 82 and 84.
• the output of the accelerometer 420 is connected to the analog-to-digital converter 434.
• the analog-to-digital converter provides a digital signal to the microcontroller 100 that is responsive to the device's vertical movements.
• the program of the micro-controller monitors vertical acceleration changes to detect special hand gestures by the operator. For example, a quick up and down gesture of the operator may indicate a command to enter the view navigation mode.
• the program may be adapted to identify additional types of hand gestures as different commands. While special hand gestures may be identified by the set of two X and Y accelerometers, the program needs to isolate the hand gesture which is intended as a command from similar signals that are in response to operator navigation of the display.
• the Z-axis accelerometer 432 provides an extra measurement to allow the program to identify vertical movements from rotational navigation motion. For clarity purposes, FIG. 15 omits the display 12, the display controller 104 and the memory 102 which are still connected to the micro-controller 100 as shown in FIG. 8 and FIG. 5.
• sensors that provide elevation information can be used instead of the Z- axis accelerometer.
• This may include an air pressure sensor that is sensitive to changes in air pressure, or a distance detector that measures the distance of the device from the floor.
• FIG. 15 illustrates the placement of the three accelerometers of the embodiment of FIG. 14 on the main PCB 450 of the hand-held device, the PCB 450 carries the micro-controller 100, connection means 452 for the LCD display as well as other components, which are not shown for clarity. While many commercially available accelerometers incorporate two or more accelerometers on the same chip, the drawing illustrates the placement of the accelerometers assuming one accelerometer per chip.
• the X-axis accelerometer 82 and the Y-axis accelerometer 84 are placed at a right angle to each other.
• the Z-axis accelerometer 432 is mounted so that its sensitivity axis is perpendicular to the PCB 450. With high volume manufacturing, all three accelerometers may be incorporated in the same chip.
• FIG. 16 is a timing chart illustrating the use of the embodiment of the present invention of FIG. 14 with three accelerometers.
• FIG. 16A shows the acceleration signal received from the Z-axis accelerometer 432.
• FIG. 16B shows the signal that represents the change of orientation along the X-axis.
• FIG. 16C shows the signal that represents the change of orientation along the Y-axis.
• FIG 16D is a logic signal indicating with logic "high" when the device is set to the view navigation mode. We assume that the gesture to enter or exit view navigation mode is a rapid vertical up and down motion while the device is at a relatively stable orientation.
• the Z-axis accelerometer exhibits a strong signal 472 that corresponds to a vertical gesture by the operator between time t10 and t11.
• the other accelerometers show signals 474 and 476 since some orientation changes are affected even when the operator tries to keep the device at a relatively stable orientation.
• the micro-controller's program determines from this combination of signals that a valid operator gesture has been received to enter view navigation mode. After a preset time delay of t12-t11 the program sets the device to the view navigation mode 490 at time t12. The relatively small time delay insures that the program ignores artifacts 478 and 480 which tend to follow the entry gesture.
• the program acquires the orientation signals 482 and 484 to navigate the display. The program also keeps in storage a short trail of the accelerometer signals.
• the operator makes another gesture 486.
• the program completed the identification of the gesture and exits the view navigation mode.
• the program then returns the view to its setting just before the initiation of the gesture at time t14 using the stored trails of the accelerometer signals.
• the operator is therefore attempting to stabilize the view at time t13 so that the desired final view is present during the period 488 before the gesture to terminate the view navigation mode. If it is not desired to store the data trail of the accelerometers so that the view can be restored after the exit gesture, the exit method of FIG. 12C or FIG. 12D may be used.
• FIG. 17 outlines the software flow diagram for the embodiment of FIG. 14 that uses a vertical movement gesture of the operator to enter and exit view navigation mode.
• the process starts at block 500 in accordance with the overall tasking scheme of the operating systems of the micro-controller 100.
• the process acquires all sensor data at block 502 and keeps a data trail in memory to allow data shape analysis at block 504. If the analysis currently shows no gesture, the process terminates at block 506.
• block 504 determines that a gesture was made, the process continues with the initialization block 508 where the current boundary of the display view is marked in comparison to the stored virtual display.
• a delay corresponding to the time length t12-t11 of FIG. 16 is introduced at block 509 to allow the device some time to stabilize after the gesture.
• the system is set to view navigation mode in block 510 to start the view navigation. It also provides the visual indication 54 to alert the operator that changes in orientation of the hand-held device will navigate the display.
• the process acquires all sensor data at block 512 and keeps a data trail in memory to allow gesture shape analysis at block 514. If a change in orientation is detected at block 516, the program computes the new boundary for the view at block 518. It also refreshes the display to show the new view and it saves the new current orientation as the basis for comparison in the next iteration of the process. The process continues with the next iteration at block 512 to continue the navigation of the view. If no change in orientation was detected in block 516, the program continues to monitor the sensors at block 512. If a vertical gesture is detected in block 514, the program uses the data trail from the accelerometers to restore the view to its state just before the gesture 520. It then continues to block 522 to turn off the navigation indication 54 and the process ends at block 524.

Landscapes

• Engineering & Computer Science (AREA)
• Business, Economics & Management (AREA)
• Theoretical Computer Science (AREA)
• Computer Hardware Design (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Finance (AREA)
• Economics (AREA)
• Accounting & Taxation (AREA)
• Marketing (AREA)
• Development Economics (AREA)
• General Engineering & Computer Science (AREA)
• Human Computer Interaction (AREA)
• Strategic Management (AREA)
• General Business, Economics & Management (AREA)
• Human Resources & Organizations (AREA)
• Entrepreneurship & Innovation (AREA)
• Operations Research (AREA)
• Quality & Reliability (AREA)
• Tourism & Hospitality (AREA)
• Position Input By Displaying (AREA)
• User Interface Of Digital Computer (AREA)
• Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
• Navigation (AREA)
• Telephone Set Structure (AREA)

Applications Claiming Priority (3)

Publications (3)

Family

ID=24169004

Family Applications (1)

Country Status (6)

Cited By (2)

Families Citing this family (471)

Citations (3)

Family Cites Families (19)

• 2000
  • 2000-04-05 US US09/543,660 patent/US6466198B1/en not_active Expired - Lifetime
• 2001
  • 2001-04-04 DE DE60132201T patent/DE60132201T2/de not_active Expired - Lifetime
  • 2001-04-04 AT AT01928361T patent/ATE382889T1/de not_active IP Right Cessation
  • 2001-04-04 EP EP01928361A patent/EP1290672B1/de not_active Expired - Lifetime
  • 2001-04-04 WO PCT/US2001/010962 patent/WO2001078055A1/en active Search and Examination
• 2002
  • 2002-08-20 US US10/224,073 patent/US6933923B2/en not_active Expired - Lifetime
• 2003
  • 2003-09-10 HK HK03106445.5A patent/HK1054610A1/zh unknown

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events