WO2009114821A9

WO2009114821A9 - Apparatus and method of finger-motion based navigation using optical sensing

Info

Publication number: WO2009114821A9
Application number: PCT/US2009/037185
Authority: WO
Inventors: Brett Spurlock; Brian Todoroff; Yansun Xu; Jahja Trisnadi; Steven Sanders; Clinton Carlisle
Original assignee: Cypress Semiconductor
Priority date: 2008-03-14
Filing date: 2009-03-13
Publication date: 2010-02-18
Also published as: WO2009114821A3; CN102105895A; KR20110051165A; KR101612023B1; WO2009114821A2; CN102105895B

Abstract

One embodiment relates to an optical navigation apparatus. The apparatus includes a hole in a surface of the apparatus, a light source providing an illuminating beam through said hole, an imaging system configured to receive light generated by an illuminated portion of a finger placed above said hole and to produce an image from the light at a detector plane, and a tracking sensor array positioned at the detector plane that is configured to detect lateral movement of said finger relative to said hole. In addition, the apparatus includes a lift sensor positioned at the detector plane that is configured to detect lifting of said finger above said surface of the apparatus. Other embodiments, aspects and features are also disclosed.

Description

APPARATUS AND METHOD OF FINGER-MOTION BASED NAVIGATION USING OPTICAL SENSING

TECHNICAL FIELD

The present invention relates generally to computers and electronics and, more particularly, to navigation apparatus and methods for computer and electronic devices.

BACKGROUND OF THE INVENTION

Pointing devices, such as computer mice, trackballs, touchpads, pointing sticks (eraser nubs), joy sticks, and scroll wheels are well known for inputting data into and interfacing with personal computers and workstations. Such devices allow rapid relocation of a cursor on a monitor, and are useful in many text, database and graphical programs. A user controls the cursor, for example, by moving the mouse over a surface to move the cursor in a direction and over distance proportional to the movement of the mouse. Alternatively, movement of the hand over a stationary device, such as a touchpad, may be used for the same purpose.

SUMMARY

One embodiment relates to an optical navigation apparatus. The apparatus includes a hole in a surface of the apparatus, a light source providing an illuminating beam through said hole, an imaging system configured to receive light generated by an illuminated portion of a finger placed above said hole and to produce an image from the light at a detector plane, and a tracking sensor array positioned at the detector plane that is configured to detect lateral movement of said finger relative to said hole. In addition, the apparatus includes a lift sensor positioned at the detector plane that is configured to detect lifting of said finger above said surface of the apparatus.

Other embodiments, aspects and features are also disclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

These and various other features and advantages of the present invention may be apparent upon reading of the following detailed description in conjunction with the accompanying drawings and the appended claims provided below.

FIG. 1 is a schematic diagram of an apparatus for finger-motion based two- dimensional navigation using optical sensing in accordance with an embodiment of the invention.

FIG. 2 is a schematic diagram illustrates vertical finger motion over the hole in the device of FIG. 1.

FIG. 3 depicts a two-dimensional comb array for an optical sensor in accordance with an embodiment of the invention.

FIG. 4A is a schematic diagram of a tracking sensor array with elements electronically grouped as a bi-cell to detect lift in accordance with an embodiment of the invention.

FIG. 4B is a schematic diagram of a tracking sensor array with elements electronically grouped as a quad-cell to detect lift in accordance with another embodiment of the invention.

FIG. 5 is a schematic diagram of a tracking sensor array and an adjacent bi- cell lift sensor in accordance with another embodiment of the invention.

FIG. 6 is a flow chart of a method of finger-motion based navigation using an optical sensor in accordance with an embodiment of the invention. DETAILED DESCRIPTION

FIG. 1 is a schematic diagram of an apparatus 100 for finger-motion based two-dimensional (x-y dimensions) navigation using optical sensing in accordance with an embodiment of the invention. As shown, the apparatus may preferably be a portable or handheld device. For example, the apparatus may be a cellular phone, a personal digital assistant, a portable music player, a digital camera, a global positioning system (GPS) device, a laptop computer, a tablet computer, a game console, a remote control, or a combination of such devices. For such portable or handheld devices, a user's motion is typically limited.

Hence, a compact implementation of a user navigation system is highly desirable. The present application discloses a very compact user navigation system which uses optical sensing of finger motion. In other words, the present application discloses a technology that provides computer-mouse like functionality (and potentially further functionalities) through optical sensing of a user's finger movements.

The use of optical sensing to sense finger movement in the presently- disclosed apparatus contrasts with and has advantages over the use of capacitive sensing by touchpads. For example, the presently-disclosed apparatus has a much greater tracking resolution than a conventional touchpad. As shown in FIG. 1, a surface 101 of the device 100 has a small hole 102 configured therein. The hole 102 may have a diameter, for example, of less than one centimeter. The hole is preferably designed ergonomically to provide a comfortable interface with the user finger and to facilitate smooth motion during finger navigation. A user's finger 110 may be positioned on or above the hole 102. As disclosed herein, the finger motion 112 in the x-y plane (parallel to the device surface 101) may be used to generate a two-dimensional navigation signal. As further disclosed herein, the two-dimensional navigation signal may be advantageously generated by optical means.

In accordance with an embodiment of the invention, a laser and sensor package 150 may be configured beneath the hole 102 within the device 100. An expanded view of the laser and sensor package 150 is shown on the lower portion of FIG. 1. The laser and sensor package 150 may include a substrate 152 upon which a laser source 153 (for example, a vertical cavity surface emitting laser or VCSEL) and a sensor array and circuitry 154 are configured. The package 150 may also include integrated optics 156 including optics 157 configured to focus and direct the laser beam from the laser 153 to be outputted from said hole 102, and optics 158 configured to focus and direct scattered light from a user's finger 110 onto the sensor array 154.

In accordance with an embodiment of the invention, the scattered light produces a speckle pattern at the plane of the sensor array 154. The circuitry for the sensor array 154 may be preferably configured to implement a two-dimensional comb aσay for accurate tracking of horizontal (x-y dimensional) movement of the speckle pattern. Circuitry to implement a two-dimensional comb array is described further below in relation to FIG. 3.

Advantageously, very small finger movements may be tracked with this technology. For example, finger movements may be tracked at a resolution of greater than 3,000 dots per inch (dpi) using such a system. The actual resolution will depend on the implementation, including the wavelength of operation.

FIG. 2 is a schematic diagram illustrates vertical (z-dimension) finger motion 202 over the hole 102 in the device 100 of FIG. I. Besides the vertical (instead of horizontal) motion, FIG. 2 shows the same components as FlG. 1. The vertical finger motion 202 may also be called lifting motion or lift height changes.

A preferred embodiment of the invention includes a sensor array and circuitry 154 which is configured to detect not only two-dimensional horizontal finger motion 112 above the hole 102, but also to detect vertical finger motion 202 of a finger 110 above the hole 102.

In accordance with an embodiment of the invention, the circuitry for the sensor array 154 is configured to implement a two-dimensional comb array. A small-sized example of a two-dimensional (2D) comb array 302 of photodiode detector elements is shown in FIG. 3. The 2D comb array 302 is made up of 64 sub- arrays 304 organized in an 8-by-8 matrix. An expanded view of one such sub-array

304 is shown on the left side of the figure.

Each sub-array 304 comprises 16 detector elements organized in a 4-by-4 matrix. The 16 detector elements in each sub-array 304 are each identified as being a member of one of eight groups of elements. The group number associated with each detector element of each sub-array 304 is shown by the number (1, 2, 3, 4, 5, 6,

7, or 8) labeling the element in the expanded view. The signals from each group are electrically ganged together for the entire array 302. The resultant group signals (numbered 1 through 8) are output from the array 302 (as shown on the right side of the figure). Differential circuitry 306 is used to generate differential signals from pairs of the group signals. A first differential signal CC is generated by the difference of signals 1 and 2. A second differential signal SC is generated by the difference of signals 3 and 4. A third differential signal CS is generated by the difference of signals 5 and 6. A fourth differential signal SS is generated by the difference of signals 7 and 8. These four differential signals contain the information of the in- phase and quadrature signals in the x and y directions.

These in-phase and quadrature signals in the x and y directions may be utilized as two-dimensional motion signals to track horizontal movement of a speckle pattern. In other words, from the in-phase and quadrature signals in the x and y directions, horizontal motion 112 of a user's finger 110 above the hole 102 of the device 100 may be tracked.

FIG. 4A is a schematic diagram of a tracking sensor array 402 with elements electronically grouped as a bi-cell to detect vertical lift (in addition to detecting horizontal displacement) in accordance with an embodiment of the invention. In other words, in FIG. 4A, the tracking array 402 itself, besides having circuitry to detect lateral movements of the speckle pattern imaged from the surface, has additional circuitry to detect lifting of a user's finger 110 in relation to the hole 102 in the surface 101 of the portable device 100. In this case, for the purpose of lift detection, the signals from the various photo-detector elements of the array 402 are computationally grouped into two groups or cells: a left cell 402-L and a right cell

402-R.

In this configuration, a lifting motion of the user's finger 110 causes the beam centroid to move in a left-to-right (or right-to-left) direction. Three imaged beam positions are shown: a centered beam position 404 which is centered on the array 402; a right-shifted beam position 406; and a left-shifted beam position 408. When the beam is shifted to the right, the left cell 402-L detects less intensity compared to the right cell 402-R and hence the location of the beam centroid may be estimated as being positioned towards the right. When the beam is shifted to the left, the right cell 402-R detects less intensity compared to the left cell 402-L and hence the location of the beam centroid may be estimated as being positioned towards the left. In one implementation, the amount of lift Δz in the vertical direction may be estimated as the centroid shift Δx divided by the tangent of the incident angle θ (tan θ). FIG. 4B is a schematic diagram of a tracking sensor array 412 with elements electronically grouped as a quad-cell to detect vertical lift (in addition to detecting horizontal displacement) in accordance with another embodiment of the invention. In other words, in FIG. 4B, the tracking array 412 itself, besides having circuitry to detect lateral movements of the user's finger 110 relative to the hole 102, has additional circuitry configured to detect lifting of the user's finger 110 relative to the hole 102. In this case, for the purpose of lift detection, the signals from the various photo-detector elements of the array 412 are computationally grouped into four groups or cells: an upper left cell 412-A; an upper right cell 412-B; a lower right cell 412-C; and a lower left cell 412-D. In this configuration, lifting of the finger 110 relative to the hole 102 causes the beam centroid to move in a left-to-right (or right-to-left) direction. Three imaged beam positions are shown: a centered beam position 404 which is centered on the array 412; a right-shifted beam position 406; and a left-shifted beam position 408. When the beam is shifted to the right, the left cells 412-A and 412-D detect less intensity compared to the right cells 412-B and 412-C, and hence the location of the beam centroid may be estimated as being positioned towards the right. When the beam is shifted to the left, the right cells 412-B and 412-C detect less intensity compared to the left cells 412-A and 412-D, and hence the location of the beam centroid may be estimated as being positioned towards the left. In one implementation, the amount of lift Δz in the vertical direction may be estimated as the centroid shift Δx divided by the tangent of the incident angle θ (tan θ).

FIG. 5 is a schematic diagram of a tracking sensor array 502 and an adjacent bi-cell lift sensor 504 in accordance with another embodiment of the invention. Both the tracking sensor array 502 and the lift sensor 504 may be illuminated by the same light beam. Alternatively, separate illumination beams may be used.

Three imaged beam positions are shown: a centered beam position 404 which is centered on the array 412; a right-shifted beam position 406; and a left- shifted beam position 408. Here, as in FIGS. 4A and 4B, lifting of the finger 110 relative to the hole 102 causes the beam centroid to move in a left-to-right (or right- to-left) direction.

When the beam is shifted to the right, the left cell 502-L detects less intensity compared to the right cell 502-R and hence the location of the beam centroid may be estimated as being positioned towards the right. When the beam is shifted to the left, the right cell 502-R detects less intensity compared to the left cell 502-L and hence the location of the beam centroid may be estimated as being positioned towards the left. In one implementation, the amount of lift Δz in the vertical direction may be estimated as the centroid shift Δx divided by the tangent of the incident angle θ (tan θ).

FIG. 6 is a flow chart of a method 600 of finger-motion based navigation using an optical sensor in accordance with an embodiment of the invention. As described above, a coherent light beam (i.e. a laser beam) is generated and output 602 through a small hole in a surface of a portable electronic device.

The laser light is scattered 604 from a skin surface of a user's finger which may be positioned by the user in a vicinity above the small hole. The scattered light is detected 606 (after passing back through the hole) at a sensor device.

The array and circuitry for the sensor device are preferably configured to both track 608 a two-dimensional (x-y) horizontal displacement of the finger surface moving relative to the hole and, at the same time, detect 610 changes in lift height (z displacement) of the finger surface relative to the hole in the device surface. The 2D horizontal displacement signal may be used 612 as a user input signal for the portable device. For example, the 2D horizontal displacement signal may be used to control x or y scroll or to control cursor movement in two dimensions.

Similarly, the lift signal may be used 614 as a user input signal for the portable device. For example, the lift signal may be used as a click signal (similar to the press of a mouse button). In one specific embodiment, a rapid up-down finger motion may be used as an input signal which corresponds to a "left button click" on a mouse device, while two rapid up-down finger motions may be used as an input signal which corresponds to a "right button click" on a mouse device. In another specific embodiment, a small finger lift (for example, 2 to 4 millimeters above the surface) and motion in a single (for example, y) direction may be used as an input signal which corresponds to moving a scroll wheel.

In accordance with an embodiment of the invention, a lift height determined from the lift signal may be advantageously utilized to automatically scale 616 a resolution of the horizontal tracking. Decreasing a tracking resolution means that a same displacement of a detected speckle pattern would correspond to a smaller magnitude displacement of a cursor, for example. Conversely, increasing a tracking resolution means that a same displacement of a detected speckle pattern would correspond to a larger magnitude displacement of a cursor, for example. Generally, a higher lift height (i.e. a finger farther away from the hole) would be automatically scaled to a higher tracking resolution. Conversely, a lower lift height (i.e. a finger closer to the hole) would be automatically scaled to a lower tracking resolution.

The foregoing description of specific embodiments and examples of the invention have been presented for the purpose of illustration and description, and although the invention has been described and illustrated by certain of the preceding examples, it is not to be construed as being limited thereby. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and many modifications, improvements and variations within the scope of the invention are possible in light of the above teaching. It is intended that the scope of the invention encompass the generic area as herein disclosed, and by the claims appended hereto and their equivalents.

Claims

IN THE CLAIMS WHAT IS CLAIMED IS:

1. An optical navigation apparatus comprising: a hole in a surface of the apparatus; a light source providing an illuminating beam through said hole; an imaging system having an optical axis and configured to receive light generated by an illuminated portion of a finger placed above said hole and to produce an image from the light at a detector plane; a tracking sensor array positioned at the detector plane that is configured to detect lateral movement of said finger relative to said hole; and a lift sensor positioned at the detector plane that is configured to provide a lift signal which is representative of a lifting of said finger above said surface of the apparatus.

2. The apparatus of claim 1, wherein the tracking sensor array comprises a two-dimensional comb array detector.

3. The apparatus of claim 1, wherein the lift sensor comprises at least two groupings of photo-detecting elements in the tracking sensor array.

4. The apparatus of claim 1, wherein the lift sensor utilizes photo- detecting elements which are separate from photo-detecting elements of the tracking sensor array.

5. The apparatus of claim 1, wherein the lift signal is used to automatically scale a horizontal tracking resolution for the apparatus.

6. The apparatus of claim 5, wherein the horizontal tracking resolution is automatically increased for a higher lift height.

7. The apparatus of claim 1, wherein the apparatus comprises a portable music player device.

8. The apparatus of claim 1, wherein the apparatus comprises a cellular phone.

9. The apparatus of claim 1, wherein the apparatus comprises a portable global positioning system device.

10. The apparatus of claim 1, wherein the apparatus comprises a remote control device.

1 1. A method of finger-based navigation using optical sensing for a portable electronic device, the method comprising: providing an illuminating beam through a hole in a surface of the portable electronics device; receiving light generated by an illuminated portion of a finger placed above said hole; producing an image from the light at a sensor array positioned at a detector plane; tracking lateral movement of said finger relative to said hole using signals generated by the sensor array; and detecting lifting of said finger above said hole using signals generated by the sensor array.

12. The method of claim 1 1, wherein the sensor array comprises a two- dimensional comb array detector configured to track lateral movement of a speckle pattern.

13. The method of claim 12, wherein the sensor array comprises photo- detecting elements which are separate from the two-dimensional comb array detector and which are configured to detect shifts in a beam centroid for purposes of detecting lift.

14. The method of claim 11, wherein the sensor array comprises at least two groupings of elements configured to detect shifts in a beam centroid for purposes of detecting lift.

15. The method of claim 11, wherein a lift signal generated by the sensor array is used to automatically scale a horizontal tracking resolution for the apparatus.

16. The method of claim 15, wherein the horizontal tracking resolution is automatically increased for a higher lift height.

17. The method of claim 1 1, wherein the method is used to provide user input for a portable music player device.

18. The method of claim 1 1, wherein the method is used to provide user input for a cellular phone.

19. The method of claim 1 1 , wherein the method is used to provide user input for a portable global positioning system device.

20. The method of claim 1 1, wherein the method is used to provide user input for a remote control device.

21. A portable electronic device configured to provide finger-based navigation using optical sensing, the device comprising: means for providing an illuminating beam through a hole in a surface of the portable electronics device; means for receiving light generated by an illuminated portion of a finger placed above said hole; means for producing an image from the light at a sensor array positioned at a detector plane; means for tracking lateral movement of said finger relative to said hole using signals generated by the sensor array; and means for detecting lifting of said finger above said hole using signals generated by the sensor array.