WO2002050762A1

WO2002050762A1 - Virtual keyboard

Info

Publication number: WO2002050762A1
Application number: PCT/GB2001/005629
Authority: WO
Inventors: Alun Morris
Original assignee: Ubinetics Limited
Priority date: 2000-12-19
Filing date: 2001-12-18
Publication date: 2002-06-27
Also published as: AU2002222271A1; GB2370395A; GB0030976D0

Abstract

The invention provides a virtual keyboard that may be projected onto any suitable surface. The size of the keyboard, and the characters displayed thereon are selectable. A scanning light source (4) of a generating unit (2) is provided to generate the virtual keyboard (10) and the user places fingers on the projected keys (12) in the usual manner. The fingers interrupt light reflected back to the generating unit (2) and the position of the interrupted reflected light is sensed (14) and interpreted as being the particular key (12) that is touched in order to derive the text or symbols being typed.

Description

VIRTUAL KEYBOARD

This invention relates to display apparatus.

Although the present invention is concerned with the display of information in general, it is particularly applicable with reference to keyboards, which will now be discussed by way of example. Conventional keyboards of typewriters and personal computers are comparatively large objects, typically comprising five rows of keys, with each row containing approximately 13 keys. The size of each key is such as to allow convenient striking by the fingers of a human hand. The trend towards laptop computers, and even more particularly towards palmtops, has resulted in the size of the keys, and thus the keyboards, being significantly reduced. Keyboards in the form of keypads for telephones are also progressively being made available in reduced size, particularly in respect of mobile, or cell, phones. The reduction in size of such keyboards is effectively limited by the size of the human fingers, and with some devices currently available a significant degree of dexterity can be required in order to ensure that two, or more, keys are not inadvertently depressed at the same time. This problem is exacerbated when keys are required to perform more than one function, such as when the keypad of a mobile telephone is being used to transmit text messages, whereby the number of repeated depressions of a key corresponds to a different letter, for example.

It is one object of the present invention to overcome, or at least alleviate, disadvantages brought about by the reduction in the size of keyboards, or other displays of information, when the design of the operating device associated therewith becomes ever smaller.

In accordance with one aspect of the present invention, there is provided apparatus for interactively displaying an array of information, comprising: a light source; means for scanning the light source thereby to derive items of the information; means for projecting the information generated by the scanning means onto a, preferably substantially planar, surface; means for detecting light reflected from said surface and for sensing interruptions of the reflected light from portions of the surface corresponding to respective ones of said items of information; and means for generating an output in response to said interruptions.

The present invention, therefore, provides a projected display of information onto a surface, which other than providing legibility, requires no specific features other than being substantially flat. The surface may be for example a desk top or a book cover, and may be mechanically totally isolated from the apparatus that provides and projects the array of information.

Advantageously, the array of information comprises an optically-projected, substantially two-dimensional, keyboard, and the portions of the said surface may then comprise, or represent, respective keys thereof, with the items of information corresponding to letters or functions of a typical keyboard.

The scanning that is carried out in order to provide the displayed array of information is preferably carried out at such a rate that a complete set of items of information, for example a complete keyboard, is displayed as a substantially static display. That is to say, to the eye there is no impression of scanning or flickering. Thus, preferably, the scanning frequency is at least 25 Hz.

Since the array of information is generated optically, a plurality of sets of items of information can be selectively projected so as to display, for example, a variety of keyboard layouts and key symbols. The sets may comprise, for example, a plurality of languages and alphabets so that the displayed array of information (keyboard) may quickly be converted electro-optically from one to another.

The nature of the projection of the display also means that various degrees of magnification can be provided so suit the user or the flat surface available, and this selection may be made without any need to change the mechanical size or hardware of the displaying apparatus. For example, the size of the projected keyboard may be adjusted by changing an optical system or by software control of computerised display apparatus. The interactive display apparatus preferably itself comprises a conventional, small, display screen, such as that of a palmtop or mobile telephone, for example, and may display the output that is generated in response to interruption of the reflected light, for example representations of those "keys" that have been depressed. Thus as a human finger is placed over each projected item of information, for example the projected key of a keyboard, the interruption of the reflected light is sensed by the apparatus and the location of the interruption within the array is correlated with the associated item of information, for example, key of the keyboard, which can then be displayed on a display screen of the apparatus.

It will be appreciated that each item of information, as exemplified by a key of a keyboard, can in this way be arranged to exercise precisely the same function as if a key of a mechanical keyboard had been depressed by a human finger.

It will be appreciated, therefore, that the array of information as displayed by the present apparatus, in the preferred application of the invention, can be regarded as a virtual or zero-mass keyboard.

A plurality of sets of items of information may be arranged to provide a hierarchy of keyboards, and these can be provided in respect of languages with a very large number of symbols, for example Chinese or Japanese.

The apparatus of the invention may also be arranged to provide an audio feedback of the interruption of reflected light from a particular portion of the surface, corresponding, for example, to a particular key of a keyboard being depressed, to assist the typist.

The larger scale of the array of information thus provided with the present invention, and in particular with respect to a two-dimensional keyboard, allows typing to be carried out much faster and to be less error prone than is the case with many existing solutions, with mobile phones, for example, where the 26 letters A-Z are mapped onto only 9 keys. The number of keys that can be made available is also much greater than with conventional configurable touch-sensitive keyboards that are incorporated in smaller devices.

Advantageously, the light source is a laser or a focussed light beam, and this, preferably reflected by one or more mirrors, is then deflected under computer control to form the items of information successively.

The sensing of reflected light by the apparatus may be carried out by a light sensor that faces the projected array of information. Alternatively, a sensor may be incorporated into the projecting mechanism. In the latter case, the sensor may be arranged to detect light at or near a scanning light spot using the reverse optical path, that is to say along a path from the projected portion of the two-dimensional surface back along the beam to its projection from the apparatus. A half-silvered mirror (semi-transparent reflector) is provided in the optical path to split the reflected incoming light from the projecting beam. In an alternative arrangement, one or more sensors can be placed very close to the light source, thus avoiding the need for a half-silvered mirror. Lenses can then be arranged to focus light from the sensed spot onto the sensor.

The means for detecting the light reflected from the surface is advantageously arranged to compare the reflected light in successive scans that generate that information, thereby making the apparatus self-calibrating so as to take into account the ambient light and the reflective nature of the surface onto which the light is projected.

The resolution of the apparatus, that is to say the accuracy with which a particular item of information can be identified from the interruption of light reflected from the surface, can be enhanced by arranging the scanning such that it does not successively display items of information, for example keys of a keyboard, that lie adjacent to one another when projected onto the two-dimensional surface.

In a further embodiment of the invention, the accuracy of detecting which one of the items of information has been selected, which key of a keyboard has been struck for example, can be enhanced by providing a vibration sensor, for example a microphone, to detect the vibration of each finger striking the projected surface. Information from the vibration sensor can then correlated with the optical information to enhance the accuracy of identifying which item of information has been selected. Such a vibration sensor may optionally be auto-calibrated to the force of the fingers of the user.

Although it may generally be arranged that the surface onto which the array of information is projected is at a known angle with respect to the projecting apparatus, and is preferably orthogonal thereto, should this not be the case then geometrical correction may be applied using mirrors, formulae, and/or look up tables, which will allow the projected display of information to appear flat and linear. Deflection of the or each mirror can be effected by a magnetic coil, a piezo-electric transducer, or by a rotating mirrored polygon, or by a combination of these means. Such correction may be effected by adjustment of the components of the apparatus (hardware) or by suitably changing the software in the case of computer-controlled apparatus.

In accordance with another aspect of the present invention, there is provided a method of interactively displaying an array of information on a surface, wherein a light source is scanned so as successively to generate a plurality of items of information, the items of information are projected onto the surface, light reflected from portions of the surface corresponding to respective ones of said items of information is sensed, and an output is generated in response to interruption of the reflected light.

It will be appreciated that the method may be carried out using the apparatus of the said one aspect of the present invention.

Embodiments of apparatus, each in accordance with the present invention, will now be described, by way of example, with reference to the accompanying drawings in which:

Figure 1 is a diagrammatic representation of the apparatus and display produced thereby; and Figures 2 to 5 show various optical systems for use in the apparatus of Figure 1.

Referring to Figure 1, a generating unit 2 is arranged to project light, represented by a beam 4, through an aperture 6 thereof onto a planar surface 8. The unit 2 is arranged such that the light beam 4 scans the surface 8 and projects thereon a display in the form of a keyboard 10 having a plurality of virtual keys 12. The keys 12 may represent any of the keys of a keyboard, for example being a alpha-numeric character, symbol, word, or a function. In operation, the user contacts the keys 12 of the keyboard 10 in the same manner as is it were a real keyboard. The generating unit 2 is provided with a light sensor 14 that detects the light reflected thereon from the keyboard 10, and is arranged to determine the position of each key 12 that is pressed by the operator, by sensing changes in the light reflected therefrom. The generating unit 2 is also provided with a display 16 that, like the monitor associated with the keyboard of a computer, will display which of the keys 12 has been pressed by the operator. The display 16 of the generating unit 2 may also be arranged to display other information, under control of software contained therewithin.

By means of an optical system within the generating unit 2, or by means of appropriate software, the size, magnification, of the keyboard 10 can be adjusted to suit the user, and/or the surface 8 that is available to receive the projected light. Likewise, the unit 2 can be arranged to correct for any distorting effect of the projected keyboard should the surface 8 not be orthogonal to the generating unit 2.

The surface 8 onto which the light beam 4 is projected need not be provided as a dedicated planar surface, but may be, for example, a desktop or a book cover.

By use of suitable software within the generating unit 2, a variety of keyboard layouts and key symbols, for example using a plurality of languages and alphabets, may be projected from a single generating device 2.

The generating unit 2 may also have a microphone associated therewith that is arranged to detect sound from the surface 8 associated with a user pressing individual ones of the keys 12. The audio output from the microphone may be correlated with the output of the optical sensor 14, so as to enhance the resolution of the unit 2 in determining which one of the keys 12 has been pressed by the user.

One example of an optical system within the generating unit 2 will now be described with reference to Figure 2.

A semi-conductor laser 20 acting as the light source is located within a double convex lens 22 along the optical axis thereof. The laser beam is directed on to a first mirror 24 that is rotatable about a first axis, reflected therefrom onto a second mirror 26 that is rotatable about a second axes, and then emerges through the aperture 6 of the generating unit 2. The first and second axes are orthogonal to one another, so that the correct visual alignment of the keys 12 of the projected display 10 can be adjusted if necessary. The mirrors 24 and 26 may be moved about their axis by actuators, which may be magnetic (voice) coils piezo-electric transducers, or rotating mirrored polygons (or a combination of such actuators).

In the optical arrangement of Figure 2, a light sensor 28 of the generator unit 2 is placed in the optical path beyond the lens 22 such that light reflected back from the virtual keyboard 10 via the mirrors 26 and 24 is focussed thereon by the lens 22.

In a modification of the arrangement shown in Figure 2, the light sensor may be placed in front of the generating unit 2 in order to allow it to detect ambient light. This is, however, not a preferred arrangement, since it is not as accurate as the above method using a lens.

The embodiment of optical arrangement shown in Figure 3 is a simplification of that in Figure 2, in that only a single scanning mirror is employed. In this embodiment, a mirror 30 is hinged at a fixed point 32 and is rotatable thereabout under the action of two actuators 34 and 36 whose actuating rods are arranged to tilt the mirror 30 about respective hinges 38 and 40. This arrangement can be made very small, and consequently of low mass so that less force is required to deflect the laser beam at a given rate. The actuators 34 and 36 are electronically controlled and can be arranged to perform a geometric transformation if required, from Cartesian coordinates which would be used in the arrangement of Figure 2, to provide the non-orthorgonal deflections required of the actuators 34 and 36.

Should a greater deflection of the mirror of the optical system be required than can be achieved with the two actuators of Figure 3, then the arrangement of Figure 4 may be adopted in which a mirror 50 is hingedly deflected under the action of three actuators 52, 54, and 56.

Although in the embodiments of Figure 3 and 4 the actuators are arranged to cause deflection of the mirrors by a hinge interconnection, it is envisaged that flexible arms may be used instead. In the embodiment of Figure 5, a mirror 60 is connected to actuators 62, 64 and 66 by respective flexible arms 68, 70 and 72. It is envisaged that the arms 68, 70 and 72 will provide greater reliability and less mass than hinges, and thus be cheaper to manufacture.

Claims

1. Apparatus for interactively displaying an array of information, comprising: a light source; means for scanning the light source thereby to derive items of the information; means for projecting the information generated by the scanning means onto a surface; means for detecting light reflected from said surface and for sensing interruptions of the reflected light from portions of the surface corresponding to respective ones of said items of information; and means for generating an output in response to said interruptions.

2. Apparatus according to claim 1, wherein the array of information comprises an optically-projected keyboard, and wherein the portions of said surface represent respective keys thereof.

3. Apparatus according to claim 1 or claim 2, wherein each portion of the surface corresponds to a unique item of information.

4. Apparatus according to any one of the preceding claims, wherein the scanning means is arranged to project a complete set of said items of information onto said surface as a substantially static display.

5. Apparatus according to claim 4, wherein the scanning means is arranged selectively to project each one of a plurality of complete sets of items of information as a substantially static display.

6. Apparatus according to any one of the preceding claims, wherein the means for detecting light reflected from said surface compares the light reflected therefrom during successive scans that generate the information.

7. Apparatus according to any one of the preceding claims, wherein the light source comprises a laser, or a focussed beam of light.

8. A method of interactively displaying an array of information on a surface, wherein a light source is scanned so as to generate a plurality of items of information, the items of information are projected onto the surface, light reflected from portions of the surface corresponding to respective ones of said items of interruption is sensed, and an output is generated in response to interruption of the reflected light.

9. A method according to claim 8, wherein the items of information generated are projected onto the surface in the form of a keyboard.

10. A method according to claim 8 or claim 9, wherein the scanning of the light source is such as to display the information as a substantially static display.