EP0769176A1

EP0769176A1 - Apparatus and method for providing information to a control system or computer

Info

Publication number: EP0769176A1
Application number: EP95923468A
Authority: EP
Inventors: Christoph Dobrusskin
Original assignee: Central Research Laboratories Ltd
Current assignee: Central Research Laboratories Ltd
Priority date: 1994-07-01
Filing date: 1995-06-29
Publication date: 1997-04-23
Also published as: WO1996001455A1; GB9413214D0

Abstract

Apparatus for providing information to a computer comprises a display (51) producing a viewable image (14) suspended in space, and a detector having two photosensitive element arrays (53, 54) for sensing the presence and/or position of an object (e.g. a finger) (55) in a given space close to the image, this information being fed to the computer. The elements are sensitive to infrared radiation which is either emitted passively or is scattered and/or reflected via the object from a source (58) of infrared radiation which illuminates the space. The detector elements and source are arranged so that light does not illuminate the detector elements directly. The detector element arrays identify which possible subregion of the given space the infrared signal associated with the object emanates. The invention enables interaction with a suspended image or a virtual reality system without surrounding the space with light sources and/or detectors.

Description

APPARATUS AND METHOD FOR PROVIDING INFORMATION TO A

CONTROL SYSTEM OR COMPUTER

This invention relates to apparatus for providing information to a control system or computer, the apparatus comprising:- a) display means for producing a viewable image, and b) sensing means for sensing the presence and/or position of an object in a given region of space in the vicinity of the viewable image, this information for communication to the control system or computer, the sensing means having at least one detector element sensitive to electromagnetic radiation. The invention also relates to a method for providing information to a control system or a computer.

In a known apparatus disclosed in US 4,198,623 a grid of horizontal and vertical infra-red light beams is provided in front of a cathode ray tube display. A finger or other object may then be placed adjacent the display to mark a position thereon, and this event and its position is detected by the interruption of one or more infra-red light beams. Infra-red sources and detectors are located around the periphery of the CRT screen and operate in different planes to more closely follow the curvature of the CRT screen to minimize parallax errors. This apparatus has the disadvantage that a large number of sources are required, and sources and detectors have to be carefully pre-aligned. It is also designed to work against a substantially flat screen, and is difficult to fabricate to give full 3 dimensional sensing of the position of the finger in a large space as would be required, for example, for interaction with a real image suspended in space or in a virtual-reality data input device.

According to a first aspect of the invention there is provided apparatus as defined in the first paragraph above characterized in that the display means is arranged to produce a real image being suspended in space, and the sensing means senses from which of a plurality of sub-regions of the given region of space electromagnetic radiation, being emitted by or reflected from the object, emanates. This arrangement makes interaction with a 3 dimensional and/or suspended real image more convenient and intuitive. The apparatus preferably includes a source of electromagnetic radiation for scattering and/or reflection by the object, the source being arranged such that electromagnetic radiation emitted therefrom will not impinge directly upon any said detector element sensitive to such radiation, the source being further arranged to sequentially illuminate successive sub-regions of the given region of space. This feature can make direct alignment of detectors and sources, and expensive two dimensional detector arrays unnecessary. The sensing means may advantageously comprise a plurality of arrays of detector elements sensitive to electromagnetic radiation. This can enable the sensing of the object in a three dimensional space without the requirement of sensors located round the periphery of the space. The electromagnetic radiation emanating from the object can conveniently have been scattered and/or reflected from the display means. This eliminates the necessity for additional sources of electromagnetic radiation and can reduce power consumption of the apparatus.

The electromagnetic radiation may be modulated. This can give the advantage of making the signal easier to extract from background noise caused by ambient lighting which may be constant or modulated in a different manner.

According to a second aspect of the invention there is provided a method of providing information to a control system or computer having a display, comprising the steps of a) placing an object into a given region of space (having a plurality of sub- regions) in the vicinity of a real viewable image from the display being suspended in space, b) sensing into which sub-region the object has been placed by means of electromagnetic radiation scattered and/or reflected by the presence of the object or emitted by the object, and c) communicating this information to the control system or computer. This method has the advantage that it can enable interaction between, for example, a human or animal and a real suspended image or virtual reality apparatus without the necessity of electrical connections or cabling.

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings, in which Figure 1 is a perspective view of an apparatus for providing a real image suspended in space,

Figures 2 and 3 are cross-sectional side views of the apparatus of Figure 1, Figure 4 shows a first embodiment of the apparatus according to the invention, Figure 5 shows a second embodiment of the apparatus according to the invention, and

Figure 6 shows a flow diagram of a method according to the invention. In the above figures like integers are assigned the same number whenever possible in subsequent drawings.

Before describing the invention in detail, an apparatus for forming a real image suspended in space for use with the invention will be described. Referring to Figures 1 and 2, beamsplitter 4 is mounted for example in an aperture in an opaque surface 2. In this embodiment, retroreflector means comprising an array of retroreflectors, such as a sheet 6 of corner-cubes, is positioned in a path for light which has been reflected by the beamsplitter 4. In this example it is positioned at an acute angle thereto, which is about 45°, so as to form the second side of a triangle therewith. An object 8 as a picture displayed on a VDU screen is positioned along the third side of the triangle, at approximately 90° to the beamsplitter. For example, the screen 8 may be positioned at an angle in the range from 70 to 110° to the beamsplitter 4.

In one specific arrangement, means for screening the retroreflector 6 from the object 8 is provided, such as a louvre film 12 in front of the object 8. As indicated by arrows in figure 2, the beamsplitter 4 reflects light from the object 8 towards the retroreflector 6. In addition some of the light is transmitted by the beamsplitter 4 and is "lost". The beamsplitter 4 also transmits light reflected from the retroreflector 6 to form the suspended image 14 opposing the object 8 with respect to the beamsplitter 4. In addition, some of the light is reflected by the beamsplitter back towards the original image 8. In the apparatus shown in Figure 3, a further array of retroreflectors 16 is positioned in a path for light which has been transmitted by the beamsplitter. In this example, the further array opposes the retroreflector 6 with respect to the beamsplitter 4; that is, it forms a triangular arrangement with the beamsplitter 4 and the suspended image 14. The further array of retroreflectors 16 directs light which is transmitted by the beamsplitter 4 directly from the original image 8 back towards the beamsplitter 4 to prevent 'loss' of this light. In an alternative arrangement, the array of retroreflectors 6 in a path for light which is reflected by the beamsplitter 4 could be omitted, leaving only the array in the path of transmitted light.

In the example shown if Figure 3, the object 8 is a projection screen, and an image is formed thereon by a projector 18. The projector 18 is shown positioned in the gap 10 between the retroreflector array 6 and the object 8, but may have other positions. For example it may be positioned in an aperture in the retroreflector array 6.

As indicated by plus and minus signs adjacent arrows in the Figure, the beamsplitter may be polarization selective, transmitting light having one type of circular polarization and reflecting light having the other type. For example, it may comprise layers of cholesteric liquid crystal polymers reflecting different spectral regions of circularly polarized light of one sense so as to reflect white light (i.e. the whole visible spectrum) of this sense.

Additionally, the object may be arranged to emit circularly polarized light, in which case one of the arrays 6, 16 may be omitted, since all of the light from the object could be of the type which is either reflected or is transmitted by the selective beamsplitter. This could be achieved by using a transmissive or reflective projection screen which either preserves or reverses polarization, and a projector which produces linearly polarized light, such as an LCD projector, combined with a quarter wave plate. A transmissive such screen is disclosed in GB 1509180.

An apparatus for producing a suspended image 14 in an image plane has been described above. Figures 4 and 5 show such an image together with an input device according to the present invention.

In Figure 4, apparatus for providing information to a control system or computer comprises display means (51) as described above for producing a viewable image (14), and sensing means (53, 54) for sensing the presence and/or position of an object (55) in a given region of space (56) in the vicinity of the viewable image, this information for communication to the control system or computer (57), the sensing means having at least one detector element sensitive to electromagnetic radiation. The sensing means (53, 54) senses from which of a plurality of sub-regions of the given region of space electromagnetic radiation, emanating from the object, is emitted. In this embodiment, the sensing means comprises two 2-dimensional photosensitive element arrays (53, 54) for sensing the presence and/or position of an object (e.g. a finger) in a given space close to the image, this information being fed to the computer. The elements are sensitive to infra-red radiation which is scattered and/or reflected via the object (55) from a source (58) of infra-red radiation which illuminates the space. The detector elements and source are arranged so that infra-red light does not illuminate the detector elements directly. The detector element arrays identify which possible sub-region of the given space the infra-red signal associated with the object emanates. The sensing means may be made as described in US patent number 5,196,900, which is incorporated herein by reference. The display means (51) in the present embodiment comprises apparatus as described above and shown in Figures 1 and 2 which display means projects a real image (14) suspended in space (56) above the display means. The display means includes a transparent sheet (2) which isolates the light sources from the space in which the image is formed. The image is formed by visible light which passes through the transparent sheet from the display means. In the present embodiment, electromagnetic radiation in the form of infra-red rays scattered by the object of interest (55) from the source (58) passes through the transparent sheet to be detected by the photosensitive element arrays (53, 54). These arrays each have an optical system (60), in the present embodiment comprising a converging lens which makes each detector element sensitive to light incident from a narrow range of solid angles. The intersection of these solid angles from each of the two detector element arrays define a lattice of sub- regions of the given space in the vicinity of the image (14). Each sub-region excites a different pair of detector elements, making the position of the object (55) in space correspond to a unique sub-region identified by the pair of addresses of the excited elements in the two arrays (53, 54).

Although in this embodiment two separate infra-red sources are used to illuminate the given space, a single source may be used as an alternative. As a further alternative, no infra-red sources are present and the detector element arrays detect the infra-red radiation emitted passively by the object (such as for example a part of the human or animal body). The infra-red source can be modulated at a given frequency so as to enable the detector elements to discriminate against ambient infra-red light. Although in the above embodiment the optical system (60) is separate from the two arrays, the optical systems may be made integral with the arrays if desired.

In another alternative, the light emanating from the display means to form the image may be scattered and/or reflected by the object when it intersects or penetrates the image, this light being detected by the sensing means. Because this light is likely to be visible Ught, ambient light present close to the real image may result in interference when using the last alternative. This disadvantage can be mitigated by causing the intensity of the light source to fluctuate periodically. In order to make this less disturbing for the viewer, it is advantageous if the fluctuations occur at a frequency higher than a critical fusion frequency dependent upon local illumination conditions. Although in the above examples two two-dimensional arrays of detector elements have been used, two one-dimensional arrays at an oblique angle to one another can be used, as can a one-dimensional array together with a two-dimensional array to provide sensing of the object in a more restricted given space or with reduced precision in one direction. It is preferable to use one-dimensional arrays in which the detector elements have acceptance angles which are wide in a direction orthogonal to the line of array elements to give adequate position triangulation.

In a second embodiment, shown schematically in Figure 5, the sensing means comprises a single photosensitive element (64) which can sense the presence of incident electromagnetic radiation over a large acceptance angle, together with means to sequentially illuminate sub-regions of the given space. In this embodiment the means comprise two one-dimensional LED arrays (65, 66) coupled to respective converging lenses (67, 68). Each LED in (65) is illuminated in turn for a given time whilst the photosensitive element is ready to receive a scattered or reflected signal from the object (55). Each LED is controlled to emit light in a specific time interval so that the output from the photosensitive element may be identified with a specific LED. When all the LEDs in 65 have been illuminated, the LEDs in 66 are illuminated one at a time in turn as before. Once again each LED may be assigned a specific time so that a signal from the photosensitive element may be unambiguously be identified. If no signals are present the procedure starts over again and no data need be communicated to the computer. If an object is present, two signals at respective times will be generated. These signals wUl have a one-to-one correspondence with one LED in 65 and one LED in 66, and triangulation techniques may be employed to identify in which sub-region of space the object was present. This information is then communicated to the computer. As an alternative the computer may perform the triangulation analysis itself from the detected information - in this case the identity of the relevant LEDs.

Although in this embodiment two LED arrays have been used, any method of generating two sets of Uluminated stripes which intersect may be used, for example two sets of raster scanned laser beams.

As a further alternative, a two dimensional array of detector elements together with a raster scanned laser beam may be employed to determine the position of any object in a given sub-region of space. As an alternative to the above Unear light source, point Ught sources may be used. In order to minimize the number of such light sources, one may share each lens in the above example, but shine Ught perpendicular to the arrays. In this case a beam spUtter is used to deflect the Ught through the lens without illuminating the arrays directly. Using this arrangement electromagnetic radiation reflected directly back towards its source may be partially transmitted towards the detector elements by the beam-spUtter.

The intensity of the VDU (or source of Ught for iUuminating the object 8 such as a fluorescent lamp) in the examples above fluctuates at a frequency of approximately 100 Hz. This regular fluctuation may be used to discriminate between stray illumination to give a better signal to noise ratio for the Ught signals scattered by the presence of a finger, provided that the ambient Ught fluctuates at a different frequency or does not fluctuate. Non-regular pulsed fluctuations may be used provided there is means to identify when the scattered signal is due. Other methods of modulating the electromagnetic radiation in the above examples may be used - specifically modulating the intensity, frequency, polarization state or phase relationship. An object comprising a retroreflective portion at the end of a wand may be used to increase the signal reflected towards the detector elements if they are close to the optical position of the source.

A flow diagram of a method according to a second aspect of the invention is shown in Figure 7. In this diagram the numbered blocks correspond to steps in the method, and the blocks have the foUowing significances:- Block 40 denotes placing an object into a given region of space (having a pluraUty of sub-regions) in the vicinity of the suspended image from the display,

Block 41 denotes sensing into which sub-region the object has been placed by means of electromagnetic radiation scattered and/or reflected by the presence of the object or emitted by the object, and

Block 42 denotes communicating this information to the control system or computer.

Claims

1. Apparatus for providing information to a control system or computer comprising:- a) display means for producing a viewable image, and b) sensing means for sensing the presence and/or position of an object in a given region of space in the vicinity of the viewable image, this information for communication to the control system or computer, the sensing means having at least one detector element sensitive to electromagnetic radiation, characterized in that the display means is arranged to produce a real image being suspended in space, and the sensing means senses from which of a plurality of sub-regions of the given region of space electromagnetic radiation, being emitted by or reflected from the object, emanates.

2. Apparatus as claimed in claim 1 in which the sensing means further comprises a source of electromagnetic radiation for scattering and/or reflection by the presence of the object, the source being arranged such that electromagnetic radiation emitted therefrom wiU not impinge directly upon said detector element sensitive to such radiation, the source being further arranged to sequentially iUuminate successive sub-regions of the given region of space.

3. Apparatus as claimed in claim 1, the sensing means comprising a plurality of arrays of detector elements sensitive to electromagnetic radiation.

4. Apparatus as claimed in claim 3, the sensing means further comprising a source of electromagnetic radiation for scattering and/or reflection by the presence of the object, the source being arranged such that electromagnetic radiation emitted therefrom wtfl not impinge directly upon the detector elements sensitive to such radiation.

5. Apparatus as claimed in claim 3 in which the electromagnetic radiation sensed by the sensing means is emitted by the display means and scattered and/or reflected by the presence of the object.

6. Apparatus as claimed in claim 4 in which the electromagnetic radiation is modulated.

7. Apparatus as claimed in claim 1 in which the object comprises a source of electromagnetic radiation.

8. Apparatus as claimed in claim 3 in which at least one of the arrays is a two- dimensional array.

9. Apparatus as claimed in any preceding claim in which the object is a part of the human body.

10. A method of providing information to a control system or computer having a display, comprising the steps of a) placing an object into a given region of space (having a plurality of sub-regions) in the vicinity of a real viewable image from the display being suspended in space, b) sensing into which sub-region the object has been placed by means of electromagnetic radiation scattered and/or reflected by the presence of the object or emitted by the object, and c) communicating this information to the control system or computer.