WO1990010882A1

WO1990010882A1 - Image display system with position indicator

Info

Publication number: WO1990010882A1
Application number: PCT/US1990/001277
Authority: WO
Inventors: Alexander Schure; William E. Glenn
Original assignee: New York Institute Of Technology
Priority date: 1989-03-08
Filing date: 1990-03-07
Publication date: 1990-09-20

Abstract

An apparatus is disclosed for displaying an image and also detecting positional indications on the displayed image, typically a viewer pointing with a stylus (130) to a position or region on the displayed image. A fiber optical image magnifier (100) is provided, and has a multiplicity of fibers arranged with respective ones of their ends spaced relatively closely together at an image input port (20) and the respective others of their ends spaced relatively further apart at an image display port (40). The input image is received at the image input port (20). The input image may be from any desired source of still or moving images. A stylus (130) is movable over the image display port, and includes a light source (131) directed at said image display port (40). An optical sensor array (140) receives, from the input port, light from the position indicating means which has traveled from the image display port (40) to the input port (20), and is used in generating a signal indicative of the position on the input port from which the detected light emanates. Among the advantages of the apparatus are a cable-free stylus and a position detection function which need not be synchronized with the video scan being used for the image display.

Description

IMAGE DISPLAY SYSTEM WITH POSITION INDICATOR

FIELD OF THE INVENTION

This invention relates to the field of image display and, more particularly, to an apparatus for displaying an image and detecting positional indications on the displayed image.

BACKGROUND OF THE INVENTION

There are various existing techniques whereby a viewer can "point" to a particular position on a display screen to select an element or region on the displayed image. For example, a so-called "light pen" is commonly employed in conjunction with a cathode ray tube display, such as the monitor of a computer system, to indicate the viewer's selection of an element or region on the displayed image. This information can be used, for example, to select from a menu or to draw or "paint" on an existing image. A light pen typically includes an optical sensor, and the position of the pen is determined from the time at which light from the display is detected by the light pen. The timing of the cathode ray tube scanning pattern is related to position on the screen, so an output signal representative of position can be generated from the light pen sensor. Other techniques are also known for determining the position of a stylus or other pointer on a display screen area, for example, digitizers which determine the stylus position using the propagation of energy, such as electromagnetic or acoustic energy, between the stylus and known locations.

In the U.S. Patent No.s 4,116,739 and 4,208,096, assigned to the same assignee as the present application, there is disclosed a device for magnifying an image which employs a multiplicity of fiber optical strands. Other fiber optical image magnifiers are also known in the art. It is among the objects of the present invention to provide an apparatus which utilizes a fiber optical image magnifier, and also includes an improved technique for detecting positional indications on the display screen of such a device.

SUMMARY OF THE INVENTION

The present invention is directed to an improved apparatus for displaying an image and also detecting positional indications on the displayed image, typically a viewer pointing with a stylus to a position or region on the displayed image. A fiber optical image magnifier is provided, and has a multiplicity of fibers arranged with respective ones of their ends spaced relatively closely together at an image input port and the respective others of their ends spaced relatively further apart at an image display port. The input image is received at said image input port. The input image may be from any desired source of still or moving images. A position indicating means, which will typically, although not necessarily, be in the form of a stylus, is movable over the image display port, and includes a light source directed at said image display port. The light may be any suitable optical radiation that can be carried by at least some of the fibers of the fiber optical image magnifier. An optical detection means is provided for receiving, from the input port, light from the position indicating means which has traveled from the image display port to the input port. The optical detection means, which may comprise for example an image sensing array such as a charge coupled device ("CCD") array, operates to generate a signal indicative of the position on the input port from which the detected light emanates, and thereby indicative of the position of the position indicating means over the image display port.

In the preferred embodiment of the invention, optical separating means are provided for optically coupling the input image to the input image port and for also optically coupling light from the input image port to the optical detection means. In a form of this embodiment, a dichroic mirror is used for this purpose, and it reflects the image to be displayed while transmitting, in the opposite direction, infrared light to be detected. If desired, a plurality of inputs can be employed in conjunction with either a single optical detector array or a plurality of det.ector arrays.

The present invention has the advantage of not requiring a cable or other communication between the stylus and the position determining circuitry, as in a conventional light pen system, and this enhances the convenience of operation. Also, the position detection function can be unrelated to the generation of the displayed image and, for example, is not tied to the scanning rate of the displayed image. This asynchronous operation provides greater flexibility in the position determining function.

The system hereof is also particularly advantageous for use in a system wherein the viewer is utilizing certain regions or groups of fiber strands to convey special information, as can be employed for image generation or modification.

Further features and advantages of the invention will become more readily apparent from the following detailed description when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a diagram, partially in schematic form and partially in block form, of an apparatus in accordance with an_.embodiment of the invention.

Fig. 2 illustrates an image magnifier utilized in the Fig. 1 embodiment.

Fig. 3 is a block diagram of a processor subsystem utilized in the Fig. 1 embodiment.

Fig. 4 is a flow diagram of a routine for controlling the processor subsystem of the Fig. 1 embodiment in accordance with a form of the invention.

Fig. 5 is a diagram, partially in schematic form and partially in block form, of an apparatus in accordance with another embodiment of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to Fig. 1, there is shown a diagram of an apparatus in accordance with an embodiment of the invention. A fάber optical image magnifier device 100 is provided, and may be of the type described in the abovereferenced U.S. Patent No.s 4,116,739 and 4,208,096. The image magnifier 100 utilized in this embodiment is illustrated in Fig. 2. The magnifier includes an image input port 20 defined by one end 31 of each of a multiplicity of elongated fiber optical strands (e.g. shown at 30), the ends 31 being arranged in a relatively closely spaced array of rows and columns. An image display port 40, substantially perpendicular to the image input port 20, is defined by the other ends 41 of the fiber optic strands 30. The ends 41 are seen to be arranged in a relatively remotely spaced array of rows and columns.

Referring again to Fig. 1, an input image source 110 is provided, the input image preferably being a small relatively bright image, as can be obtained, for example, utilizing a solid state light modulator device of the type described in U.S. Patent No. 3,882,271, and driven by a video signal. As described in said patent, a charge pattern on a semiconductor device is utilized to cause deformations of a deformable surface that are converted into an image by an optical system. However, any suitable image source of a still or moving image, for example a small cathode ray device, could alternatively be employed. The input image is coupled to the input port 20 of magnifier device 100 via a dichroic mirror 120 which, in the present embodiment, reflects light wavelengths in the visible range.

A position indicating stylus 130 is provided. The stylus includes a light source 131 which, in a form of the present embodiment, is in the infrared region of the optical spectrum. The source 131 may be, for example, a light- emitting diode which emits in the infrared. If desired, a lens or other light focusing or directing means can be used to determine the size and shape of the light spot on the image display port. In the present embodiment, the stylus is powered by a battery 132, so the stylus is advantageously not. encumbered by a cable. The stylus is also provided with a suitable operator-controllable switch, such as a microswitch 135 in the stylus tip. When depressed, the switch 135 operates to enable the energizing of the light source.

In the present embodiment, the dichroic mirror 120 is adapted to pass infrared energy received from the input port 20 of the magnifying device 100, and this infrared energy is focused on a two-dimensional detector array 140 by a lens 125. The detector array may be, for example, a CCD type of detector array with sensitivity in the infrared. The array 140 is coupled with processor subsystem 300.

In operation, the input image, which is passed to the image input port 20 of the magnifying device 100 via dichroic mirror 120, will be displayed at the image display port 40 thereof. If the image magnifier is of the type that displays with inherent mirror image reversal (as does the magnifier device described in U.S. Patent No.s 4,116,739 and 4,173,391), the reversal from reflection off mirror 120 can be used to account for the magnifier reversal. Alternatively, and if necessary, any suitable optical or electronic reversal (e.g. in generation of the input image) can be utilized. The infrared radiation from stylus 130 will enter a fiber, or fibers within a small region, on the image display port 40 of the image magnifying device 100 and travel "backwards" through the fiber(s) to be emitted at the input port 20 thereof. The infrared light will pass through dichroic mirror 120 and be focused by lens 125 onto array 140. (Again, any mirror image reversal inherent in the magnifier can be accommodated either optically or in the readout of the sensing array. ) The pixel or region on the array at which the infrared light is sensed is provided as an output signal by processor subsystem 300. This signal is indicative of the position on the image display port to which the stylus is pointing.

Fig. 3 shows a block diagram of the processor subsystem 300 of Fig. 1. A microprocessor 310 is provided, in conjunction with conventional memory 320, clocking 330, and input/output 340 capabilities. The processor and associated components may be, for example, resident in a personal computer that is conventionally provided with a keyboard and display, and, if desired, can also control generation of all or part of the image for ultimate display on the image display port of magnifier device 100. It will be understood that any suitable digital or analog processing circuitry or special purpose circuitry can be employed. The processor subsystem receives the output of image sensing array 140 via an analog-to-digital converter 350. The image sensing array receives controlling clock signals from the processor subsystem, either via input/output 340, or directly. The output position indication signal, as noted, can drive any suitable indicator or display, or can be used for control purposes.

Referring to Fig. 4, there is shown a flow diagram of a routine for controlling the processor subsystem in accordance with an embodiment of the invention. In this embodiment, it is assumed that the processor subsystem will be continuously looking for a stylus position and outputting the stylus position that it detects. However, it will be understood that other arrangements can be utilized, for example when a stylus switching output (e.g. via a cable or transmitter, if employed) is utilized to gate the position determination function. The block 411 represents the resetting of the position counter which, in the present embodiment, counts in a raster pattern fashion by counting an array position (or pixel) for each array element on the first row of the sensor array, and then continuing with elements of the second row of the array, etc.. Thus, if the array has M elements per row, and N rows, there will be M x N counts, or addresses. A "largest value" storage register is also reset at zero. The block 415 is then entered, this block representing the clocking out of the next pixel value from the array, and the block 420 then represents the reading of the next value. Hit will be understood that these are not necessarily the same pixel value, as there may, for example, be a delay between the pixel value being clocked out and one being read in.] Also, if desired, the clocking out of array values, into known storage locations, can be asynchronous with the reading of the values. Inquiry is then made (diamond 425) as to whether the current pixel value (i.e., the digital value output of analog-to-digital converter 350 that is representative of the amplitude for the current pixel) is larger than the largest pixel value stored so far for this frame. If so, the pixel value and the associated address are stored in the largest value register, as represented by the block 430. The diamond 450 is then entered (and is also entered directly from the "no" output branch of diamond 425), and inquiry is made as to whether the last pixel of the array has been interrogated. If not, the block 415 is reentered, and the loop 460 continues until all sensor array pixel values have been considered. The diamond 470 is then entered, this block representing the comparison of the largest stored pixel value against a predetermined minimum threshold. If the minimum threshold is not met, the sensed largest value is probably noise, and the block 475 is entered, this block representing the generation of an indication that the stylus is not active. If the threshold condition is met, however, the block 480 is entered, this block representing the readout of the pixel location (i.e., the address of the largest pixel value) which represents the stylus position. If desired, the pixel value can also be read out. Also, if desired, a routine can be applied for determining a region of pixel locations at which sufficient sensed signal is present.

As seen in the embodiment of Fig. 5, a plurality of positional inputs can be employed. In Fig. 5 a second stylus 530 is shown, although other types of inputs can be provided. In this case, the routine of Fig. 4 can be modified to output the two stylus positions as the two largest detected values in the frame. If it is necessary or desirable to distinguish between the two positional inputs, various techniques can be utilized for this purpose. For example, different light intensities can be utilized for the light sources, or different radiation wavelengths can be employed in conjunction with different filters that are associated with a plurality of detectors or are switched in front of a single detector.

The invention has been described with reference to a particular preferred embodiment, but variations within the spirit and scope of the invention will occur to those skilled in the art. For example, the use of an infrared light source in the stylus is predicated on the ability of some or all of the fibers in the magnifying device to carry the infrared light. For an image generated using an RBG system, a visible yellow stylus light can be utilized, with a dichroic mirror that passes (or reflects, as the case may be) in the yellow region of the visible spectrum. Suitable filtering can also be employed, if desired.

Claims

CLAIMS :

1. Apparatus for displaying an input image and detecting positional indications on the displayed image, comprising: a fiber optical image magnifier having a multiplicity of fiber strands arranged with respective ones of their ends spaced relatively closely together at an image input port and the respective others of their ends spaced relatively further apart at an image display port, said input image being received at said image input port; a position indicating means movable over the image display port, said position indicating means including a light source directed at said image display port; and optical detection means for receiving, from said image input port, light from said position indicating means which has traveled through said image magnifier, said detection means being operative to generate a signal which is indicative of the position on said image input port from which the detected light emanates and thereby indicative of the position of said position indicating means over the image display port.

2. Apparatus as defined by claim 1, wherein said optical detection means includes an array of optically sensitive elements.

3. Apparatus as defined by claim 2, wherein said array is a two dimensional array.

4. Apparatus as defined by claim 3, wherein said optical detection means further comprises means coupled with said array for determining the position on said array which detects the largest amplitude of said light.

5. Apparatus as defined by claim 3, wherein said optical detection means includes a lens for focusing light from said image input port onto said array.

6. Apparatus as defined by claim 4, wherein said optical detection means includes a lens for focusing light from said image input port onto said array.

7. Apparatus as defined by claim 3, further comprising optical separating means for optically coupling said input image to said input image port and for optically coupling light from said input image port to said optical detection means.

8. Apparatus as defined by claim 5, further comprising optical separating means for optically coupling said input image to said input image port and for optically coupling light from said input image port to said optical detection means.

9. Apparatus as defined by claim 7, wherein said optical separating means comprises a dichroic mirror.

10. Apparatus as defined by claim 8, wherein said optical separating means comprises a dichroic mirror.

11. Apparatus as defined by claim 1, wherein said position indicating means comprises a stylus, and said light source is at the tip of said stylus.

12. Apparatus as defined by claim 3, wherein said position indicating means comprises a stylus, and said light source is at the tip of said stylus.

13. Apparatus as defined by claim 7, wherein said position indicating means comprises a stylus, and said light source is at the tip of said stylus.

14. Apparatus as defined by claim 11, wherein said stylus includes an operator-controllable switch for controlling said light source.

15. Apparatus as defined by claim 13, wherein said stylus includes an operator-controllable switch for controlling said light source.

16. Apparatus as defined by claim 1, wherein said light source is an infrared light source.

17. Apparatus as defined by claim 12, wherein said light source is an infrared light source.

18. Apparatus as defined by claim 15, wherein said light source is an infrared light source.

19. Apparatus as defined by claim 1, wherein said light source is a yellow light source.

20. Apparatus as defined by claim 12, wherein said light source is a yellow light source.

21. Apparatus as defined by claim 1 further comprising a second position indicating means movable over the image display port, said second position indicating means including a second light source directed at said image display port, and wherein said optical detection means also receives light from said second position indicating means.