GB2315859A

GB2315859A - Input device simulating touch screen

Info

Publication number: GB2315859A
Application number: GB9715615A
Authority: GB
Inventors: Donald Smith; Pat Orchard; Gordon Freedman
Original assignee: Mitel Corp
Current assignee: Microsemi Semiconductor ULC
Priority date: 1996-07-29
Filing date: 1997-07-25
Publication date: 1998-02-11
Anticipated expiration: 2017-07-25
Also published as: GB2315859B; CA2182238A1; GB9715615D0

Abstract

An input device comprises a two-dimensional surface 12 capable of presenting predefined regions 13 and 14 to a user, a video camera 6 directed at said surface for creating real-time images thereof, a device using a Rybak method to analyse the real-time images to identify a human hand and therefore decide which predefined region is being pointed to by the user, and a device for outputting a signal dependent on the identified region pointed to by the user. The input device can act as an alternative pointing device for a computer, for example.

Description

INPUT DEVICE SIMULATING TOUCH SCREEN This invention relates to a input device, and more particularly an input device capable of simulating a touch screen.

Some means must be found for users to provide input to computers. The keyboard is the most obvious means, but is not always practical because it requires typing ability and is not well suited to situations where rapid input is required. The touch screen has been developed as one solution to this problem. This allows a user to touch a part of a screen to indicate a certain input. For example, when moving through menus, the user can touch certain portions of the screen to select desired menu choices.

The problem with touch screens is that they are expensive to produce and also they only work on cathode ray tubes.

There are situations where "touch screen" capability is desirable on static devices, for example posters and the like.

An object of the invention is to alleviate this problem.

According to the present invention there is provided an input device comprising a two-dimensional surface capable of presenting predefined regions to a user, a video camera directed at said two-dimensional surface for creating real-time images thereof, means for analyzing said realtime images to identify which pre-defined region is being pointed to by a user, and means for outputting a single dependent on the identified region pointed to by the user.

The device preferably includes a pattern recognition device capable of recognising the form of a human hand, for example using a modified Rybak method.

The two-dimensional surface can be a computer screen or alternatively a static display, such as a poster or the like.

The invention will now be described in more detail, by way of example, only with reference to the accompanying drawings, in which: Figure 1 is a block diagram of an input device in accordance with the invention; Figure 2 is a diagrammatic view of one embodiment of the invention; Figure 3 illustrates the process for determining the region pointed to; Figure 4 is a block diagram of a system in accordance with the invention; Figure 5 is a more diagrammatic view of an alternative arrangement; and Figure 6 is a block diagram of the arrangement shown in Figure 5.

Referring to Figure 1, a central processing unit 1, for example an Intel Pentium microprocessor, is connected to a bus 2, to which is attached a random access memory 3, an output 4 for connection to a device to be controlled, for example a computer (not shown), and a camera interface 5.

A still-frame video camera 6 is connected to the camera interface 5. The camera 6 is fixedly mounted so as to face the computer screen 7 and generate a still-frame video images thereof in real time. The camera interface 5 digitizes the output of the camera 6 for connection to the bus 2. The images are stored in the RAM 3 as data.

7 In use, the video camera 5 continually generates a series of still frame images of the computer screen 7.

The CPU 7, under instructions from a program also stored in RAM 3, continually analyzes the stored digital images to look for the shape of a human hand using known pattern recognition techniques, such as a modified Rybak method.

When a human hand is detected, the CPU 3 then determines its terminal location relative to the borders of the image and matches this location with predefined areas on the image. The CPU 1 then instructs the output device 4 to generate a signal identifying which predefined area is being pointed to by the identified hand. This output signal can then be passed directly to a computer as X-Y co-ordinates in the same way as a signal from a mouse or other conventional pointing device.

Figure 2 shows an alternative embodiment wherein a camera 6 is directed at a billboard 12 having two well-defined regions one representing a graphic 13 and the other text 14. As shown in Figure 3, the camera 6 forms a clean image 15 and a current image 16, which includes the hand 17 of a user. The system subtracts the current image from the clean image to leave only the hand 17. The terminal point 18 of the difference image, here point (100, 75), is taken to be the point indicated 19, and the system is calibrated 19 to know that this relates to a particular predefined region, in this case the graph portion 13 of the image.

Figure 4 illustrates a device for implementing this process. The output of the camera 6 is sent to a clean image buffer 21 and a current image buffer 22. The outputs of the buffers 21, 22 are subtracted in subtractor circuit 20 and passed to a device 12, which implements a modified Rybak method to recognize the shape of the human hand. The Rybak method, described in a paper entitled "BMV: Behavioural Model of Active Visual Perception and Invariant Image Recognition" by I.A. Rybak, A.V. Golovan, V.I. Gusakova, N.A. Shetsova and L.N. Podladchikovs located on the World Wide Web at http://www.voicenet.com/ rybak/vnc.html, permits the recognition of certain shapes, such as the form of a pointing hand.

Figure 5 shows a system for monitoring a computer screen, which can be used as an input device for the computer.

The system could, for example, be used in public places such as airports where users are require to make menu choices. The camera 6, which is fixed in position, is directed onto a computer monitor 31 by means of a mirror 30.

First a snapshot" of the computer screen is taken through the operating system of the computer to create a computer image 32, referred to as the OS image. This is done automatically and can be easily achieved using the builtin capability of the operating system. For example, in the case of the Windows operating system, pressing the key Print Scrn causes a bitmap image of the screen to be passed the Windows clipboard, and it is a simple matter to achieve this internally under program control.

Next a reference image, which will be referred to as the camera image 33, is taken of the clean screen with the camera 6. A translation function 34 is then derived to translate the camera image to the OS image. This is referred to as the delta function A.

The same procedure is repeated in the presence of an object such as a hand 34 and the delta function 35 of the current image is formed, which is called the TRANSLATED image 36. The TRANSLATED image is then subtracted from the current OS image 37 to form a STRIPPED image 38, which contains only the difference between the calibration image and the present image.

This procedure makes feature recognition easier using the modified Rybak method referred to above. It can also be used in conjunction with the other examples referred to herein.

The modified Rybak method is used to determine where the hand is pointing to 39, and this point is referred to as the HOT SPOT. The X, Y co-ordinates of the HOT SPOT are then determined and these are sent to the operating system as a mouse position 40. The current image 34 can be taken as often as necessary.

Figure 6 shows an arrangement for implementing the above method. The snapshot of the screen created by the operating system 50 is stored in buffer 41 and that by the camera 6 stored in buffer 42. The delta function is created in block 43 and passed through block 44 to subtractor 45 where it is subtracted from the OS image create the STRIPPED image, which is passed to the modified Rybak processing unit 46. Unit 47 derives the X,Y coordinates of the hot spot and passes these to the operating system of the computer as if they were a mouse input 48.

The invention thus permits a computer screen or other display device, including passive display devices, to be given a "touch screen" capability in a particularly economical manner.

Claims

1. An input device characterized in that it comprises a two-dimensional surface capable of presenting predefined regions to a user, a video camera directed at said twodimensional surface for creating real-time images thereof, means for analyzing said real-time images to identify which pre-defined region is being pointed to by a user, and means for outputting a signal dependent on the identified region pointed to by the user.

2. An input device as claimed in claim 2, characterized in that it further comprises pattern recognition means for recognizing the form of a human hand in said real-time image, and said analyzing means generating said output signal according to the location of said hand form on said two-dimensional surface.

3. An input device as claimed in claim 2, characterized in that said recognizing means uses a modified of the Rybak method of pattern recognition.

4. An input device as claimed in any one of claims 1 to 3, characterized in that it further comprises means for storing a clean image of said two-dimensional surface, and wherein said analyzing means subtracts a real-time image from said stored clean image to create a difference image which is analyzed to find the location of said hand form relative to said clean image.

5. An input device as claimed in any one of claims 1 to 4, characterized in that said two-dimensional surface is a computer screen and said analyzing means subtracts said real time image from a snapshot image produced by an operating system thereof to produce a delta function which translates said current image to the operating system image, and said analyzing means compares a real-time image translated with said delta function with the operating system image to obtain a stripped image that said recognition means uses to identify the shape of a human hand.

6. An input device as claimed in any one of claims 1 to 5, characterized in that said output signal is operative to control the operation of a computer.

7. An input device as claimed in any one of claims 1 to 6, characterized in that said video camera is a stillframe video camera.

8. An input device substantially as herein described, with reference to the accompanying drawings.