WO2007135404A2

WO2007135404A2 - Computer input device

Info

Publication number: WO2007135404A2
Application number: PCT/GB2007/001868
Authority: WO
Inventors: John Edley Wilson
Original assignee: Spiral Scratch Limited
Priority date: 2006-05-18
Filing date: 2007-05-18
Publication date: 2007-11-29
Also published as: GB2438667A; GB0609858D0; WO2007135404A3

Abstract

A contactless computer input arrangement comprising a light pen and a camera, and software to determine a position on a monitor screen corresponding to the position of the light pen in relation to the screen.

Description

COMPUTER INPUT DEVICE

This invention relates to computer input devices.

It is well known to produce drawings or coloured pictures using a pointing device such as a mouse or a stylus moving over a pad or a touch screen. Software in the computer keeps track of the movement of the pointing device and produces lines or shapes on a monitor screen according to the movement of the pointing device and instructions input usually through the pointing device, for example a click of a mouse button or a stylus button.

Contactless systems have been proposed. Japanese Patents 6301474 and 6301475 disclose the use of pen lights as position indicating devices in computer games. A player stands before a monitor in view of a camera and moves his hands, carrying pen lights. The monitor may display a drum set, and may also display the player in such relation to the drum set that the player's image appears to strike the drums, which yields a corresponding sound. The monitor may display, in another game, a monster, which the player smites, in the screen image, using a sword.

Japanese Patent 6131442 discloses the use of a stylus pen with a position indicating point to access a virtual image surface and reshape it.

Taiwanese patent publication TW479190 discloses a pen-shaped light emitting device which can emit at least two colours of light, and a camera connected to a computer capturing the light from the light emitting device, and determining its colour, and converting the captured moving trace for the spots of the moveable light emitting device to XY coordinates. This can be used e.g. to generate handwriting on the screen.

DE4336677 discloses a pen-like input device having a light source and a camera arranged adjacent to a user interface and serving to receive the light generated by the input device. X, Y coordinates of the position of the user-controlled input device are ascertained by the camera. An acceleration sensor arranged in the input device generates depth information, which is transmitted to the system via the light beam, which can be modulated to transmit additional information to the system. These are all fairly complex and expensive arrangements that do not appear to have been sufficiently commercially attractive, for the most part, to merit international patent protection. For serious technical drawing and design work, it is difficult enough to draw much more than straight lines and predetermined shapes with a mouse mat or tablet defining a plane of movement. To create such drawings in free space (whether or not a third dimension is taken into account) is much more difficult, and the arrangements would appear to be no easier to use, and to produce no better results, than conventional "contact" arrangements.

The present invention provides an inexpensive, contactless computer input arrangement that can be used to control and interact with a computer to perform a variety of tasks and functions. The invention comprises a contactless computer input arrangement comprising a light pen and a camera, and software to determine a position on a monitor screen corresponding to the position of the light pen in relation to the screen. The invention in one aspect comprises a contactless computer input arrangement comprising a light pen having a light source, a camera arranged to detect the light source, and software to determine and mark a position on a monitor screen corresponding to the position of the light source in the view of the camera, the light pen being adapted to determine the nature of the mark on the monitor screen.

The term 'light pen' as used herein connotes an implement that can be wielded by a person, having a resemblance to a pen, such, for example, as a pen, pencil, crayon or other drawing implement, or another implement not specifically for the purpose of drawing or colouring, but which has the same general shape, such, for example, as a sword or a conductor's baton or a tennis or squash racquet, or any implement that can be held and aimed or pointed at a screen, such as a pistol or other gun,

The light pen may comprise a transmitter communicating with the computer to determine the nature of the mark. The transmitter may transmit but a single signal. A plurality of light pens may transmit signals interpreted by the software as different colours of crayon or pastel, and may be comprised as a set. The light pens may be associated with physical colourants, and may in fact be crayons or pastels with associated light sources and transmitters, so that they may be used to create drawings both on screen and on paper. The transmitter may comprise an infra red transmitter, which can be realised commercially at very little cost.

The light pen may, however, comprise a modulator for the light source, which may itself be an infra red source, or, indeed, any other radiation source detectable by the camera.

The transmitter may, of course, transmit more than one signal, selected by a switch arrangement. A two-position switch may serve to produce two colours, for example, or two textures or line thicknesses. Three or more position switch arrangements could also be used.

One light pen, or one mode of a multi-mode light pen, could, of course, be an eraser.

The mark on the screen need not necessarily be a permanent mark or trace. The mark may comprise, for example, a depiction of a sword for a fencing game, or a bat or racquet for a ball game.

Software can present a background on the monitor screen against which a game or a pastime may be carried on. For example, a painting-by-numbers outline may be displayed. The background may contain active elements, such, for example, as a ball, or a fencing opponent. The software may contain rules concerning the progress of the game or pastime, for example if a red crayon is used for an area that should be coloured green, an alarm will be raised, and if a sword blow strikes an opponent, the opponent will be declared dead and removed from the screen.

The camera may image the player and present the image on the screen for player- participation games or perhaps as a template for a self portrait.

The light pen may be battery powered. The battery may be a semi-permanent, rechargeable battery. A set of light pens may sit in a charger base.

This invention also relates to light guns, more specifically, guns as used in video games aimed at targets on a video screen, the term light gun, as used herein, meaning more particularly, implements that can be used in shooting games..

Such guns customarily comprise an optical system which focuses an image of a small area of the screen on to a photodiode. Deciding which area is aimed at - to decide whether a shot is a hit or a miss - is done in one of a number of ways.

In one arrangement, actuation of the gun's trigger causes the entire screen area to black out, except for the target area, which is turned white. If the gun as aimed accurately at the target, white light from the screen after vertical and horizontal raster scans will be focussed on the photodiode, which will return a signal to circuitry which indicates a hit or a miss in ways appropriate to the game.

In another arrangement, actuation of the trigger causes the entire screen to black out and then to be illuminated white. In video screens with which this technique is employed, the screen is painted by an electron beam writing a raster pattern. The photodiode in the gun is illuminated when the spot on the screen at which the gun is aimed is illuminated by the electron beam. The time after the beginning of the raster scan at which this occurs is evaluated, and returns the position on the screen at which the gun is aimed. These, of course, do not work with video screens that do not have raster scans.

The present invention, an another aspect, then provides a light gun which works with any kind of screen, and which is more versatile in use. The invention, in said other aspect, comprises a light gun comprising a camera.

The camera may be arranged to image a small area of a monitor screen at which it is pointed. A game arrangement may be adapted to detect, from the characteristics of the image, the area of the screen at which the gun is pointed. On actuation of the gun's trigger, the screen may be arranged to display, briefly, a pattern from which the position on the screen at which the gun is pointed may be determined. The pattern may fill the screen, or at least a general target area thereon, and be different at each location, the arrangement being such that matching the image pattern with the screen pattern returns the location. However, games may be played which do not depend on the gun being aimed at a specific target, but which return a value depending on where the gun was pointed when the trigger was pulled. If the image is time dependent, the value may also depend on when the trigger was pulled.

The light gun may comprise an aligned beam projector, which may comprise a laser pointer, to assist with aiming. The invention also comprises a shooting game system, comprising a light gun comprising a camera. The light gun camera may image a small portion of a target screen, and the system may comprise screen object generator means adapted to generate a screen object, and locator means identifying where on the screen the gun is pointing by analysing the camera image.

The screen object generator means may generate a screen object of which any one camera image uniquely determines its location on the screen.

However, the screen object may comprise a pattern which may have multiple occurrences on the screen, and may comprise multiple patterns which return different values when images thereof are analysed.

The screen object generator means generates an object extending over the entire screen or a portion thereof, and may generate a transient screen object synchronised with actuation of the gun. Such a screen object may be displayed for about one sixtieth of a second, which is long enough for image capture, but not so long that it will noticeably interfere with the screen image it interrupts.

The screen object generator means may generate a time-dependent object.

The screen object generator means may generate an object including multiple patterns, and the system comprise pattern analysis means returning different a value for each patterns, and perhaps different values for different patterns, whereby different kinds of game may be played, which do not necessarily involve shooting at targets.

Embodiments of contactless computer input arrangements according to the invention will now be described with reference to the accompanying drawings, in which:

Figure 1 is a diagrammatic illustration of light pens interacting with a computer via a camera and infra red receiver;

Figure 2 is a diagrammatic illustration of a 4-pen set with a monitor screen displaying a paint-by-numbers template: Figure 3 is a screen image for a tennis game; Figure 4 is a diagrammatic illustration of a light gun;

Figure 5 is an illustration of a screen pattern associated with the light gun of Figure

4; and

Figure 6 is a diagram of a game system involving the light gun of Figure 4.

The drawings illustrate contactless computer input arrangements comprising a light pen and a camera, and software to determine a position on a monitor screen corresponding to the position of the light pen in relation to the screen.

Figures 1 to 3 of the drawings illustrate a contactless computer input arrangement comprising a light pen 11 having a light source 12, which may be an led, a camera 13 arranged to detect the light source 12, and software in a computer 14 to determine and mark a position on a monitor screen 15 corresponding to the position of the light source 11 in the view of the camera 13, the light pen 11 being adapted to determine the nature of the mark on the monitor screen 15.

Each of the light pens 11 illustrated in Figure 1 comprises a transmitter 16 communicating with the computer 14 to determine the nature of the mark. The transmitter 16 transmits but a single signal. A plurality of light pens 11 may transmit signals interpreted by the software as different colours of crayon or pastel, and may be comprised as a set. The light pens 11 may be associated with physical colourants, and the light pen 11 illustrated in Figure 1 has a crayon or pastel 1 Ia at the end opposite to the light source 12, so that it may be used to create drawings both on screen and on paper. A crayon will, of course, eventually be used up, and it may be arranged that the crayon is replaceable.

The transmitter 16 comprises an inexpensive infra red transmitter.

The light pen 11 includes a battery power source. This may use replaceable batteries or may have a rechargeable battery, in which case, a set of light pens may sit in a holder 21, Figure 2, with a built-in charger. Figure 1 shows two light pens 11, which paint different colours on the monitor screen 15, as represented by coded signals, illustrated diagrammatically as pulse trains 16a, emitted by the transmitters 16 of the light pens 11.

Movement along Arrow A shown adjacent one of the lights pens 11 results in a mark M on the screen 16 in a screen colour corresponding to the colour assigned to that light pen. It is, of course, possible to have a more complex transmitter, which is capable of being switched between different pulse trains representing different colours, with a switch arrangement built into the light pen. Then, only one light pen need be provided for multiple colours. It is possible, moreover, that the light source can double as the transmitter. A single led can be used both as the light source and the means by which a signal is transmitted to the computer, being pulsed according to a code for red, green etc. Provided the camera can pick up infra red signals, the led may be an infra red emitting diode.

Instead of a writing implement, a light pen 11 may represent a sword or a sports implement such as a tennis racquet, as illustrated in Figure 3. Software may generate a game background, with an opponent 31 hitting a ball 32 over a net 33, to be returned by the played appropriately wielding the light pen.

Figures 4 to 6 of the drawings illustrate a light gun 111 and a game system incorporating the same.

The gun 111 comprises a barrel 112, a handle 113 and a trigger 114. In the barrel 112 is a lens 115 and a ccd array 116, comprising a digital camera 117. The barrel also houses a laser 118 which is aligned with the camera 117 and acts as an aiming aid, placing a visible spot on a video screen 121 of a video game arrangement 122. The gun 111 is connected via a cable 119 to the arrangement 122 (Figure3), which could of course be a dedicated game machine or a computer running game software.

The camera 117 images a small area 123 of the screen display. The area 122 is shown in

Figure 5 and is a few pixels square.

When the trigger 114 is pulled, a signal is sent to the arrangement 122 to flash on the screen 121 a pattern, a part of which is the area 122 shown in Figure 5. Each like-sized area of the screen is unique. The camera 117 images the area 122 and details of the image sent to the arrangement 122 which identifies from such details the position on the screen 121 at which the gun 111 is aimed. Provided the pattern remains long enough on the screen 121, the area 122 will be imaged by the camera 117, regardless of the nature of the screen, i.e. CRT, plasma display, led or led display. A duration of one sixtieth of a second should suffice for most purposes. Such a transient appearance would not be noticed by the player. In another configuration, the pattern need not be placed over the entire screen 121, but only over a target area.

Instead of uniquely identifying a single location, the pattern may have areas corresponding, for example, to rings of a bulls eye, so that one pattern is on the centre and other patterns in concentric rings, different patterns returned through the camera 17 having different scoring values.

Patterns may be deployed randomly, in another game version, scoring being not so much a matter of skill in aiming for a precise target but rather a matter of luck in hitting randomly arranged areas of different scoring values. The pattern distribution may also vary with time, so that different areas of the screen return different values at different times.

A pattern may comprise an arrangement of pixels as shown in Figure 5, or may be based on colour, the top left hand corner of the screen being, for instance, red, and the bottom right hand corner blue, with a gradual change between these two extremes across the screen. This, of course, would not be compatible with monochrome screens.

Where a pixel pattern is used, image processing techniques would be deployed to cater for camera tilt and other image artefacts.

The gun and associated systems represent a step forward over the conventional photo diode gun, while being compatible with existing games based on that technology.

Claims

1 A contactless computer input arrangement comprising a light pen and a camera, and software to determine a position on a monitor screen corresponding to the position of the light pen in relation to the screen.

2 A contactless computer input arrangement according to claim 1, comprising a light pen having a light source, a camera arranged to detect the light source, and software to determine and mark a position on a monitor screen corresponding to the position of the light source in the view of the camera, the light pen being adapted to determine the nature of the mark on the monitor screen.

3 An arrangement according to claim 2, in which the light pen comprises a transmitter communicating with the computer to determine the nature of the mark.

4 An arrangement according to claim 3, in which the transmitter transmits a single signal. 5 An arrangement according to claim 4, in which a plurality of light pens transmit signals interpreted by the software as different colours of crayon or pastel.

6 An arrangement according to claim 5, in which the plurality of light pens is comprised as a set.

7 An arrangement according to any one of claims 1 to 4, in which a light pen is associated with a physical colourant.

6 An arrangement according to claim 5, in which the physical colourant comprises a crayon or pastel.

7 An arrangement according to any one of claims 1 to 6, in which the transmitter comprises an infra red transmitter. 8 An arrangement according to any one of claims 1 to 7, in which the light pen comprises a modulator for the light source.

9 An arrangement according to claim 8, in which the modulator comprises an infra red or other radiation source detectable by the camera.

10 An arrangement according to any one of claims 1 to 9, in which the transmitter is adapted to transmit more than one signal, selected by a switch arrangement.

11 An arrangement according to claim 10, in which the switch arrangement comprises a two-position switch. 12 An arrangement according to claim 11, in which the switch serves to produce two colours, for example, or two textures or line thicknesses.

13 An arrangement according to claim 10, in which the switch arrangement comprises a switch with three or more positions.

14 An arrangement according to any one of claims 1 to 13, in which one light pen, or one mode of a multi-mode light pen, is adapted to operate as an eraser. 15 An arrangement according to any one of claims 1 to 15, in which the mark on the screen is an enduring mark or trace.

16 An arrangement according to any one of claims 1 to 16, in which the mark is a transient mark, such as a depiction of a sword for a fencing game, or a bat or racquet for a ball game.

17 An arrangement according to any one of claims 1 to 16, in which software presents a background on the monitor screen against which a game or a pastime may be carried on.

18 An arrangement according to claim 18, in which the software presents a painting- by-numbers outline.

19 An arrangement according to claim 18, in which the background contains active elements.

20 An arrangement according to any one of claims 17 to 19, in which the software contains rules concerning the progress of the game or pastime. 21 An arrangement according to any one of claims 1 to 20, in which the camera images the player and presents the image on the screen for player-participation games.

22 An arrangement according to any one of claims 1 to 20, in which the camera images the player as a template for a self portrait.

23 An arrangement according to any one of claims 1 to 22, in which the light pen is battery powered.

24 An arrangement according to claim 24, in which the battery comprises a semi- permanent, rechargeable battery.

25 An arrangement according to claim 24, comprising a set of light pens and which sit in a charger base, and a set of light pen inserts. 26 A contactless computer input arrangement according to claim 1, comprising a light gun comprising a camera. 27 A light gun according to claim26, in which the camera is arranged to image a small area of the screen at which it is pointed. 28 A light gun according to claim 26 or claim 27, comprising an aligned beam projector to assist with aiming.

29 A light gun according to claim 28, in which the pointer comprises a laser pointer. 30 A shooting game system, comprising a light gun according to any one of claims 26 to 29.

31 A system according to claim 30, in which the light gun camera images a small portion of a target screen.

32 A system according to claim 30 or claim 31, comprising screen object generator means adapted to generate a screen object, and locator means identifying where on the screen the gun is pointing by analysing the camera image. 33 A system according to claim 32, in which the screen object generator means generate a screen object of which any one camera image uniquely determines its location on the screen.

34 A system according to claim 32 or claim 33, in which the screen object comprises a pattern having multiple occurrences on the screen.

35 A system according to claim 34, in which the screen object comprises multiple patterns which return different values when images thereof are analysed. 36 A system according to any one of claims 32 to 35, in which the screen object generator means generates an object extending over the entire screen.

37 A system according to any one of claims 32 to 36, in which the screen object generator generates an object extending over a portion of the screen.

38 A system according to any one of claims 32 to 37, in which the screen object generator generates a transient screen object synchronised with actuation of the gun.

39 A system according to claim 38, in which the screen object is displayed for about one sixtieth of a second.

40 A system according to any one of claims 32 to 39, in which the screen object generator means generates a time-dependent object. 41 A system according to any one of claims 32 to 40, in which the screen object generator means generate an object including multiple patterns, and the system comprises pattern analysis means returning different a value for each patterns. 42 A system according to claim 41, in which the pattern analysis means returns different values for different patterns.