GB2345538A

GB2345538A - Optical tracker

Info

Publication number: GB2345538A
Application number: GB9824339A
Authority: GB
Inventors: David James
Original assignee: Individual
Current assignee: Individual
Priority date: 1998-11-06
Filing date: 1998-11-06
Publication date: 2000-07-12
Anticipated expiration: 2018-11-06
Also published as: GB9824339D0; GB2345538B

Abstract

A tracker having a purely optical interface comprises a monocular video camera to observe the actions of a user, a video capture card to convert the video image into numbers, and a computer with dedicated software for video analysis and subsequent 3D and 2D tracking of incoming images of the body or a body part eg for controlling a computer cursor or a virtual reality image and sound. The computer decodes 3D body information via intelligent computer algorithms and optical image recognition which determine the position and size of recognised objects. There is no need for body worn transmitter or reflectors, VR dataglove or headset, mouse or lightpen. A mouse pointer may be moved around the screen merely by pointing at the desired position in free space. Also the viewer's head may be optically tracked. The viewer can view a computer object from different angles and move it in 2 or 3 dimensions merely by pointing at it.

Description

Optical Tracker Video cameras are being connected to computers in increasing numbers and new uses for these cameracomputer interfaces are being found.

An optical tracker is a method of interacting with a computer, by allowing the computer to observe the actions of the user. This interface is purely optical in nature and relies on a video camera, observing the user's actions, a video capture card, which converts the video image into numbers for the computer to analyse, a computer to perform the video analysis and subsequent body or body part position tracking.

The primary function of such a system is as a virtual mouse pointer (V-mouse), virtual reality image (VR) and sound control, (3D object manipulation and stereo and 3D sound positioning).

All this is done with no body worn transmitter or detectable device such as reflective pads/objects. There is no need to wear a VR dataglove or VR headset. There is no need to touch any pointing system such as a mouse, lightpen or the like.

The computer is programmed to decode 3D body information via intelligent computer algorithms and optical image recognition.

This described Optical Tracker allows the mouse pointer can be moved around the computer screen merely by pointing at the desired position in free space.

Also the user can view computer 3D objects from different angles. The viewer's head is optically tracked by a video camera. The user can also move any computer object in 2 or 3 dimensions merely by pointing at it. This is all achieved without the user having to wear or touch any special equipment. The system is completely passive and relies on the computer intelligently analysing the viewer's limb and body positions.

GENERAL DESCRIPTION The Optical Tracker is a video image analysis system that allows a single video camera to locate a known object in 3D space. Co-ordinates received in this manner can be used to control a mouse pointer, VR environment 3D and 2D computer controlled/generated objects.

The equipment needed is: a video camera, a video capture card, a computer and operating software. The video camera sits on the computer monitor and watches the user/s. The video information is received by the on-board video capture card. The video information is decoded and analysed by the Optical Tracker software. The computer can be programmed to continually search for particular objects, such as a human head or hand. Once it has detected, say a hand it can then track the hand as it is moved around the camera's field of view The software has four main levels: I] Video capture card interface, 2] Background detection, 3] Object edge detection.

4] Known object comparison, movement and 3D position detection.

It contains a 3D model of the objects it is trying to locate. It emulates the way we match our memorised visual models onto incoming images. It attempts to determine an incoming object's position by overlaying it's internal model of the object type it is trying to detect on top the incoming object.

It tries to move its internal 3D object overlay, until best match with incoming object is found, thereby determining the incoming object's 3D positioning.

Optical Tracker This invention relates to an optical tracker The optical tracker is a 3D object tracker that requires a monocular optical input, eg: a video camera.

An example of the use of the optical tracker is as a VR body tracker that does not require the user to wear trackable device such as a helmet worn transmitter, or to be exposed to trackable force fields such as electrostatic force fields.

The computer is programmed to see and recognise the person's body position, in much the same way that a human being can determine the position someone who they are looking at.

The optical tracker operates on visual information alone.

Because the optical tracker can track 2D and 3D body position it can also be used to track a pointed finger for use as computer cursor pointer, ie: a virtual mouse It can generally be used to track the 2D or 3D position of any known object.

Images viewed by a video camera are fed into a digitising video capture card.

The video capture card produces a colour level for each pixel (picture element) in the camera's field of view.

There are 4 levels of software control required to perform this optical tracking : 1] Video capture card interface, 2J Background detection, 3] Object edge detection, 4] Known object comparison and 3D position detection.

Level 1 of the optical tracking software allows access to the hardware registers containing the video pixel colour information, so that a complete digital picture of video events is available to the optical tracking software for further analysis.

Level 2 of the optical tracking software measures relatively static pixel levels and assigns these pixel levels to the background, ie anything in the cameras field of view that does not move for a prolonged period is assumed to be background information.

Level 3 of the optical tracking software measures any changes in pixel colour levels Scanning pixels in turn for numerical colour level changes.

If a pixel level change is found between consecutive video frames AND no prior pixel level changes are found, then this constitutes finding an object PRIMARY EDGE. Object SECONDARY EDGES are found by reversing this process.

Objects are scanned left to right for boundary information, starting in the top left corner of a video frame. It is immaterial where the scan starts and in which direction it proceeds as long as the picture area containing the object/s for detection is scanned.

Once object boundaries are determined relative to background, then an object shape is known and stored in a numerical array.

This INCOMING OBJECT SHAPE or 2D slice is then compared to a KNOWN OBJECT SHAPE or 2D slice.

A comparison is made of two 2D slices or PROJECTIONS of 3D objects.

One is the incoming video of 3D objects and the other is the internally held 3D object Level 4 of the optical tracking software measures the diameters of the incoming edge detected object and adjusts the internal 3D model scale to fit the incoming object.

Edge overlap is then measured for each vertex of the computer model relative to the incoming object.

If object vertices overlap within required tolerance, then the software decides that it has found a incoming object that matches its internal model.

If the incoming object is then moved, the intemal model can be moved to try and re-match the incoming object.

In this way the computer can detect changes in incoming object orientation [TRANSLATION, SCALE, ROTATION, ARTICULATION1.

By way of example: A fully articulated human hand 3D model is held in computer memory and can be changed into any configuration at high speed to try and match incoming images of the human hand.

Once a match is found between incoming hand and the virtual hand, the software knows the TRANSLATION, SCALE, ROTATION, and ARTICULATION of the incoming hand.

This information means that the software knows where for example a hand is pointing to in 3D space.

THESE COORDINATES CAN MOVE A MOUSE POINTER CURSOR ON THE SCREEN.

Furthermore, because incoming 3D object information is detectable, full 3D virtual world control is possible eg : operator head and body movement can control 3D worlds and object in the computer.

According to the present invention there is provided an optical tracker comprising a visual sensing device, (eg: video camera), video capture hardware, (eg: video capture card) a computer and optical tracking software for video image 3D and 2D image position analysis.

A specific embodiment will now be described by way of example to the accompanying drawings in which :-- Figure I shows in block diagram the optical tracker where a person (item 5) points towards a video camera (item 4) The output from the video camera (item 4) is fed as an electronic video signal into a video digitiser (item 3) this shown in both Figures I and 2.

The output from the digitiser is fed to a computer (item 2) which is programmed to analyse the video camera image this shown in both Figures I and 2 The position of the person relative to the background is calculated in 3D space as shown in Figure 1 The captured image of the person is used to move 3D computer generated object/s as shown in Figure I.

The person may move the computer generated images (item 6) on the computer monitor (item 1) by moving their body as shown in Figure 1.

The computer generated 3D image/s move in sympathy to detected body motion as shown in Figure 1.

Claims

Optical Tracker Claims 1) An optical tracker comprising a monocular video camera, video capture card, computer and dedicated software for 3D and 2D tracking of incoming images, used to control the computer cursor, ie: used in lieu of a mouse.

2) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of incoming object are used to control virtual world configuration and positioning.

3) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of incoming object are used to control a 3D cursor, ie: a computer pointer that can be moved around the computer display and in and out of the computer display, ie: the z-axis.

4) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the head are used to control virtual world configuration and positioning.

5) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the head are used to control the computer cursor.

6) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the hand are used to control virtual world configuration and positioning.

7) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the hand are used to control the computer cursor.

8) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of any part of a body are used to control virtual world configuration and positioning.

9) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of any part of a body are used to control the computer cursor.

10) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a wand, ie: pointing tool are used to control virtual world configuration and positioning.

11) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a wand, ie: pointing tool are used to control the computer cursor.

12) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a body are used to position image overlays in virtual reality assisted surgery 13) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a body are used to control stereo and 3D sound positioning.

Amendments to the claims have been filed as follows 1) An optical tracker comprising a monocular video camera, video digitiser (internal or external to camera), computer and dedicated software for 3D and 2D tracking of incoming images, used to control the computer cursor, ie: used in lieu of a mouse.

2) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of incoming object are used to control virtual world configuration and positioning.

3) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of incoming object are used to control a 3D cursor, ie: a computer pointer that can be moved around the computer display and in and out of the computer display, ie: the z axis.

4) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the head are used to control virtual world configuration and positioning.

5) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the head are used to control the computer cursor.

6) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the hand are used to control virtual world configuration and positioning.

7) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of the hand are used to control the computer cursor.

8) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of any part of a body are used to control virtual world configuration and positioning.

9) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of any part of a body are used to control the computer cursor.

10) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a wand, ie : pointing tool are used to control virtual world configuration and positioning.

11) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a wand, ie: pointing tool are used to control the computer cursor.

12) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a body are used to position image overlays in virtual reality assisted surgery 13) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of a body are used to control stereo and 3D sound positioning.

14) An optical tracker as claimed in claim 1, wherein optically tracked co-ordinates of incoming object are used to move in 3D projection an articulated 3D computer generated jointed object, in which the computer generated joints move in sympathy to real-world camera viewed jointed object.

I) An optical tracker as claimed in claim 14, wherein the computer 3D jointed object is used as a puppet (avatar) for the purpose of moving through a 3D game or VR environment..

16) An optical tracker as claimed in claim I or claim 14, wherein the computer 3D jointed or non-jointed, articulated or non-articulated object is used as a 3D video target object recognition and (3D or 2D) tracking system, 17) As claimed in claim 1 wherein the capture card analogue to digital converter electronics is internal or integral with the video camera