WO2002007430A1

WO2002007430A1 - Camera-tracking system and method

Info

Publication number: WO2002007430A1
Application number: PCT/NO2001/000300
Authority: WO
Inventors: Harald Moengen
Original assignee: Spotzoom As
Priority date: 2000-07-14
Filing date: 2001-07-12
Publication date: 2002-01-24
Also published as: NO20003635L; NO20003635D0; AU2001272859A1

Abstract

A system and method for registering a camera's position and orientation comprising: at least two transponders arranged in known positions relative to a reference on the camera; at least two detectors designed to be capable of determining the distance from the respective detector to the said transponders; equipment for, in a selected coordinate system, calculating a position for the said transponders and a vector representing the direction for a reference axis on the camera based on data from the said detectors and; communication connections between the respective units in the system. From at least two basic positions the distance to the transponders is determined when the basic positions are known positions in a selected coordinate system, the position of the transponders in the selected coordinate system is calculated, base on which a direction vector is calculated representing the direction of a reference axis on the camera.

Description

Camera-tracking system and method

The present invention relates to a method and a system for registering and storing the positions and orientation of one or more cameras in connection with television and video production. When video recordings are made in a studio or in other locations, synthetic elements are often introduced into the image in addition to the physical objects which are being filmed, for example in the case of weather forecasts where it is only the person presenting the weather forecast who is actually present in the studio ("virtual studios"), while the weather map, symbols and animations are inserted in the video image in a manner which makes them appear to be a part of the studio environment. To enable the synthetic elements to be placed in the correct position in the image, accurate information is required concerning the position and orientation of the camera which is filming. This has traditionally been solved by making the cameras assume fixed, known positions in the studio and only controlling zoom and focusing while an operator constantly monitors the image.

US patent 5,930,740 discloses a method and a system for finding the field of view of a camera. The camera has a fixed position and "sees" 4 fixed, known reference points which are used to find the field of view by means of triangulation.

A fixed position for the camera creates limitations in a recording situation since the position thereby restricts the camera's field of view. It will therefore be necessary to have a large number of cameras and/or extremely strict direction while recording. US patent 5,889,550 discloses a system for determining the 3 -dimensional position and orientation of the film plane of a film camera by means of several light-reflecting cursors placed in a geometrical pattern on the camera. The system employs a number of video cameras which are placed along various lines of sight towards the film camera and transmit pulsed IR light which is reflected by the cursors on the film camera.

Detection of a reflected optical signal depends on there being no obstacles in the optical path which block the signal. This can easily be a limiting factor in a studio where the objects which have to be filmed are moving. Nor can the cameras which receive the reflected light distinguish between the reflecting cursors and they may therefore be hidden behind one another. Reflections from other reflecting surfaces in the studio, moreover, may provide false signals, resulting in noise in the measurements.

For the record it should be pointed out that the terms video and video production in this context refer both to live broadcasts for television and other productions for television and video viewing.

The object of the invention is to provide a system and a method for registering the position and orientation of a camera for use, e.g., in connection with "virtual studios". The invention utilises position indicators for calculating and registering the position and orientation of a camera. The position indicators are arranged at defined positions on the camera, for example at the end of each arm of a cross which is attached to the camera and is horizontal when the camera is horizontally oriented. In principle, the number of arms/position indicators may be changed, but four position indicators, together preferably with a fifth in the centre of the cross, will be preferred.

The position indicators may be active radio transmitters/receivers based on battery operation, or they may be active or passive transceivers. In a preferred embodiment transponders without batteries are employed, preferably implemented by means of surface acoustic wave technology (SAW technology).

A number of detectors which can transmit and receive signals transmit polling signals to the transmitters or transponders which return the signals, preferably in altered form in a manner which enables the individual transponder to be identified when the detector receives the signal. Based on a measurement of the time between transmission and receipt of the signal, the distance to the transponder is calculated from each detector. By means of trigonometry the positions of the respective transponders can therefore be found, and the camera's position and orientation thereby determined. In principle the number of detectors required for determining the position of a transponder depends on the nature of the area in which the transponder is located. If the area is essentially flat and relatively limited, two detectors could be sufficient to achieve a precise position determination. If, however, the area is three-dimensional, i.e. where there are three coordinates, x, y, z which have to be found, or the area is relatively large in relation to the positions of the detectors, it could be necessary to have up to four position detectors in order to find the position of a given transponder. A restriction of the number of position detectors to fewer than is strictly necessary in order to achieve an unambiguous mathematical solution when calculating the transponder's position depends on whether alternative solutions can be ruled out since they are not meaningful, for example because they are located under the ground, outside the studio or the like.

If the camera concerned will always have the same direction in the vertical plane, with the result that it is only the camera's direction in the horizontal plane which has to be determined, a situation will exist where it is not necessary to determine the transponders' vertical position, since it does not alter. This could be the case in a studio with a restricted operational area, where for example the camera is always directed towards objects which are permanently located at the same height. However, it will normally be necessary to determine the camera's direction both in the horizontal plane (panning) and in the vertical plane (tilt), which means that there will have to be three coordinates for the transponders' positions.

The object of the invention is achieved by means of the features in the patent claims.

The invention will now be described in greater detail in the form of an embodiment, with reference to the attached drawings, in which: fig. 1 is a principle drawing illustrating the set-up of a system according to the invention, fig. 2 illustrates a transponder for use in position determination in a possible embodiment of the invention, and fig. 3 illustrates the mounting of transponders on a camera which may be used in an embodiment of the invention.

Figure 1 illustrates a principle drawing of a system according to the invention. In this example four detectors 1, 2, 3, 4 are placed in a studio. In the studio there is a camera 5, whose position and orientation have to be registered continuously over a given period of time. The camera is equipped with a number of transponders. The four detectors employed in this example are each capable of detecting the distance to each of the transponders. The registered distance is transferred via a data bus 11, or by means of another known per se form of data transfer, to a data processing device 12.

The detectors comprise antennas which can transmit and receive an electromagnetic signal sequence and are equipped with a clock for registering time between transmitted and received signal sequence and a counter which counts the number of signallings in each signal sequence. On the basis of information concerning the electromagnetic signal, the registered time between transmitted and received signal and the number of signals in the signal sequence, a signal processing device linked to the detectors calculates the distance to the transponder which has reflected the signal.

By means of the registered distances and the respective detectors' known positions, the data processing device 12 calculates the transponder's position in the form of coordinates x_l5 y_l5 zγ in a defined coordinate system. It will be natural to define an orthogonal coordinate system, with the result that the studio floor is substantially located in the x,y plane, while the z axis is located perpendicularly to this plane. The position is located in the point of intersection between theoretical spherical shells where the registered distances represent the radius of the spheres.

Using trigonometry the angle between the transponders is then calculated represented as a vector in the coordinate system in the x,y plane, the y,z plane and the z,x plane. All the transponders' positions are known in relation to a reference point on the camera, thus enabling the vector representing the angle relative to the camera's reference axis to also be calculated.

Since extreme precision in the position calculations is of vital importance for the angle calculations, this precision must be optimised.

If a distance accuracy of 5 cm is required for an individual distance measurement, this means that the time measurement must have an accuracy of 0.33ns. This demands an exceedingly high clock frequency with a digital implementation and is difficult to achieve in practice. The transmitter frequency in the radar system will be determined by the resolution, where the wavelength should be a good deal smaller than the resolution. With a resolution of 5 cm the transmitter frequency should be greater than 6 GHz.

By performing repeated measurements with smaller resolution and averaging them, the precision of measurement is increased. The measurements are therefore preferably performed by transmitting a new polling signal as soon as a return signal is received, whereby the system oscillates between polling signals and return signals in a signal sequence, typically a few hundred times in order to provide a suitable statistical basis for the position measurement. The signalling is initiated from the detector and the signal then oscillates freely between the measuring points by the received signal from the detector triggering the transmission from the transponder and vice versa. It will be possible to measure the time from transmission of the individual polling signal until receipt of the associated return signal, but in a preferred embodiment the detector clock takes the time from the point when the first signal in a signal sequence is transmitted until the last signal in the signal sequence is received. Combined with information from the counter in the detector which counts the number of transmitted signals in a signal sequence, this permits an averaging over the whole signal sequence each time the distance between a detector and a transponder is calculated.

The number of measurements required is proportional to the ratio between the accuracy of the measurement and the square of the desired accuracy. In practice this will mean that if the measurement per se has a resolution of 50 cm, and an accuracy of 5 cm is required, approximately 130 measurements will be necessary in order to obtain 95% (2σ) probability of having the desired accuracy. By reducing the accuracy of the measurements by a factor of 10 in this way, it will be possible to reduce the clock frequency and the transmitter frequency can be 600 MHz - 1 GHz.

In addition more transponders than is strictly necessary may be employed for calculating the angles. In a preferred embodiment an extra transponder is used. This extra transponder is used to provide redundancy and each time it is used in the calculations, normally every second, it will give an indication of the precision of the system.

As already described, the camera is equipped with transponders. A transponder will preferably be employed which utilises surface acoustic wave technology (SAW technology). A transponder of this kind is illustrated in figure 2. The transponder comprises a substrate 20, which, e.g., may be composed of a crystal such as lithium niobite which has a surface pattern of metal which constitutes transducers, reflectors, etc. A polling pulse from a position detector (1, 2, 3, 4, fig. 1) is received by the antenna (not shown) of a transducer 21, which is illustrated here in the form of a so-called interdigital transducer. The electromagnetic energy received in the polling pulse is converted in the transducer 21 to an acoustic surface wave which travels along the substrate's surface. Reflectors 22 are placed at a certain distance from the transducer 21. When the acoustic surface wave meets the reflectors, reflections are created which travel back towards the transducer 21. The transducer will convert the reflection waves to electromagnetic pulses which form the response signal which is transmitted via the transponder's antenna. At the ends of the transponder surface wave absorbers 23 may be provided in order to prevent undesirable reflections. The number and position of the reflectors 22 will be able to ensure that each transponder emits a unique return signal. When a detector receives a return signal after having transmitted a polling pulse, the distance to the transponder which has returned the return signal will be able to be determined from the time it takes from transmission of the polling pulse until receipt of the response signal, taking into account any delay in the transponder, while the transducer's identity can be established on the basis of the characteristics of the return signal.

In the event of approximately simultaneous detection of several response signals, in order to achieve unambiguous detection it may be expedient to use special detection techniques, e.g. based on correlation between detected, coded response pulse sequences and pre-stored code sequences for each individual transponder. Another possibility is to use polling signals on different frequencies and corresponding frequency-tuned transponders. Finally, transponders with different delays may be employed so that no transponders located in the area concerned will produce return signals which collide in time.

Figure 3 illustrates a camera equipped with five transponders 6, 7, 8, 9 and 10. These are mounted at the end of each arm in a preferably right-angled, isosceles cross which is attached to the camera and which is horizontal when the camera is horizontally oriented. The four detectors employed in this example (fig. 1) are capable of detecting the distance to each of the transponders 6, 7, 8, 9, 10. The five transponders each transmit a unique return signal, thus enabling them to be identified. The angular position of the camera involves a corresponding angular position of the cross, and on the basis of the calculated positions, a vector representation of the direction to each of the sides of the angle can be calculated by trigonometry. All the transponders' positions are known in relation to a reference point on the camera, and thus the camera's position and direction vector will be known in relation to a reference axis on the camera. The reference axis is preferably the central axis of the lens, i.e. the camera's optical axis.

Claims

PATENT CLAIMS

1. A system for registering a camera's position and orientation comprising:

- at least two transponders (6, 7, 8, 9, 10) arranged in known positions relative to a reference point on the camera (5), - at least two detectors (1, 2, 3, 4), designed to be capable of determining the distance from the respective detector to the said transponders,

- equipment (12) for, in a predefined coordinate system and based on data from the said detectors, calculating a position for the said transponders and a vector representing the direction for a reference axis on the camera, - communication connections (11) between the respective units in the system, characterized in that

- the detectors comprise equipment for transmitting and receiving radio wave signals, where the equipment is designed in such a manner that the receipt of radio wave signals triggers the transmission of new radio wave signals, thus providing a signal sequence, and that

- the distance between detector and transponder is determined by an averaging over the whole signal sequence.

2. A system according to claim 1, characterized in that each transponder (6, 7, 8, 9, 10) is a SAW chip (20) which supplies a unique response signal which enables the individual transponder to be identified.

3. A system according to claim 1, characterized in that it comprises at least four transponders (6, 7, 8, 9, 10) and that these are mounted at the end of each arm of a cross which is attached to the camera and which is horizontal when the camera is horizontally oriented.

4. A method for registering position and orientation of a number of transponders mounted in known positions relative to a reference on a camera, comprising the following steps:

- determining from at least two basic positions the distance to the transponders when the basic positions are known positions in a selected coordinate system,

- calculating a position for the transponders in the selected coordinate system,

- calculating a direction vector which represents the direction of a reference axis for the camera, characterized by - transmitting radio wave signals between the basic positions and the transponders, with the result that the receipt of a radio wave signal triggers the transmission of a new radio wave signal in a signal sequence, and

- calculating the distance to and the position of the transponders by means of averaging over the whole signal sequence.

5. A method according to claim 4 , characterized in that the direction vector is calculated on the basis of the position of at least two transponders by means of trigonometry.