GB2437773A

GB2437773A - Image capture control using identification information via radio communications

Info

Publication number: GB2437773A
Application number: GB0608877A
Authority: GB
Inventors: Nicholas Theodore Taptiklis
Original assignee: Individual
Current assignee: Individual
Priority date: 2006-05-05
Filing date: 2006-05-05
Publication date: 2007-11-07
Also published as: GB0608877D0

Abstract

A method of controlling an image capture device to capture an image of an object, such as a person or animal 505, carrying a radio communications device and using device identification information from the radio communications device, comprises receiving identification information corresponding to the radio device to identify when the object is in view of the imaging device 503, capturing an image of the object in response and then outputting the image. The captured image data, which can be part of a sequence of images, may be sent to the object carrying the radio device. Image capture may be in response to registered identification (ID) information, and further dependent on distance or direction. The ID information may be a header of a packet based stream. Radio communications may be via a mobile telephone (Figure 4).

Description

Imaae Capture Control System This invention relates to an image capture

control system, in particular to methods and apparatus for controlling an image capture device using information from a wireless communication system.

Image capture devices such as digital cameras are known in the art for taking pictures, particularly digital photographs and movies of scenes, people and events. They are usually configured for manual operation, requiring an individual person to operate their controls in order to take pictures. This may prove inconvenient and/or expensive, especially for taking large numbers of pictures on a continuous basis. Automatic systems are known having timers for taking sequences of pictures at predetermined intervals, but this may produce large numbers of images, not all of which have useful content, and these require manual sorting to determine the content of each one and whether or not it contains any useful content EP 1480450 describes a method for producing video coverage of an event by automatically switching between available camera views based on data gathered from the event. Sources of data include the locations of participants derived from tracking cameras and image analysis, and automatically updated scoring information in the case of sports coverage.

However, the system described is extremely computationally expensive and is not suited to implementation on a large scale. It may be used, for example, by broadcasting organisations for televising sporting events, where the initial capital cost of the equipment is not prohibitive. Furthermore the image analysis techniques described are still in their infancy, and may not be very good at distinguishing large numbers of different people or objects, particularly ones which look very similar.

It would be advantageous to provide a system for automatically capturing images of people or objects, associating the images with the particular people or objects, and making the images available to those people or objects. This would prove particularly advantageous at sporting events, for example sporting events having many competitors such as skiing, mountain biking, cross-country running or cycling, as it would be possible to generate content associated with each individual competitor and make it available to them, for example for a fee, or to others interested in seeing content associated with them.

According to an aspect of the present invention, there is provided a method of controlling an image capture device to capture an image of an object carrying a radio communications device using device identification information from the radio communications device, the method comprising receiving identification information corresponding to the radio communications device using a radio communications receiver to identify when said object is in view of the image capture device, capturing an image of said object using said image capture device responsive to receiving said identification information, and outputting image data corresponding to said image.

The term radio' is used here for clarity; it will be understood by one skilled in the art that any wireless communication system could be used. The object may comprise people or animals, or any article carrying the radio communications device. Particularly advantageously the object may be in motion. Image data may be sent to the object using the radio communications device, or by using another communications device also associated with the object. For example, the radio communications device may be a Bluetooth transceiver, but the image data may also be sent to the object via another communications method such as GSM or 30. Alternatively the radio communications device may comprise an RFID tag.

The device identification information may be registered with the system before image capture, so that image capture may take place only for registered devices. The device identification information may comprise data in a header of a packet-based data communications stream, for example Bluetooth or WiFi (802.11 a/big wireless Ethernet). Image capture may take place only once the radio communications device is within a certain distance of the radio communications receiver, for example 10 metres.

Image capture may also take place only when the radio communications device is in a certain direction relative to the radio communications receiver. The radio communications receiver may have a directional antenna such as a Yagi array for this purpose. Image capture may take place in the same direction as the antenna, so that (for example) the field of view of image capture substantially overlaps the region in which the radio communications receiver is most sensitive.

Alternatively, the system may use more than one radio communications receiver, and determine a location of the object using signals from the plurality of radio communications receivers. Phase array antennae may be used, optionally with beam steering to concentrate the signal. This allows the camera locations to be decoupled from the antenna locations, and allows the use of multiple cameras. The base station may determine when an object is in view of a particular camera and record image information coming from that camera. The cameras need not receive any control signals; they may send image information continuously and the base station may choose to store it or discard it according to object location information determined from the radio communications receivers.

Image capture may comprise capturing a sequence of images, for example a movie' or motion picture file. Data may be provided in a format such as MPEG (1, 2 or 4), or another video format. Still image data may be provided in JPEG format or another image format. The radio communications device may comprise a mobile telephone, for example a mobile telephone having a Bluetooth interface, or another device such as a personal digital assistant (PDA). Several image capture units may be provided, the method taldng place at each unit. Additionally, the units may send the image data to a base station for further processing and/or sending to the radio communications device as a single data unit.

The invention further provides processor control code to implement the above-described methods, for example on a general purpose computer system or on a digital signal processor (DSP). The code may be provided on a carrier such as a disk, CD-or DVD-ROM, programmed memory such as read-only memory (Firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and/or data) to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code. The above described methods may also be implemented, for example, on an FPGA (field programmable gate array) or in an ASIC (application specific integrated circuit). Thus the code may also comprise code for setting up or controlling an ASIC or FPGA, or code for a hardware description language such as Verilog (Trade Mark), VHDL (Very high speed integrated circuit Hardware Description Language), or RTL code or SystemC. Typically dedicated hardware is described using code such as RTL (register transfer level code) or, at a higher level, using a language such as C. As the skilled person will appreciate such code and/or data may be distributed between a plurality of coupled components in communication with one another.

According to another aspect of the present invention, there is provided a system for controlling an image capture device to take photographs of people or animals carrying a radio communications device using device identification information from the radio communications device, the system comprising a radio communications receiver for receiving identification information corresponding to the radio communications device to identi& when said person or animal is in view of the image capture device, a controller for controlling said image capture device to capture an image of said person or animal responsive to receiving said identification information, and an output for outputting image data corresponding to said image.

The system may implement the above described methods; in particular it may capture images of any object carrying a radio communications device, including people or animals.

According to a further aspect of the present invention, there is provided a method of taking photographs of an object responsive to receiving device identification information from a radio communications device carried by the object.

The system enables fully automatic production of video footage of registered individuals from multiple cameras in a location, by means of continuously scanning for the Bluetooth id's transmitted by Bluetooth-enabled mobile phones and using this information to sequence multiple clips from several fixed cameras into a single sequence of sports-broadcast style video, which is then transmitted to the participant's Bluetooth device.

Examples of locations especially suited to this system are ski fields and mountain bike tracks, where sports participants progress on a defined path, allowing for multiple cameras to be installed on the route.

The benefits of the system will be in allowing users to view their own skiing, snowboarding, cycling etc for training or entertainment purposes, and to enable fully automatic continuous competitions at the venue, as the Bluetooth id tracking can be used to time the participant's journey. Top scoring individual runs could then be automatically displayed on video screens around the facility, and additionally, the Bluetooth tracking system could detect when a top scoring individual enters a certain area (such as bar facilities at the location) and then replay the video footage on a large screen.

By taking advantage of Bluetooth phones, the system also allows for fully automatic registration, as the user can use the standard Bluetooth system for pairing' with other devices to pair' with the video system. This pairing will register the user, and trigger video production and delivery to their phone at the completion of each run.

Furthermore, wireless tracking for video production is also possible using other device types, such as RFJD and OSM.

These and other aspects of the invention will now be described in further detail, with reference to the accompanying drawings, in which: Figure 1 shows a process diagram according to an aspect of the present invention.

Figure 2 shows a camera and antenna assembly according to an aspect of the present invention.

Figure 3 shows superimposed antenna range and camera view range according to an aspect of the present invention.

Figure 4 shows a network overview according to an aspect of the present invention.

Figure 5 shows an alternative overview according to an aspect of the present invention.

Figure 6 shows an exemplary bJock diagram of a capture unit according to an aspect of the present invention.

A system according to an embodiment of the present invention consists of a base station for user registration, video production and delivery, and a series of capture devices distributed at the location.

Figure 1 shows a process diagram illustrating user interaction with the system including registration. It describes the process of user registration and interaction with the system, in a simple example where the video service is provided for free to skiers by the ski resort as part of their enhanced services. Registration may be initiated by the user pairing with the system using the Bluetooth interface implemented by their phone, as illustrated, or it may be initiated by the system continuously scanning for new devices, and then initiating pairing with the newly detected phone. The user can then accept or decline the pairing, and be registered with the system accordingly.

Alternative registration processes can be used if the service is to be charged for separately. One method is to provide users with a scratch-card with a bar code, or unique number. The user is required to capture and image of the scratch card using a camera-equipped Bluetooth handset, and then send the image to the base station. Image processing on the base station decodes the bar code or unique number within the image, checks the validity of this number by means of database lookup or some form of checksum algorithm, and registers the user if valid.

An alternative registration and billing technique is for the user to pair with the base station, and receive an application written in a format compatible with the handset operating system via Bluetooth, to install and run to complete the registration process.

This application then enables the user to pay for video clips via premium rate text messages. This method has the additional benefit of providing an enhanced user interface for managing the users interaction with the system, including previewing short or low-resolution video clips before purchase, changing background music, toggling between cameras where more than one camera is available, and requesting video clips to be sent to other users of the system.

After registration, the user can commence the sporting activity. However, in order for the system to be able to track them, the user's Bluetooth enabled handset must be detectable by the system. This can be achieved by the user using the phone operating system to toggle their device to a discoverable' state, or alternatively, if an application has been installed on the handset as part of the registration process, the application can either toggle the discoverable state automatically if allowed by the handset, or alternatively the handset can initiate the discovery process in the background to ensure the handset will be detectable by each camera node in the system.

Each capture devices consists of one or more Bluetooth detectors, and one ore more cameras aimed at the same scene. In a simple example there is a single camera and detector in the same physical device. The detector is a device that is capable of detecting a Bluetooth id with additional directional information.

This may simply be a directional (Yagi or other) antenna focused in parallel to the camera. In this configuration, the base station device is simply sent the time the users Bluetooth device enters and leaves the field of view' of the Bluetooth detector, and this time information is used to cut the appropriate video clip from the video capture stream of the corresponding camera.

The camera implementation may be a high-speed CCD or CMOS camera, with some high speed data output format such as USB2.O, Firewire or CameraLinic, or alternatively may be interfaced with a FrameGrabber board in a microcontroller included as part of the assembly.

Figure 2 shows an example camera and antenna assembly with Yagi antenna (I), CCD camera (2), and microcontroller (3) to interface with antenna and camera and relay data streams to the base station. It illustrates an example implementation of this method, with a CCD (or CMOS) camera (1) focused in the same direction as the antenna (2), with both devices interfaced to a microcontroller that streams the captured video and Bluetooth data to the base station, by means of a transport network.

The transport network may be implemented by multiple means, provided that it has the capacity to transport the video footage. It may be wired Ethernet or fibre network, or a form of wireless network. If wireless connections are used, the performance may be enhanced by means of directional antennae. It can also be implemented by means of point-to-point laser communication.

Figure 3 shows a superimposed antenna range (4) and camera field of view (5). Region 6 is the effective video capture area for the unit. The Figure illustrates an exemplary network with capture units (1) connected via a daisy chain of point-to-point wireless links. The video and Bluetooth timing data is transmitted to the base station (2) workstation that edits and formats the clip which is sent to the phone via Bluetooth antenna at the base station area, or alternatively, located near ski lift queues.

Alternatively, the Bluetooth detecting devices may be phase-array multi-antenna systems that can detect multiple devices in multiple directions simultaneously. In this configuration the detectors may not necessarily be co-housed with the cameras, but deployed in a mesh across the location, in order to triangulate the location of the phones. The base station software is configured with a map of the location including the field of vision of each camera, and uses this map to determine which video streams to use in the assembled sequence.

Figure 4 shows a network overview with capture units (1), base station rendering server (2) and base station wireless unit (3) for transmitting the mixed and formatted video footage to the phone.

In a simple example the base station software assembles a continuous sequence of action footage of one individual by splicing clips from each camera the individual parses using the Bluetooth id timing data.

The video data may be converted to an intermediate digital format (such as MPEG) at the capture units, and sent to the base station by standard networking data protocols (TCPIJP or ATM for example).

By using standard vision processing techniques in conjunction with the timing data, the system will be able to improve the quality of the product. For example, ski footage is generally on a convenient white background ideal for using detection of clothing colour to detect the exact location of the individual within each video frame.

An example technique to perform this analysis is to use standard threshold techniques to eliminate camera noise, and detect rectangular movement blocks within each frame.

Multiple blocks of movement are likely within each frame, corresponding to different skiers. Each block of movement is assigned a colour profile based on the average colour of the top and bottom half of the image (corresponding to jacket and trousers). The colour profile can be normalized with respect to the background colour of the snow to account for changing light levels, in order to improve colour profile matching between units as the user passes from camera to camera. The system then determines the best matching motion block representing an individual passing through the field of vision of each unit, by correlating the timing data associated with colour-profiled motion blocks and the Bluetooth timing data.

This allows the system to implement a software zoom facility, so the produced clip appears to have been produced from a moving camera tracking the user rather than a single wide-angle camera.

Alternatively, the system may include additional motorized tracking and zooming cameras that are guided by the motion tracking in order to provide additional higher quality physical zooming and panning video footage.

The user may also be able to register a designated ski-instructor who will receive additional copies of the video clips on their phone.

Additionally, the system may use timing data, and vision processing techniques to select appropriate music indicative of the speed and ability of the skier, and mix this with the video. Skiing errors may be detected by recording a sudden inversion about the X axis of trouser and jacket color within a motion block, and accompanied by amusing sound effects on the produced video clip.

The base station may use manufacturer code information embedded in Bluetooth devices to determine the exact device type, and format the video sequence using a video encoding format compatible with the device, and when the user completes their run, the base station may use standard Biuetooth data transfer techniques (Obex-push, ftp etc) to download the assembled clip to the users phone.

Figure 5 shows an overview of an alternative embodiment of the present invention. In this embodiment, multiple Bluetooth receivers (501) are used to determine locations of Bluetooth-enabled devices. This may be achieved using triangulation methods as known in the art. The antennae of the receivers may be phase array antennae. The server (502) receives information from the Bluetooth receivers and calculates or otherwise determines locations of Bluetooth-enabled devices in range of the receivers.

Determining the distance of a device from each receiver allows its position to be determined. With two distance measurements from two receivers there are two possible locations for the device; a third distance measurement limits the possible locations to one, although with careful positioning of the receivers the alternative' locations may not be possible in practice, for example if the receivers are located on an edge of the site. A variant of this system uses phase-array antennae to determine the direction of the device from each receiver, thereby allowing the position of the device to be determined by determining where along lines extending from each receiver in the direction of the device the lines cross.

Camera (503) may be situated anywhere within the range of the Bluetooth receivers, and its location may be entered into the server. There need not be any intelligence in the camera; it may simply capture images and output image data continuously, and the server may decide whether to use image data from the camera at any instant or whether to ignore it. Alternatively, to reduce traffic on the network between cameras and the server (which may be either a wired network or a wireless network), the server may command the camera to output image data only at particular times. The Figure shows an example embodiment having only one camera, with a wired link between the base station and the camera and wireless links between the Bluetooth receivers and the base station, but as will be appreciated by one skilled in the art, any number of cameras could be utilised in this way, using any communications system. Output device (504), for example a television display in a club house at a ski resort, may display captured image data from the cameras, and with a two-way communications link between the output device and the server, images may be chosen from selected Bluetooth devices (corresponding to particular users of the system carrying those devices).

In this example, one person (505) is not in view of the camera. Although the Bluetooth receivers may receive the Bluetooth ID from the Bluetooth device he is carrying, the server determines from his position that he is not in view of the camera and does not associate captured images from the camera with his Bluetooth ID. On the other hand, another person (506) is in view of the camera. The server determines from her position that she is in view of the camera, and selects image data coming from the camera for outputting with her Bluetooth ID. This may be displayed on the display along with her name, for example as a subtitle.

Figure 6 shows a block diagram of an exemplary capture unit according to an embodiment of the present invention. This may comprise a conventional computer with a Bluetooth transceiver and a digital camera attached, and software running on the computer to carry out the method steps described hitherto. The Bluetooth transceiver and digital camera may be connected directly to the control bus, for example using a PCI interface, or they may be connected via standard ports such as USB, serial RS232 or the parallel (printer) port. Bluetooth and camera control software may be provided separately from the image capture application, for example as standard device drivers for these devices, or the entire executable may be provided as a single application.

The network base station may also be implemented using architecture of this type. It may not need a digital camera interface, but a Bluetooth transceiver may be provided for sending image data to the Bluetooth device after image processing. Alternatively, one of the capture units may also function as the base station. In highly optimised designs in which the bill of materials for each capture unit is reduced to a minimum, one or more of the standard interfaces shown in Figure 5 may be omitted (for example the audio interface, keyboard & pointing device interface and graphics interface). The capture units may also be implemented using custom logic and/or FPGAs.

No doubt many other effective alternatives will occur to the skilled person. It will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.

Claims

CLAIMS: 1. A method of controlling an image capture device to capture

an image of an object carrying a radio communications device using device identification information from the radio communications device, the method comprising: receiving identification information corresponding to the radio communications device using a radio communications receiver to identify when said object is in view of the image capture device; capturing an image of said object using said image capture device responsive to receiving said identification information; and outputting image data corresponding to said image.

2. A method according to claim 1, wherein said outputting comprises sending the image data to the object carrying the radio communications device.

3. A method according to claim 2, further comprising using the radio communications device to receive said captured image data.

4. A method according to any preceeding claim, the method further comprising registering identification information corresponding to the radio communications device, said capturing being responsive to receiving said registered identification information.

5. A method according to any preceeding claim, wherein said identification information comprises a header of a packet-based data communications stream.

6. A method according to any preceding claim, said capturing being further responsive to said radio communications device being within a predetermined distance of said radio communications receiver.

7. A method according to any previous claim, said capturing being further responsive to said radio communications device being in a predetermined direction of said radio communications receiver.

8. A method according to claim 7, wherein a direction of said capturing comprises said predetermined direction of said radio communications receiver.

9. A method according to any preceeding claim, wherein a range of said radio communications receiver substantially overlaps a field of view of said image capture device.

10. A method according to any preceding claim, wherein said capturing an image comprises capturing a sequence of images.

11. A method according to any preceding claim, wherein said radio communications device comprises a mobile telephone.

12. A method according to any preceding claim, wherein a radio communications system of said radio communications device comprises Bluetooth.

13. A method according to any preceding claim, wherein said capturing comprises capturing a plurality of images using a plurality of image capture devices and said outputting comprises sending image data corresponding to said plurality of images over a network to a base station.

14. A method according to any one of claims I to 12, wherein said receiving identification information comprises receiving identification information from said radio communications receiver and receiving distance or direction information from a plurality of radio communications receivers, and using said distance or direction information to determine whether said identified object is in view of said image capture device.

15. Processor control code for implementing the method of any one of claims Ito 14.

16. A carrier carrying the processor control code of claim 15.

17. A system for controlling an image capture device to take photographs of people or animals carrying a radio communications device using device identification information from the radio communications device, the system comprising: a radio communications receiver for receiving identification infonnation corresponding to the radio communications device to identify when said person or animal is in view of the image capture device; a controller for controlling said image capture device to capture an image of said person or animal responsive to receiving said identification information; and an output for outputting image data corresponding to said image.

18. A system according to claim 17, wherein said person or animal is in motion.

19. A system according to claim 17 or 18, wherein an antenna of said radio communications receiver comprises a directional antenna.

20. A system according to claim 19, wherein a direction of said directional antenna and a direction of image capture of said image capture device comprise substantially the same direction.

21. A system according to any one of claims 17 to 20, wherein a range of said radio communications receiver substantially overlaps a field of view of said image capture device.

22. A system according to any one of claims 17 to 21, the system further comprising a network interface for sending said image data to a base station.

23. A method of taking photographs of an object responsive to receiving device identification information from a radio communications device carried by the object.