WO2009071715A1

WO2009071715A1 - System and method for increased observation

Info

Publication number: WO2009071715A1
Application number: PCT/ES2008/000738
Authority: WO
Inventors: José Luis HERRERO JARABA; Jesús MARTINEZ DEL RINCON; Miguel Angel MONTAÑEZ LABORDA; Jorge Raúl GOMEZ SANCHEZ; Ignacio Julve Castro; Carlos ORRITE URUÑUELA; Francisco Javier Martinez Contreras
Original assignee: Universidad De Zaragoza
Priority date: 2007-12-03
Filing date: 2008-11-26
Publication date: 2009-06-11
Also published as: ES2324266B1; ES2324266A1

Abstract

The invention relates to a system for increased observation, enabling the improvement of the information received by a user observing an event, said system comprising: a means for capturing images located at the same point, said means capturing individual images of the event; a processing means provided with storage means, for linking the individual captured images in order to obtain a single mosaic image, and for storing all of the images; a transmission means for transmitting the individual captured images to the processing means; a display means comprising a processor, used by the user to view both the current images and the previously recorded images and to obtain additional analysis functions; and a communication means for communicating the images from the processing means to the display means.

Description

EXTENDED OBSERVATION SYSTEM AND PROCEDURE

D E S C R I P C I Ó N

OBJECT OF THE INVENTION

The main object of the present invention is to improve the point of view of a person who observes an event using cameras located in an optimal location. Another object of the present invention is to provide the user with additional analysis information about said event.

BACKGROUND OF THE INVENTION

When a person observes any event, they only receive the information available from their point of view and in real time. However, in certain cases it may be necessary for the user to receive additional information. For example, the situation of a football coach at the level of the pitch is very little advantageous for controlling the different positions of the players of his team. Such additional information would also be useful for the person in charge of operations in the case of military applications.

There are different methods that have been used to improve the observer's point of view of an event. One of the simplest consists of taking pictures and viewing them in real time on a screen or television. This is the case, for example, of television screens located in the boxes of football stadiums. However, although in this way the viewer's point of view is improved, he cannot decide which part of the field he wishes to visualize but must comply with seeing the part of the field decided by the director. In addition, frequently the image shows Only part of the pitch, which although it is the most relevant from the viewer's or filmmaker's perspective, may not be from the tactical point of view. In order to solve this problem, other systems have been developed, some of which are briefly described below.

Document US 6,952,558 describes a system that allows attendees to a sporting event, such as a car race, to choose through which camera they wish to follow the event at any time, using screens arranged in front of their seats. For this, a network of cameras and microphones is installed throughout the track and all the signals are sent to a control panel that each viewer can access via wireless protocol.

US 2004/0263626 describes a similar system, in which several cameras follow an event and users can choose which image to see through a network, such as Ethernet or the Internet.

DESCRIPTION OF THE INVENTION

The observation system and procedure of the present invention offers a single real-time image of the entire terrain on which an event is developed. It also incorporates, among other functions, the possibility of repetition of the actions that occur, which will be carried out at the request of the user and immediately. For this, the system uses image capture means where individual images of the different parts of the land are acquired from a more advantageous position than the one the user has. Subsequently, the system proceeds to the composition of a mosaic image that encompasses all of said land on which the event takes place. In the present invention, we will call "event" any event whose monitoring can be improved by using the system and method of the present invention. Some examples of events are sporting events (football, rugby, etc.), military operations or social events (demonstrations, concerts, etc.). The marked tactical nature of the activities to be monitored, or the need for an improvement in the visualization, for a correct follow-up and surveillance of the events that occur, make the use of the present invention in such situations especially recommendable.

In turn, the "terrain" refers to the place on which the event takes place physically. It may be a portion of the land itself, as in the case of military operations, the playing field of a stadium, as in the case of sporting events, the streets of a city, as in the case of demonstrations, etc.

The term "individual images" refers herein to each of the images of parts of land acquired by image capture means. The term "mosaic image" refers to the image of the complete terrain that forms the system by means of the union of all the individual images. Finally, the term "images" is intended to refer to the set formed by the mosaic image and the individual images.

Thus, according to a first aspect of the invention an expanded observation system is described to optimize the information received by a user who observes an event, characterized in that it comprises:

- image capture means, located at the same point, that capture individual images of the event; - a processing means, comprising processor and means of storage, which joins the individual images captured to obtain a mosaic image, and stores all images;

- a transmission medium, which transmits the individual images captured to the processing medium; - a visualization means by which the user visualizes the images and obtains additional analysis functions;

- a communication medium, which communicates the images from the processing medium to the display media.

Next, each of the means used to carry out the present invention is described, although it is understood that both the internal architecture and the design of each of them can vary depending on the complexity of the event or the terrain on which it is I developed.

Image capture media

The expanded observation system of the invention obtains the data from means of capturing images arranged in one or several strategic positions around the terrain on which the event takes place. The main objective is to offer a vision of the event much more advantageous than the one the user has. For this, the image capture means are installed, in general, in high places near the terrain on which the event takes place, thus avoiding the decrease of the visual field due to natural or artificial obstacles, and also in strategic places chosen by the user due to its special tactical importance. For example, in the case of sporting events, the ideal places would be roofs or roofs.

For the realization of the mosaic image a special arrangement of the image capture means is required. To avoid distortions, do not there should be horizontal displacement between them, but only a pure rotation around a vertical axis perpendicular to the plane of the terrain on which the event takes place. For this reason, they must be placed on the same vertical with the necessary rotation to completely cover the ground. It is also convenient that the difference in height between the image capture means be as small as possible. In this document, the expression "located at the same point" refers to this special provision.

The minimum level of quality that the image capture means must provide is determined by the type of event to which this extended observation system is applied. Thus, the image rate of the image capture means is usually about 15-30 images per second, close to the threshold of the human eye, while the resolution depends on the needs of the user.

Regarding the number, the number of image capture means necessary to cover the entire land on which the event is taking place is used. This number depends fundamentally on the following parameters:

- Focal length of the chosen lenses: The greater the focal length, more means of image capture will be required to cover the entire terrain on which the event takes place.

- Distance from the camera to the ground: The greater the distance, the lower the number of image capture means will be needed.

- Terrain size: The larger the plot size, the greater the number of image capture media. - Resolution required.

- Quality of the presentation.

According to a particular embodiment of the invention, the image capture means comprise digital video cameras, analog video cameras and infrared cameras.

Processing medium

The processing medium comprises processor and storage medium. Its function is to receive the individual images from the image capture media and join them together, creating a single mosaic image that covers the entire terrain on which the event takes place. For this, the processing medium transforms the individual images of the capture media to a common reference, which may be one of the image capture means used (in the case of an odd number of capture media) or a means of virtual capture positioned in the geometric center of the installed image capture media (in the case of even number). In addition, the processing means performs a temporary synchronization of the signals.

The processing means comprises storage means for recording the images from the capture means and the mosaic image obtained.

The computational load made by the processing medium depends on the number of image capture media and the size and cadence of the individual images captured. Depending on this load computational, and according to particular embodiments of the invention, the processor can be a computer, a microcontroller, an FPGA, a DSP, etc.

Transmission medium

It is responsible for the transmission of the individual images acquired by the image capture means to the processing medium, where the calculations are made to join all the individual images into a single mosaic image. You must have sufficient bandwidth to allow all images to be sent without loss of quality.

Thus, according to a particular embodiment of the present invention, the transmission means comprises optical fiber. However, it should be noted that depending on the bandwidth needs that are available, the transmission medium is not restricted solely to optical fiber.

Display media

A visualization means allows a user to view the current and already recorded images, either the mosaic image obtained or any of the individual images. In accordance with preferred embodiments of the invention, the display means comprises a television, a computer, a PDA and a mobile telephone. It is obvious that any of these devices can have any type of screen known, such as a TFT screen, a touch screen, an OLED screen, etc.

In addition, the display means offer additional analysis functions of the mosaic image. These additional analysis functions they depend on the quality of the mosaic image received in the display medium, quality which in turn depends on the bandwidth of the communication medium used to send said image from the processing medium.

When, for example, optical fiber is used as a means of communication, the visualization means receive the mosaic image with the original resolution and cadence, which allows to offer a greater variety of additional analysis functions.

However, if the bandwidth of the media requires that the processing medium compresses the mosaic image for sending, the additional analysis functions offered by the display means are smaller.

Media

A communication medium transmits the images from the processing medium to a viewing medium. Depending on the means of communication used, it may be necessary for the processing medium to compress the transmitted images, that is, decrease their cadence and / or resolution. In addition, the communication means allow transmitting communication signals in both directions between the processing means and the display means.

In accordance with preferred embodiments of the invention, the communication means comprise fiber optic, Internet, an Ethernet network, a LAN network, a WLAN network, a terrestrial telephone network, a GPRS mobile network, a UMTS mobile network, a Wi-Fi network Fi or a Bluetooth network. In addition, the processing medium can send the images to several display media at the same time, using different means of communication, as will be clear from the example shown later in this document.

On the other hand, and in accordance with a second aspect of the invention, an extended observation procedure is provided to optimize the information received by a user who observes an event, characterized in that it comprises the operations of: - capturing, by means of capture means located at the same point, individual images of an event. Normally, none of the individual images cover the entire field, but each of them comprises only a part of it. send said captured individual images to a processing medium through a transmission medium. The raw images are sent, that is, as they have been taken. join, by means of said processing means, the individual images captured in a single mosaic image, and store and compress all the images. - send, by means of at least one communication medium, the images to at least one visualization medium. There is the possibility of sending the images to several visualization means, through several communication means. visualize, through at least one visualization means, the desired images and use additional analysis functions.

In addition, depending on the media used to send the images to the display media, it may be necessary to decrease the cadence and / or resolution of the images. We will call this operation "compress" the images. Therefore, according to a embodiment of the invention, the extended observation procedure further comprises the operation of compressing the images in the processing medium.

In accordance with a preferred embodiment of the invention, the additional analysis functions comprise at least one of the following options: incorporate comments to the images, incorporate indexes to the images, request the visualization of specific parts of the terrain, request the visualization of concrete previous moments of the images, draw on the images, etc.

DESCRIPTION OF THE DRAWINGS

To complement the description that has been made and in order to help a better understanding of the characteristics of the invention, according to a preferred example of practical implementation thereof, a set of drawings is attached as an integral part of said description. where, for the purposes of illustration and not limitation, the following has been represented:

Figure 1.- Shows a general scheme of the extended observation system according to a preferred embodiment of the present invention.

Figure 2.- Shows a possible arrangement of the chambers according to a preferred embodiment of the present invention.

Figure 3.- Shows another possible arrangement of the chambers according to another preferred embodiment of the invention.

Figure 4.- Shows the location of the different elements that make up the system of the present invention applied to the visualization of A soccer match.

Figure 5.- Shows the field of vision of a realization of three cameras applied to a football match.

Figure 6 and Figure 7.- They show the way in which the images taken by the cameras of Figure 5 are joined.

Figure 8 and Figure 9.- Show possible visualization means according to preferred embodiments of the present invention.

PREFERRED EMBODIMENT OF THE INVENTION

A preferred embodiment of the invention applied to the visualization and analysis of a football game is described below, the general scheme of which is shown in Fig. 1. In this case, the user will be the coach, whose situation at ground level ( within the "technical area") does not offer an adequate point of view to follow the evolution of its players. To solve it, the image capture means are three cameras (2, 3, 4) installed in the central part of the stadium cover, as shown in Ia

Fig. 4. The cameras (2, 3, 4) are located in accordance with the arrangement shown in Figs. 2 and 3, that is, located as close as possible to each other and with a relative rotation between them, so that each of the individual images captured (14, 15, 16) corresponds to a part of the playing field (11 ). In this example, the cameras (2, 3, 4) used have mega-pixel resolution in RGB format, with an image rate of 30 images per second.

In addition, in Fig. 4 the situation of the processing means (8) is represented, which in this case comprises a computer of processing (6) and a RAID storage system (7), and Ia of the transmission medium (5) of the individual images captured (14, 15, 16) from the cameras (2, 3, 4) to the processing computer (6), which in this example is a fiber optic cable (9). Thus, the processing computer (6) performs the necessary calculations to join the individual images captured

(14, 15, 16) by the three chambers (2, 3, 4), represented in Fig. 6, creating a single mosaic image (17), as shown in Fig. 7. Both the mosaic image (17 ) as the individual images (14, 15, 16) captured by the cameras (2, 3, 4) are stored in the RAID system (7).

The coach, within the technical area, has visualization means (10), in the example a computer with a touch screen (12) shown in Fig. 8, from which he can decide, for example, which part of the playing field (11) wishes to observe or in which to incorporate annotations or indexes to the images recorded on disc (7) to then analyze the game with its players.

Communication between the processing computer (6) and the viewing computer (12) installed in the technical area is carried out by means of a fiber optic cable (9), which constitutes one of the communication means, and provides bandwidth enough to transmit the images without reducing their quality. A processor (not shown in the figures) included in the display means, in this case the display computer (12), provides the additional analysis functions, which offer the trainer a wide range of options represented by means of active elements ( for example, buttons or editing fields) that allow the coach to interact with the images shown. Some of the possible functions could be:

- Incorporate comments: the coach can incorporate annotations linked the moment of recording time, which will appear in the moments of the event in which they were created.

- Incorporate indexes to the recording: the coach can mark parts of the event of special interest, so that they can be easily found.

- Request of recorded images: the trainer can visualize a recorded image while the processing medium (8) continues to receive the individual images captured (14, 15, 16), forming the mosaic image (17) by means of the processing computer (6) and storing all images on RAID disk (7).

- Areas of interest and zoom: in addition to observing by default the entire terrain (11), the coach can visualize specific areas of the terrain (11).

- Electronic board: the coach can use the recorded images as a support to incorporate tactical indications.

On the other hand, in the present example there is also an assistant coach who watches the game from the stands. From that place, you also have access to the images thanks to your mobile phone (13), which in this case constitutes the display medium, while the communication medium is the GPRS network that uses your mobile phone (13), represented in Fig. 9. Because the bandwidth is not sufficient to allow the communication of the images with all their quality, the processing medium (8) "compresses" (decreases the rate and / or resolution) the images before to send them to the mobile phone (13). This loss of quality also implies a smaller number of additional analysis functions.

Claims

1. Extended observation system (1) to optimize the information received by a user who observes an event, characterized in that it comprises means for capturing images (2, 3, 4) located at the same point, which capture individual images (14, 15, 16) of the event; a processing medium (8), comprising processor (6) and storage medium (7), which joins the individual images (14, 15, 16) captured to obtain a mosaic image (17), and stores all the images; a transmission medium (5), which transmits the individual images (14, 15, 16) captured to the processing medium (8); a visualization means (10), by means of which the user visualizes the images and obtains additional analysis functions; a communication medium (9), which communicates the images from the processing medium (8) to the display means (10).

2. Extended observation system (1) according to claim 1, characterized in that the image capture means (2, 3, 4) comprise at least one device from the following list: digital video cameras, analog video cameras and infrared cameras.

3. Extended observation system (1) according to claim 1, characterized in that the processor (6) comprises at least one device from the following list: a computer, a microcontroller, an FPGA and a DSP.

4. Extended observation system (1) according to claim 1, characterized in that the transmission means (5) comprises optical fiber (5).

5. Extended observation system (1) according to claim 1, characterized in that the display means (10) is chosen from the following list of devices: a television, a computer, a PDA and a mobile phone.

6. Extended observation system (1) according to claim 1, characterized in that the communication medium (9) is chosen from the following list: optical fiber, Internet, an ethernet network, a LAN network, a network

WLAN, a landline telephone network, a GPRS mobile network, a UMTS mobile network, a Wi-Fi network and a Bluetooth network.

7. Extended observation procedure to optimize the information received by a user who observes an event, characterized in that it includes the operations of: capturing, by means of image capture (2, 3, 4) located at the same point, images individual (14, 15, 16) of an event; sending said individual images (14, 15, 16) captured to a processing medium (8) through a transmission medium (5); joining, by said processing means (8), the individual images (14, 15, 16) captured in a single mosaic image (17), and storing all the images; send, by means of a communication medium (9), the images to a visualization medium (10); visualize, by means of at least one visualization means (10), the desired images and use additional analysis functions.

8. Extended observation procedure (1) according to claim 7, further comprising the operation of compressing the images in the processing medium (6).

9. Extended observation procedure according to claim 7, wherein the additional analysis functions comprise at least one of the following options: incorporate comments to the images, incorporate indexes to the images, request the visualization of specific parts of the terrain ( 11), request the visualization of specific previous moments of the images and draw on the images.