JPH09107561A - Automatic selection device of two cameras, method of automatic selection of two cameras and its application - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the device and method selecting automatically signals of two cameras from a plurality of camera signals generated from n-sets of cameras that pick up one scene from a plurality of scenes depending on observation of a scene in existence on a receiver with a simple method. SOLUTION: A head tracing unit KVE decides a head position Pj of a viewer B and selects two camera signals and makes a request of the selection to a transmission unit SE. Then the cameras on request via a switch unit S are connected to a stereoscopic photograph coding unit SC, in which a video data flow CV is generated. Then it is sent to a receiver unit EE and decoded and displayed on a stereoscopic photographic image screen DIS.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device and a method for automatically selecting two camera signals from a plurality of camera signals by determining the position of an observer's head.

[0002]

BACKGROUND OF THE INVENTION During the transmission of so-called multi-view stereo image sequences in which more than two cameras capture one scene, there are numerous problems in transmitting the camera signals representing the captured scenes. In a multi-view stereo image sequence, the receiver requires two camera signals that represent the same scene from different scenes (viewpoints). Due to the existing dissimilarity of the images of the captured camera signal, i.e. the positional displacement of the corresponding pixels between the images of the two camera signals displayed on the stereoscopic image screen at the receiver, the depth effect for the observer That is, the observer has the impression that he is observing a three-dimensional image.

Given that a scene is filmed by multiple cameras and all camera signals from all cameras are transmitted to the receiver and the two required camera signal selections are made at the receiver. , Would require a very large transmission capacity that would include an unnecessarily large amount of redundancy.

In the receiver unit, only two,
Moreover, it is desirable to transmit only the two camera signals that are actually needed, since the "correct" camera signal is needed for the observer. Furthermore, it is desirable to be able to change the scene (viewpoint) of the scene without having to incur significant cost for this.

Further, European Patent Application Publication No. 0653
From publication 01, it is known to use more than two cameras in stereoscopic photography. Furthermore, DE 40 27 471 relates to the spatial display of the image of the observer depending on the position detected by the sensor.

[0006]

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to make a single scene into a plurality of scenes (viewpoints) by a simple method.
It is an object of the present invention to provide an apparatus and method for automatically selecting two camera signals from a plurality of camera signals generated by n cameras, which are present at a receiver, by an observer of a scene.

[0007]

This object is achieved according to the invention by the device according to claim 1 and by the method according to claim 3.

The advantages of the device according to the invention are inter alia: the device is very simple and therefore a stereoscopic video data stream based on two fixedly connected cameras capturing one scene. That is, only a small amount of expansion is required compared to the device that is required anyway to realize

The method according to the invention is also characterized by its simplicity, which makes it possible to carry out the method according to the invention very quickly, which makes it possible to carry out the method according to the invention in real time. When selecting another scene (viewpoint) of the scene to be taken, there is no delay effect which is an obstacle to the observer.

The use of the method of the present invention according to claim 4 is as follows:
A so-called fallback solution when using a model-based coding method for fault handling, i.e. coding of a video data stream, which makes the model-based coding method highly reliable. To be enhanced.

Embodiments of the invention are described in the other claims.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION Next, an illustrated embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows a device according to the invention having at least the following components: a transmission unit SE having at least the following components: one scene in several views, ie different. Multiple cameras Kai shooting from different viewing angles. The first index i that uniquely identifies each camera is a number lying in the region between 1 and n, the values 1 and n being included in this region.

A switch unit S having n input sides and two output sides, each input side of the switch unit S being connected to one camera Kai.

A stereoscopic picture coding unit SC for coding a video data stream CV to be coded and sent.

Receiving unit EE having at least the following components: Stereoscopic decoding unit SD for carrying out the decoding of the received coded video data stream CV.

A head tracking unit KVE for determining the head position Pj of the observer B. Here, from the given coordinates of the head position, the two required scenes, ie the two required camera signals Ki, are selected, the first index i being taken out from the i-th camera Kai. Also referred to as a camera signal, the second index j for identifying the respective head position Pj of the observer B is 1 and n-1.
Is a number in the region between and and the values 1 and n-1 are contained within this region.

Decoded video data stream DC
A stereoscopic picture screen DIS which displays V to the viewer B so that the viewer B has a stereoscopic effect, ie an impression of depth when observing the image.

The camera Kai photographs the scene SZ. The cameras Kai are arranged by an arbitrary method.

This means that the two cameras Kai and K
It is a premise to be able to realize a stereoscopic effect for the observer B in any choice of ai + x, where the number x lies in the regions 1 to n-1.

The transmission unit SE comprises at least n cameras Kai, a switch unit S and a stereoscopic picture coding unit SC.

The switch unit S has two outputs connected to the stereoscopic picture coding unit SC. The construction of the stereoscopic picture coding unit SC is known to those skilled in the art, for example (R. ter Horst, A Demonstrator for
Codings and Transmissionof Stereoscopic Video Sig
nals, Proceedings of Steroscopic Video Signals, Pro
ceedings of 4-th European Workshop on Three-Dimens
ional Television, Rom, 20.- 21. October 1993,
Pp. 273-290).

Video data stream CV coded in the stereoscopic picture coding unit SC to be transmitted
Are transmitted to the receiver unit EE. Each of the n input sides of the switch unit S is connected to one camera Kai.

According to the camera request information KAN described below, in the switch unit S, the cameras Kai and Kai + requested by the camera request information KAN.
1 is selected so that the cameras Kai and K are selected each time.
The requested camera signals Ki and Ki + 1 of ai + 1 are coded and transmitted.

Therefore, always only the required camera signals are selected and connected through the stereoscopic coding unit SC (3 in FIG. 2), where they are coded in the coded video data stream CV (in FIG. 2). 4) and transmitted to the receiver unit EE.

The coded video data stream CV is received and decoded at the receiver unit EE (5 in FIG. 2). Subsequently, the decoded video data stream is displayed to the viewer B on the stereoscopic picture screen DIS.

The receiver unit EE comprises at least a stereoscopic picture decoding unit SD and a head tracking unit K.
VE and stereoscopic image screen DIS (6 in FIG. 2)
have.

The stereoscopic picture decoding unit SD is connected to the stereoscopic picture screen DIS. In the head tracking unit KVE, the head position of the observer B photographs and detects the head position of the observer B by another camera and determines the spatial position of the head of the observer B, that is, the head position Pj of the observer B. It is determined by

Based on the head position Pj (the second index j is in the area between 1 and n-1), the camera signals Ki and Ki + 1 corresponding to the head position Pj are obtained.
Is selected and requested by the head tracking unit KVE.

In the head tracking unit KVE, the head position Pj of the observer B is determined (1 in FIG. 2) and the movement of the observer B in the horizontal direction with respect to the image screen is tracked. For this purpose, at least one camera is used which images the head of the observer B. The head tracking unit KVE is, for example, the German patent application 1951.
It operates according to the method described in 6664.7.

The method described therein determines the spatial position relative to the object to be imaged, which is classified during the period of the method described therein. This method is the same as that of the observer B in the method of the present invention.
Used to detect the head and track the movement of the head. As a result, the German Federal Patent Application No. 1
The method described in the specification of 95166664.7 is
The spatial position of the head of the observer B on the stereoscopic image screen DIS is determined.

However, in the present invention, only the coordinates representing the horizontal movement are used. Two separate spatial coordinates are not important to the method of the invention and are not needed for this reason.

FIG. 2 describes a method for selecting the two required camera signals Ki and Ki + 1.

First, two allowable limit positions with respect to the head position Pj of the observer B, that is, the head position P1 and the head position Pn-1 are defined.

For the transmission that is faithful to the desired nature, the camera K
It is advantageous to install ai at equal intervals at eye intervals, ie in this case head position P1 and head position P
The distance between n-1 should be the same as the camera spacing between camera Ka1 and camera Kan-1. Then, depending on the position of the observer B, the required camera signal is selected and transmitted.

The observer's head is at the head position P1,
That is, in the arrangement shown in FIG. 1, when on the leftmost or further left, the camera produced by the head tracking unit KVE by the two camera signals K1 and K2, ie the two cameras Ka1 and Ka2 on the left. The signal is selected (2) and requested by the head tracking unit KVE.

The camera signals K1 and K2 are not selected,
For example, the camera Ka1 and the other camera Ka1 + x positioned on the leftmost side and the camera signals K1 and K1 + x generated by these cameras can also be selected. Camera selection is always
It depends on the configuration of the stereoscopic picture screen DIS.

When the head of the observer B is at the head position Pn-1 or on the right side thereof, the head tracking unit KV
E is the camera signal Kn-1 and K at the head position Pn-1.
n and assign these camera signals to the transmitting unit S
Request from E.

The camera signals Kn-1 and Kn are not selected, but it is also possible to select, for example, the leftmost camera Ka1 and another camera Ka1 + x and the camera signals Kn and Kn-x produced by these cameras. It is also set to be possible. The choice of camera again always depends on the configuration of the stereoscopic picture screen DIS.

The head of the observer B has a separate head position Pj.
In the meantime, the head position Pj of the observer B, which is the closest to the actual head position of the observer B, is assigned in each case.

That is, in general, the head tracking unit KVE for the observer's head position Pj provides the camera signal Ki as the left image and the camera signal Ki + x as the right image.

It is important to define the n-1 head positions Pj of the observer B depending on the camera configuration. This is done depending on the configuration of the stereoscopic picture screen DIS and the intended effect on the observer B.

This means that two new camera signals are required by the camera tracking unit KVE. In order to improve or enhance the stereoscopic effect, ie the impression of the depth to the observer B, the distance between the cameras can be extended.

When the stereoscopic effect is to be reduced, of course, the camera spacing can also be reduced.

After the head position has been determined by the head tracking unit KVE as described above (1 in FIG. 2), the head tracking unit KVE determines the two scenes required, namely the required camera signals Ki and Ki + x. Select (2 in FIG. 2) and request them from the sending unit SE.

This is done in the form of camera request information KAN transmitted from the receiver unit EE to the transmission unit SE.

The transmission unit SE uses the camera request information KAN.
After receiving, the switch unit S connects the two inputs of the switch unit S to the requested cameras Kai and Kai + x, so that the requested camera signals Ki and Ki + x are transferred to the stereoscopic coding unit S.
It is switched so that it is connected through C (3 in FIG. 2).

In the stereoscopic picture coding unit SC, the serially connected camera signals Ki and Ki + x are coded and coded into the video data stream CV to be transmitted (4 in FIG. 2).

This coded video data stream CV is then transmitted to the receiver unit EE.

This received coded video data stream CV is decoded in the stereoscopic picture decoding unit SD in the receiver unit EE (5 in FIG. 2) and is sent to the viewer B via the stereoscopic picture screen DIS. It is displayed (6 in FIG. 2).

FIG. 3 illustrates an advantageous application of the method according to the invention.

The difficulty in using the model-based coding method MC lies in the handling of fault situations that occur in the coding method MC. That is, when the model-based coding method is faulty, that is, when the image is erroneously modeled by the model-based coding method MC.

In the frame of this specification, the model-based coding method MC is a method of displaying the scene SZ captured by n cameras by a three-dimensional model. As a result, the image data of the captured scene SZ can be further reduced.

In this case, until the correct model is generated or calculated again for the images of the video data stream,
Exception handling is needed to allow correct transmission of the video data stream. The method described above is provided for this exception handling.

This means that a coding method based on the model of the video data stream CV enables a comparatively large compression of the image data when it is detected that the model for the video data stream CV was produced with poor quality. MC
Is interrupted and the method of the invention is carried out (1 in FIG. 3).
0), that is, the scene changes the head position P of the observer B each time.
Two cameras Kai and Kai selected by j
Means to be taken by +1.

When a new model is created which again correctly represents the image data (9 in FIG. 3), the model-based coding method MC can again be implemented (7 in FIG. 3).

The method of the invention is therefore associated with a model-based coding method MC of the video data stream,
It can be used as a fallback solution for the model-based coding method MC.

[Brief description of the drawings]

FIG. 1 is a schematic diagram of an apparatus with a receiver unit and a transmitter unit and a plurality of cameras selected based on a viewer's head position.

FIG. 2 is a sequence diagram describing the method of the present invention as defined in claim 3;

FIG. 3 is a sequence diagram illustrating the use of the method of the invention as a fallback solution in model-based coding.

[Explanation of symbols]

SZ scene, SE transmission unit, Ka1-Kan
Camera, K1 to Kn camera signal, S switch unit, SC coding unit, CV video data stream, EE receiver unit, SD decoding unit, DIS screen, B observer, P1 to Pn-1 head position

Claims (4)

[Claims] 1. n cameras (Kai; i = 1 ... n)
Generated by a plurality of camera signals (Ki; i = 1
, N), a device for automatically selecting two cameras that together generate two camera signals forming one stereoscopic photographic image sequence, in a transmitting unit (SE), n cameras (Kai). ) Are arranged in an arbitrary manner, the n cameras can capture a plurality of scenes of a scene and can be used via a switch unit (S), A stereoscopic coding unit (SC) is provided for coding the video data stream (CV) to be coded and transmitted, wherein in the transmitting unit (SE) the switch A unit (S) is provided, the two output sides of the switch unit (S) being the stereoscopic picture coding unit. SC) and the n input sides of the switch unit (S) are connected to the n cameras (Kai ... Kan), and the switch unit is connected to the receiver unit (EE). Two camera signals (Ki; Ki + x) selected by the head tracking unit (KVE) can be used, where the number (x) is any number in the range 1 to n,
In said receiver unit (EE) said head tracking unit (KVE) is provided, said head tracking unit determining a head position (Pj; j = 1 ... N-1) and said head position (Pj). ) Corresponding to two camera signals (Ki; Ki + x), the receiver unit (EE) is provided with a stereoscopic picture decoding device (SD), and the stereoscopic picture decoding is performed. A device is for decoding the received coded video data stream (CV), and at the receiver unit (EE) at the output of the stereoscopic decoding device (SD). An automatic selection device for two cameras, characterized in that it is provided with a connected stereoscopic picture screen (DIS). 2. The apparatus according to claim 1, wherein the n cameras (Kai) are equidistant from each other. 3. n cameras (Kai; i = 1 ... n)
Generated by a plurality of camera signals (Ki; i = 1
, N), a method for automatically selecting two cameras that together generate two camera signals forming one stereoscopic photographic image sequence in a receiver unit (EE) at a head tracking unit (KVE). ) Determines the head position (Pj; j = 1 ... n-1) of the observer (B), and the head tracking unit (KVE) determines two cameras (Ka) corresponding to the head position (Pj).
i; Kai + x) select the two camera signals (Ki; Ki + x) generated by the two camera signals (Ki) and (Ki) at the head position (Pj).
+ X) is associated and the number (x) is any number in the range 1 to n, said receiver unit (EE)
Sends camera request information (KAN) to transmission unit (SE)
And the two required camera signals (Ki; Ki + x) are requested by the camera request information, the transmitting unit (SE) receives and stores the camera request information (KAN), and the transmitting unit In (SE),
Through the switch unit (S), the two camera signals (Ki; Ki + x) selected by the head tracking unit (KVE) are connected to the stereoscopic picture coding unit (SC) through the transmission unit (SE). In the stereoscopic picture coding unit (SC), the coded video data stream (CV) to be coded and transmitted is coded, and the coded video data stream (CV) to be transmitted is transmitted to the receiver unit (EE). Storing the received coded video data stream (CV) at the receiver unit (EE) and stereoscopically receiving the received coded video data stream (CV) at the receiver unit (EE). Decoding in Photo Decoding Unit (SD) Automatic selection method of two cameras characterized by. 4. Model-based coding method (M) for coding a video data stream (CV).
4. The method according to claim 3, wherein in the implementation of C), when the model-based coding (MC) is out of order, the model-based coding (MC) can be used again. Use for.