DE102018113822A1 - Converting an image stream with stereoscopic image content - Google Patents

Abstract

A method of converting a source image stream (10) containing image data in a stereoscopic format into a target image stream (20) having images (22) in a predetermined stereoscopic format, each image (22) being left-aligned. Comprising a left eye and right field partial image (26) for a right eye, the method comprising the steps of: recognizing a stereoscopic format of image data of the source image stream (10) based on image contents of the image data; Taking left-hand sub-images (16) for a left eye and right-field images (18) for a right eye from image data of the source image stream (10), said image data of the source image stream (10) being interpreted according to the recognized stereoscopic format ; Generating images (22) of the target image stream (20) in the predetermined stereoscopic format, including the extracted left fields (16) and right fields (18); and device for carrying out the method.

Description

The invention relates to a method and a device for converting a source image stream into a destination image stream.

The incoming image stream of a 3D television usually contains separate partial images for the left eye and for the right eye of an observer in the case of an image stream in a stereoscopic format. The 3D television generates therefrom a stereoscopic 3D reproduction, e.g. by alternately displaying the partial images for the left eye and for the right eye and corresponding timing of a shutter glasses. For example, in the HDMI (High Definition Multimedia Interface) interface standard in the HDMI version 1.4a, there are, among other things, defined three transmission formats that a standard television must be able to handle: a frame packing 3D video format that defines a frame area that is a field for a left eye and a field for a right eye on top of each other and active space for separation of the partial images contains; a side-by-side (half) 3D video format that defines a frame area according to a standard 2D frame that contains horizontally compressed by a factor of 2 sub-images for a left eye and a right eye side by side; and a top-and-bottom (half) 3D video format that defines a frame area according to a standard 2D frame containing vertically compressed by a factor of 2 sub-images for a left eye and a right eye on top of each other. Information on which transmission format is used can be transmitted via metadata in the form of 3D InfoFrame data packets.

If an image stream from an image source is not output in a format suitable for the 3D output device, or if, for example, in the image output of a computer stereoscopic 3D images are contained only in a window of a user interface in a 2D image stream, a 3D reproduction is made more difficult or prevented.

The object of the invention is to enable or simplify the use of a 3D output device, such as a 3D television or data glasses, for the stereoscopic reproduction of stereoscopic image data with different stereoscopic formats.

• Erkennen eines stereoskopischen Formats von Bilddaten des Quell-Bildstroms, basierend auf Bildinhalten der Bilddaten,
• Entnehmen von links-Teilbildern für ein linkes Auge und rechts-Teilbildern für ein rechtes Auge aus Bilddaten des Quell-Bildstroms, wobei diese Bilddaten des Quell-Bildstroms gemäß dem erkannten stereoskopischen Format interpretiert werden,
• Erzeugen von Bildern des Ziel-Bildstroms im vorgegebenen stereoskopischen Format unter Einbeziehung der entnommenen links-Teilbilder und rechts-Teilbilder.

This object is achieved according to the invention by a method for converting a source image stream containing image data in a stereoscopic format into a destination image stream having images in a predetermined stereoscopic format, the images each comprising a left field image for a left eye and a right-field image for a right eye, the method comprising the steps of:

• Recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data,
• Extracting left-field images for a left eye and right-field images for a right eye from image data of the source image stream, this image data of the source image stream being interpreted according to the recognized stereoscopic format,
• Generating images of the target image stream in the predetermined stereoscopic format, including the extracted left fields and right fields.

That the image data of the source image stream is interpreted as representing the left fields and the right fields.

Thus, conversion of a source digital image stream to a destination digital image stream can be done with a predetermined stereoscopic format of the images of the target image stream. The respective image stream comprises a sequence of images or frames. The destination image stream may include separate data streams for left fields and right fields. The source image stream can be received frame by frame or frame by frame. The generation of images of the target image stream can take place while continuously receiving further image data of the source image stream. Preferably, the latency between accepting an image or frame of the source image stream and outputting an image of the target image stream generated therefrom is less than 100 ms, more preferably less than 51 ms, less than 34 ms, or most preferably less than 17 ms.

The recognition of a stereoscopic format of image data of the source image stream is based on image contents of the image data, i. based on image contents of the image data. That is, at least it may also be based on image contents of the image data.

The image contents are represented by the image data. Image contents are defined here for respective pixels defined content, which are represented by the image data. That is, the recognition is based on image contents in the form of contents defined for respective pixels represented by the image data.

• Erkennen eines stereoskopischen Formats von ersten Bilddaten des Quell-Bildstroms, basierend auf Bildinhalten der ersten Bilddaten, und
• Entnehmen von links-Teilbildern für ein linkes Auge und rechts-Teilbildern für ein rechtes Auge aus weiteren Bilddaten des Quell-Bildstroms, wobei die weiteren Bilddaten des Quell-Bildstroms gemäß dem erkannten stereoskopischen Format der ersten Bilddaten interpretiert werden.

For example, the method may include:

• Recognizing a stereoscopic format of first image data of the source image stream based on image contents of the first image data, and
• Extracting left-field images for a left eye and right-field images for a right eye from further image data of the source image stream, the further image data of the source image stream being interpreted according to the recognized stereoscopic format of the first image data.

The step of recognizing may also be repeated for new first image data to respond to a change in composition and / or the stereoscopic format of the image data of the source image stream.

Preferably, at least one of the source image stream and the destination image stream is a digital image stream having successive frames containing images, for example, a digital image stream according to the HDMI 1.4, HDMI 2.0 or HDMI 2.1 standard.

The source image stream is a raster graphics image stream. The image data of the source image stream is, in particular, raster graphics image data (or bitmap image data), i. they can be referred to as pixel-based image data. The target image stream may also be a raster graphics image stream.

A frame is understood to mean a structure of an image signal in which image data are contained in a defined time sequence. The frame usually contains horizontal blanking intervals and a vertical blanking interval. A frame area containing line by line the image data corresponds to an x-y position area of the frame, where y indicates the line and x indicates the position within a line. In HDMI image streams, not only the image data can be transmitted, but additional information can be transmitted in data packets in the blanking intervals, such as audio data. In an HDMI image stream, image data is stored in e.g. transmit three channels simultaneously in the form of a bit stream.

The stereoscopic format of image data of the source image stream may, for example, correspond to or include an arrangement of left field images and right field images in the stereoscopic image data of the source image stream. The predetermined stereoscopic format of the images of the target image stream may correspond, for example, to or include an arrangement of left and middle images in the stereoscopic images of the target image stream.

For example, recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data may include or consist of recognizing an arrangement of left and right sub-images in stereoscopic image data of the source image stream.

The predetermined stereoscopic format of the images of the target image stream may differ from the stereoscopic format of the image data of the source image stream.

For example, an arrangement of the left fields and right fields in images of the target image stream may differ from an arrangement of left fields and right fields in the stereoscopic image data of the source image stream, which arrangement may include: an array of left partial image and right partial image next to each other, an arrangement of left partial image and right partial image one above the other, a line by line nested arrangement of left partial image and right partial image. The arrangement of left fields and right fields in the stereoscopic image data of the source image stream may also include: an arrangement of left field and right field as overlay, i. at the same position in an image, in different colors of the left partial image and the right partial image (color anaglyphic stereo image), an arrangement of left partial images and right partial images at the same position alternately in successive images of the source image current (the frame interlacing or frame-interlaced format, also referred to as temporal interlacing), and an arrangement of left-field and right-field at the same position in successive images of the source image stream, wherein one field at different playback times as left field and as right Partial image serves (to exploit the Pulfrich effect).

The step of recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data may be performed, for example, when information about a stereoscopic image format of images of the source image stream could not be read from metadata of the source image stream, such as 3D InfoFrame data packets of a 3D image stream according to an HDMI standard.

• Überprüfen des Vorliegens von Information über ein stereoskopisches Bildformat von Bildern des Quell-Bildstroms in Metadaten des Quell-Bildstroms,
• Auslesen der Information über ein stereoskopisches Bildformat von Bildern des Quell-Bildstroms aus Metadaten des Quell-Bildstroms, wenn die Information vorliegt,
• wobei der Schritt des Erkennens eines stereoskopischen Formats von Bilddaten des Quell-Bildstroms basierend auf Bildinhalten der Bilddaten ausgeführt wird, wenn die Information nicht vorliegt, d.h. wenn Information über ein stereoskopisches Bildformat von Bildern des Quell-Bildstroms nicht aus Metadaten des Quell-Bildstroms ausgelesen werden konnte.

For example, the method may include:

• Checking the presence of information about a stereoscopic image format of images of the source image stream in metadata of the source image stream,
• Reading the information about a stereoscopic image format from images of the source Image stream of metadata of the source image stream, if the information is present
• wherein the step of recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data is carried out when the information is not present, that is, when information about a stereoscopic image format of images of the source image stream is not extracted from metadata of the source image stream could.

• Vergleichen einer Bildgröße eines Bildes des Quell-Bildstroms mit unterschiedlichen stereoskopischen Formaten zugeordneten Bildgrößen, wobei beim Vergleichen jeweils überprüft wird, ob eine Vergleichsbedingung für ein einer jeweiligen Bildgröße zugeordnetes stereoskopisches Format erfüllt ist,
• wobei der Schritt des Erkennens eines stereoskopischen Formats von Bilddaten des Quell-Bildstroms basierend auf Bildinhalten der Bilddaten ausgeführt wird, wenn in dem Schritt des Vergleichens einer Bildgröße eines Bildes des Quell-Bildstroms mit unterschiedlichen stereoskopischen Formaten zugeordneten Bildgrößen keine der jeweiligen Vergleichsbedingungen erfüllt ist. D.h., der Schritt des Erkennens, basierend auf den Bildinhalten, wird durchgeführt, wenn in dem Schritt des Vergleichens der Bildgröße kein stereoskopisches Format von Bildern des Quell-Bildstroms erkannt wurde.

In one or more embodiments, the method comprises:

• Comparing an image size of an image of the source image stream with image sizes associated with different stereoscopic formats, wherein in each case it is checked whether a comparison condition for a stereoscopic format assigned to a respective image size is fulfilled,
• wherein the step of recognizing a stereoscopic format of image data of the source image stream is performed based on image contents of the image data if none of the respective comparison conditions is satisfied in the step of comparing an image size of an image of the source image stream with image sizes associated with different stereoscopic formats. That is, the step of recognizing based on the image contents is performed when no stereoscopic format of images of the source image stream has been recognized in the step of comparing the image size.

• Bildgröße ist exakt gleich 1920 x 2205 - Frame-Packing 1080p, d.h. links-Teilbild 1920 x 1080 über aktivem Leerraum 1920 x 45 über rechts-Teilbild 1920 x 1080 in einem Bild, entsprechend einem Seitenverhältnis des Bildes von 128/149, gleich etwa 0,87:1;
• Bildgröße ist exakt gleich 1280 x 1470 - Frame-Packing 720p, d.h. links-Teilbild 1280 x 720 über aktivem Leerraum 1280 x 30 über rechts-Teilbild 1280 x 720 in einem Bild, entsprechend einem Seitenverhältnis des Bildes von 128/149, gleich etwa 0,87: 1;
• Bildgröße hat ein Seitenverhältnis größer gleich 2:1, beispielsweise 3840 x 1080, - Side-by-Side-(full), d.h. links-Teilbild und rechts-Teilbild (z.B. beide 1920 x 1080) direkt nebeneinander in einem Bild (ohne horizontale Stauchung);
• Bildgröße hat ein Seitenverhältnis kleiner als 1:1 (außer obige Frame-Packing-Formate), beispielsweise 1920 x 2160 - Top-and-Bottom-(full), d.h. links-Teilbild direkt über rechts-Teilbild in einem Bild (ohne vertikale Stauchung).

For example, at least the following associations can be made between stereoscopic formats and respective image sizes with the relevant comparison conditions:

• Image size is exactly the same 1920 x 2205 - Frame Packing 1080p, ie left field 1920 x 1080 over active space 1920 x 45 over right field 1920 x 1080 in an image, corresponding to an aspect ratio of the image of 128/149, equal to about 0 , 87: 1;
• Image size is exactly equal to 1280 x 1470 - frame-packing 720p, ie left-field 1280 x 720 over active space 1280 x 30 over right-field 1280 x 720 in one image, corresponding to an aspect ratio of the image of 128/149, equal to about 0 , 87: 1;
• Image size has an aspect ratio greater than or equal to 2: 1, for example 3840 x 1080, - Side-by-Side (full), ie left field and right field (eg both 1920 x 1080) directly next to each other in one image (without horizontal compression );
• Image size has an aspect ratio smaller than 1: 1 (except for the above frame packing formats), for example 1920 x 2160 - top and bottom (full), ie left field directly over right field in a picture (without vertical compression) ).

Optionally, for example, as explained below, recognition for a partial area of an image corresponding to an assumed 3D image data area should be included, the first two cases frame packing 1080p and frame packing 720p by the assignment: image size has Aspect ratio of 128/149 or approximately equal to 0.87: 1 - Frame Packing, ie left partial image over active white space over right partial image in an image, with corresponding height components of left partial image, active white space and right partial image.

For all other cases, i. for an image size with an aspect ratio ranging from 1: 1 to less than 2: 1 (exclusive), the step of recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data is performed. Preferably, the format of image data of the source image stream to be recognized based on image contents of the image data is recognized as one of one or more formats of the following formats (1) to (2) and (6) to (9).

In one or more embodiments, the stereoscopic format of image data of the source image stream to be recognized based on image contents of the image data is recognized as one of a plurality of stereoscopic formats, the plurality of stereoscopic formats comprising stereoscopic formats having a respective different arrangement of left fields and right Have partial images in the stereoscopic image data of the source image stream.

1. (1) ein Side-by-Side-(half)-Format mit einem links-Teilbild und einem rechts-Teilbild nebeneinander, die jeweils ein Seitenverhältnis aufweisen, das jeweils einem in x-Richtung mit dem Faktor 0,5 gestauchten Vollbild entspricht, wobei das Seitenverhältnis des Vollbildes vorzugsweise eines von 4:3 und 16:9 ist;
2. (2) ein Top-and-Bottom-(half)-Format mit einem links-Teilbild und einem rechts-Teilbild in y-Richtung übereinander angeordnet, die jeweils ein Seitenverhältnis aufweisen, das jeweils einem in y-Richtung mit dem Faktor 0,5 gestauchten Vollbild entspricht, wobei das Seitenverhältnis des Vollbildes vorzugsweise eines von 4:3 und 16:9 ist,
3. (3) ein Side-by-Side-(full)-Format mit einem links-Teilbild und einem rechts-Teilbild direkt nebeneinander, wobei das Seitenverhältnis des gesamten Bildes aus links-Teilbild und rechts-Teilbild vorzugsweise zusammen 32:9, also etwa 3,56:1 ist;
4. (4) ein Top-and-Bottom-(full)-Format mit einem links-Teilbild und einem rechts-Teilbild direkt übereinander,
5. (5) ein Frame-Packing-Format mit einem links-Teilbild und einem rechts-Teilbild in y-Richtung übereinander und getrennt durch aktiven Leerraum angeordnet, wobei das Seitenverhältnis des gesamten Bildes aus links-Teilbild, Leerraum und rechts-Teilbild vorzugsweise 128:149 oder etwa gleich 0,87:1 ist;
6. (6) ein Line-Alternative-Format mit einem links-Teilbild und einem rechts-Teilbild zeilenweise verschachtelt, d.h. gerade Zeilen und ungerade Zeilen sind unterschiedlichen Teilbildern zugeordnet;
7. (7) ein Frame-Interlaced-Format, bei dem ein Rahmen mit einem links-Teilbild und ein Rahmen mit einem rechts-Teilbild abwechselnd aufeinander folgen;
8. (8) ein farb-anaglyphisches Format gemäß einer oder mehreren Kombinationen von links-kodierenden und rechts-kodierenden Farben, insbesondere eines oder mehrere von einem rot-blau-anaglyphischem Format, einem rot-grün-anaglyphischem Format, einem rot-cyan-anaglyphischem Format, einem gelb-blau-anaglyphischem Format;
9. (9) ein zur Ausnutzung des Pulfrich-Effektes eingerichtetes Format.

Beispielsweise kann eines der Formate (1) bis (9) als Format von Bilddaten des Quell-Bildstroms erkannt werden, welche Bilddaten einem Vollbild oder Rahmen des Quell-Bildstroms entsprechen.In one or more embodiments, the format of image data of the source image stream to be recognized based on image contents of the image data is recognized as one of one or more formats, comprising:

1. (1) a side-by-side (half) format with a left field and a right field side by side each having an aspect ratio each corresponding to a frame compressed in the x direction by a factor of 0.5, the aspect ratio of the Frame is preferably one of 4: 3 and 16: 9;
2. (2) a top-and-bottom (half) format with a left field and a right field in the y direction superimposed, each having an aspect ratio, each one in the y direction by a factor of 0, 5 compressed frame, wherein the aspect ratio of the frame is preferably one of 4: 3 and 16: 9,
3. (3) a side-by-side (full) format with a left field and a right field directly next to each other, wherein the aspect ratio of the entire picture of left field and right field preferably together 32: 9, so about 3.56: 1;
4. (4) a top-and-bottom (full) format with a left field and a right field directly on top of each other,
5. (5) a frame-packing format having a left field and a right field in the y direction superimposed and separated by active white space, wherein the aspect ratio of the entire image of left field, space and right field preferably 128: 149 or about 0.87: 1;
6. (6) line-by-line interleaving a line alternative format with a left field and a right field, that is, even lines and odd lines are assigned to different fields;
7. (7) a frame interlaced format in which one frame with a left field and one frame with a right field follow each other alternately;
8. (8) a color anaglyphic format according to one or more combinations of left-coding and right-coding colors, in particular one or more of a red-blue anaglyphic format, a red-green-anaglyphic format, a red-cyan-anaglyphic Format, a yellow-blue anaglyphic format;
9. (9) a format designed to exploit the Pulfrich effect.

For example, one of the formats (1) to (9) may be recognized as a format of image data of the source image stream, which image data corresponds to a frame or frame of the source image stream.

For example, one of the formats (1) to (5) and (7) to (9) may be recognized as a format of image data of the source image stream based on image contents of the image data, which image data corresponds to a partial area of an image of the source image stream, especially one right-angled polygonal or rectangular section. Outside this subarea, for example, 2D image data may be present. Thus, for example, scaled reproduced video images or images can be recognized as stereo images.

The recognition may also be limited to one or more of the mentioned formats, or comprise further formats.

When recognizing the format, it can be concluded from the aspect ratio that there is a compression of the sub-images in the x-direction or y-direction, in particular in a side-by-side (half) format or a top-and-bottom (half) format. This is taken into account when interpreting the image data for the removal of the partial images.

The described comparison of the image size can be carried out, for example, if the information about a stereoscopic image format of images of the source image current is not present in metadata of the source image current.

Advantageous embodiments of the invention are specified in the subclaims.

• Überprüfen von mehreren Hypothesen über ein stereoskopisches Format von Bilddaten des Quell-Bildstroms,
• Erkennen des stereoskopischen Formats einer Hypothese als das stereoskopische Format der Bilddaten, wenn in dem Schritt des Überprüfens die Hypothese als zutreffend erkannt wurde.

In one or more embodiments, recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data comprises:

• Examining several hypotheses about a stereoscopic format of image data of the source image stream,
• Recognizing the stereoscopic format of a hypothesis as the stereoscopic format of the image data if, in the step of checking, the hypothesis was found to be true.

Preferably, a hypothesis about a stereoscopic format of image data of the source image stream includes a hypothesis about an arrangement of left field images and right field images in image data of the source image stream. For example, a hypothesis about a stereoscopic format of image data of the source image stream may correspond to a hypothesis about an arrangement of left field images and right field images in image data of the source image stream.

• Detektieren, für eine jeweilige Hypothese von zu überprüfenden Hypothesen, von paarweise korrespondierenden Merkmalen in links-Bildinhalten und rechts-Bildinhalten der Bilddaten, wobei die Bilddaten gemäß der betreffenden Hypothese als die links-Bildinhalte und die rechts-Bildinhalte repräsentierend interpretiert werden;
• Erkennen einer Hypothese als zutreffend basierend auf sich paarweise hauptsächlich in ihren x-Koordinaten unterscheidenden Positionen von detektierten, paarweise korrespondierenden Merkmalen.

For example, reviewing multiple hypotheses may include:

• Detecting, for a respective hypothesis of hypotheses to be tested, paired corresponding features in left image contents and right image contents of the image data, the image data being interpreted as representing the left image contents and the right image contents according to the hypothesis in question;
• Recognizing a hypothesis as true based on pairs of mainly differing positions of detected, pairwise corresponding features in their x-coordinates.

That is, in the step of detecting pairs of corresponding features, the image data is interpreted according to the hypothesis in question as a representation of the left image contents and the right image contents. There is an assignment of image data to left-image content and right-image content according to the hypothesis in question.

For example, in the step of recognizing a hypothesis, the hypothesis may be deemed to be true if the hypothesis among those hypotheses for which a similarity criterion for the y-coordinates of the pairwise corresponding features is satisfied for the hypothesis in question is a maximum value of one Measure for the scattering of the pairwise differences of the x-coordinates of the pairwise corresponding features.

In this way, the recognition of a hypothesis is made as appropriate based on a compatibility of coordinates of the pairwise corresponding features with typical features of a stereoscopic image content.

Depending on the hypothesis in question, the left-hand image content and the right-hand image content may be included in a frame or frame of the source image stream or contained in successive images or frames of the source image stream.

In one or more embodiments, the recognition of a stereoscopic format of image data of the source image stream is based on image contents of the image data by means of an artificial neural network. In particular, the recognition can be done by the artificial neural network. For example, the recognition of an arrangement of left partial images and right partial images in stereoscopic image data of the source image current can take place by means of an artificial neural network.

For example, recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data may include preprocessing image contents of the image data, extracting global image features from image contents of the image data, and receiving the extracted global image features as inputs to the artificial neural network. Thus, in preprocessing image contents of the image data, global image features are extracted from image contents in the form of contents defined for respective pixels represented by the image data. For example, recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data may include recognizing an arrangement of left and right sub-images in stereoscopic image data of the source image stream, which in turn may include preprocessing.

In one or more embodiments, the image data of the source image stream whose stereoscopic format is recognized may together correspond to a rectangular portion of an image of the source image stream or a rectangular portion of successive images of the source image stream. The image data of the source image stream whose stereoscopic format is recognized may also correspond to a frame or total image area of the source image stream or frames, or a total image area of successive images of the source image stream.

In one or more embodiments, the method includes: detecting a perimeter of an image area of image data having a stereoscopic format. This image data will hereinafter also be referred to as 3D image data, and the area will also be referred to as 3D image data area. Preferably, the step of recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data is then performed for image data corresponding to that image region. The method thus comprises: recognizing a stereoscopic format of the image data corresponding to this image area. In particular, the method may include: detecting an arrangement of left and right sub-images in stereoscopic image data of the source image stream corresponding to that image region. For example, the step of recognizing a scope of an image area of image data having a stereoscopic format may be performed when, in a step of examining image contents of image data of the source image stream for recognizing a stereoscopic format of the image data based on image contents of the image data the image data corresponds to a frame or entire image area of an image of the source image stream, no stereoscopic format is recognized. There may then be another step of examining image contents of image data of the source image stream to recognize a stereoscopic format of the image data based on image contents of the image data, the image data corresponding to the detected amount of the 3D image area. This step of examining image contents of image data of the source image stream then includes, for example, the step of recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data. In particular, it may comprise the step of recognizing an arrangement of left fields and right fields in stereoscopic image data of the source image stream based on image contents of the image data.

In particular, the 3D image data area may be a rectangular polygonal or a rectangular image area. This image area can also be called a 3D image area. The scope of the image area is understood to mean the extent or the size of the image area of the relevant image data. For example, recognizing a stereoscopic format of the image data corresponding to that image area based on image contents of the image data may include, as described, reviewing multiple hypotheses about a stereoscopic format of that image data of the source image stream. The recognition of a stereoscopic format of the image data that corresponds to this image area based on image contents of the image data can be done, for example, as described, by means of an artificial neural network. Also, in a step of recognizing a stereoscopic format of image data of the source image current performed for image data corresponding to a frame or entire image area of an image of the source image stream, recognizing a stereoscopic format based on image contents of the image data may be, for example, as described, include checking multiple hypotheses about a stereoscopic format of this image data of the source image stream, or, as described, using an artificial neural network.

In one or more embodiments, the method includes: detecting a perimeter of an image area of 2D image data; the method further comprising: extracting 2D subframes from 2D image data of the source image stream, generating images of the target image stream in the predetermined stereoscopic format, including the extracted left sub images and right sub images, and including the extracted 2D Partial images take place. The extracted 2D partial images are duplicated, for example.

For example, an image area lying outside an image area of image data having a stereoscopic format can be recognized as an image area of 2D image data.

For example, if no stereoscopic format is detected for image data of the source image stream, extraction of 2D images from these image data of the source image stream can be performed, and images of the target image stream are generated in the predetermined stereoscopic format including the extracted ones 2D part images, which are duplicated. As a result, with only temporary presence of image data with stereoscopic format in the remaining times still the target image stream with image data in the predetermined stereoscopic format can be generated.

For example, recognizing a perimeter of an image area of image data of the source image stream having a stereoscopic format may be based on image contents of the image data of the source image stream.

• Detektieren von horizontalen und vertikalen Kanten in Bildinhalten der Bilddaten, die zusammen einen Rand eines Bildbereichs definieren, wobei die Bilddaten als 2D-Bilddaten interpretiert werden, und
• Erkennen eines stereoskopischen Formats von Bilddaten des Quell-Bildstroms, die einem Bildbereich entsprechen, dessen Rand durch die detektierten Kanten definiert wird.

For example, recognizing a perimeter of an image area of image data of the source image stream having a stereoscopic format based on image contents of the image data of the source image stream may include:

• Detecting horizontal and vertical edges in image contents of the image data that together define an edge of an image area, the image data being interpreted as 2D image data, and
• Recognizing a stereoscopic format of image data of the source image stream corresponding to an image area whose boundary is defined by the detected edges.

For example, horizontal and vertical edges can be detected, which together define an edge of a polygonal image region, also referred to as a right-angled polygonal image region, or in particular of a rectangular image region.

For example, the source image stream may contain 2D image contents in a 2D format in which stereoscopic image contents are embedded and contained in, for example, a rectangular image area.

In one or more embodiments, in the step of extracting left-left images for a left eye and right-images for image data of the source image stream, the extracted left images and right images may correspond to different color channels of image data recognized stereoscopic format of image data is a color anaglyphic stereoscopic format. As a result, color anaglyphic coded stereoscopic image contents are converted into the predetermined stereoscopic format.

In one or more embodiments, in the step of extracting left-left images for a left eye and right-images for a right eye from image data of the source image stream, extracting a left-field for a left eye and an associated right-field for a right eye is made of respective successive images of the source image stream, wherein when generating images of the target image stream in the predetermined stereoscopic format, there is a delayed inclusion of the extracted partial images for one of the two eyes. As a result, Pulfrich-coded stereoscopic image contents can be converted into the predetermined stereoscopic format.

The object is further achieved by an apparatus for converting a source image stream containing image data in a stereoscopic format into a destination image stream having images in a predetermined stereoscopic format, the images each comprising a left field image for a left eye and a right-eye right-field image comprising: detecting means for recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data; and converting means for converting the source image stream into the target image stream, the apparatus being arranged to perform the method. In particular, the recognition means may be arranged to perform the step of recognizing a stereoscopic format of image data of the source image stream based on image contents of the image data. In particular, the converting means may be arranged to perform the steps of extracting left fields and right fields and generating images of the target image stream.

The device may, for example, be implemented in an image-current source, such as a personal computer (PC), and may be implemented, for example, in a display driver or graphics adapter (e.g., a graphics card). The device can also be, for example, a converter for connecting an image source with a 3D output device. The device can also be integrated, for example as a converter, in a 3D output device and output the target image stream in a stereoscopic format suitable for output by the 3D output device, for example in the form of a frame-sequential image stream with alternating Partial images for the left and the right eye, or in the form of separate data streams for left partial images and right partial images.

• 1 eine schematische Darstellung einer Konvertierung eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem Ausführungsbeispiel;
• 2 eine schematische Darstellung einer Konvertierung eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem weiteren Ausführungsbeispiel;
• 3 eine schematische Darstellung einer Konvertierung eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem weiteren Ausführungsbeispiel;
• 4 eine schematische Darstellung eines Konverters zum Konvertieren eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem Ausführungsbeispiel;
• 5 eine schematische Darstellung einer Erkennung eines Umfangs eines Bildbereichs von 3D-Bilddaten;
• 6 eine schematische Darstellung von horizontalen und vertikalen Kanten in Bildinhalten der Bilddaten eines Bildes des Quell-Bildstroms;
• 7 eine schematische Darstellung eines Verfahrens zum Erkennen eines stereoskopischen Formats von Bilddaten des Quell-Bildstroms mittels eines künstlichen Neuronalen Netzes;
• 8 eine schematische Darstellung eines Verfahrens zum Erkennen eines stereoskopischen Formats von Bilddaten des Quell-Bildstroms mittels Merkmalsdetektoren;
• 9 eine schematische Darstellung einer Konvertierung eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem weiteren Ausführungsbeispiel;
• 10 eine schematische Darstellung einer Konvertierung eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem weiteren Ausführungsbeispiel; und
• 11 eine schematische Darstellung einer Konvertierung eines Quell-Bildstroms in einen Ziel-Bildstrom gemäß einem weiteren Ausführungsbeispiel.

Preferred embodiments of the invention are explained in more detail below with reference to the drawing. Show it:

• 1 a schematic representation of a conversion of a source image stream into a target image stream according to an embodiment;
• 2 a schematic representation of a conversion of a source image stream in a target image stream according to another embodiment;
• 3 a schematic representation of a conversion of a source image stream in a target image stream according to another embodiment;
• 4 a schematic representation of a converter for converting a source image stream into a target image stream according to an embodiment;
• 5 a schematic representation of a detection of a scope of an image area of 3D image data;
• 6 a schematic representation of horizontal and vertical edges in image contents of the image data of an image of the source image stream;
• 7 a schematic representation of a method for recognizing a stereoscopic format of image data of the source image stream by means of an artificial neural network;
• 8th a schematic representation of a method for detecting a stereoscopic format of image data of the source image stream by means of feature detectors;
• 9 a schematic representation of a conversion of a source image stream in a target image stream according to another embodiment;
• 10 a schematic representation of a conversion of a source image stream in a target image stream according to another embodiment; and
• 11 a schematic representation of a conversion of a source image stream in a target image stream according to another embodiment.

1 schematically shows a conversion of a source image stream 10 into a target image stream 20 according to an embodiment. The source image stream 10 includes a sequence of images 12 , which are each represented by corresponding image data in the form of pixel-based raster graphics (bitmap images). The image data of a respective image 12 the source image stream 10 include 2D image data, which is a 2D sub-image 14 and 3D image data, which is a left field image 16 and a right-side image 18 according to a stereoscopic format. In 1 For example, the stereoscopic format of the 3D image data of the source image stream is a side-by-side format with x Direction with a factor of 0.5 compressed image content of the partial images 16 . 18 , The 3D image data lies in a 3D image data area in the form of a rectangular section of the image data of the entire image 12 in front. The upper part of 1 shows the pictures 12 in the aspect ratio of the 2D image data.

1 shows the conversion of the source image stream into the target image stream after recognizing the stereoscopic format of the 3D image data.

The 3D image data corresponding to the partial images 16 . 18 are interpreted according to the stereoscopic format of the 3D image data, and it becomes the left field 16 and the right-part image 18 from the image data of the source image stream 10 taken. The middle part of 1 shows the left and right fields 16 . 18 in the correct aspect ratio, ie without the compression. Next is the 2D sub-picture 14 from the image data of the source image stream 10 taken.

It will be a sequence of pictures 22 of the target image stream 12 generated in the specified stereoscopic format. The default stereoscopic format is in 1 a side-by-side (full) format eg with a double full HD image resolution of 2x1920 pixels width and 1080 pixels height, without compression in x - or y- Direction. The lower part of 1 shows the generated images 22 in the correct aspect ratio (the actual aspect ratio is shown only schematically in the figures). A respective picture 22 of the target image stream 12 is generated using the left field 16 , the right-part image 18 , and the duplicated 2D field 14 , It includes a generated left field 26 of the target image stream 20 a copy of the duplicated 2D sub-image 14 and the extracted left partial image 16 the source image stream in the target format, the given stereoscopic format of the images 22 of the target image stream 20 , Accordingly, a generated right field includes 28 of the target image stream 20 a copy of the duplicated 2D sub-image 14 and the extracted right-part image 18 the source image stream in the target format.

2 shows simplified a conversion of an image 12 a source image stream into an image 22 a target image stream 20 according to a further embodiment. The presentation is opposite 1 simplified. In particular, the intermediate steps that are in 1 are shown in the middle part of the figure, for simplicity in 2 not shown, and there is only one picture at a time 12 . 22 the source image stream 10 or target image stream 20 shown. In 2 For example, the stereoscopic format of the 3D image data of the source image stream is also a side-by-side format with image content of the partial images compressed in the x-direction by a factor of 0.5 16 . 18 , The 3D image data lies in a 3D image data area in the form of a rectangular section of the image data of the entire image 12 in front. The image data of a respective image 12 the source image stream 10 also include 2D image data representing a 2D subpicture 14. The default stereoscopic format (target format) is in 2 a frame-packing format eg with a double full HD image resolution of 1920 pixels wide and 2x1080 pixels high, without compression in x - or y Direction, plus an active white space 27 between a left field 26 and a right-part image 28 , Incidentally, the conversion corresponds to the conversion according to 1 , A respective picture 22 of the target image stream 12 is generated using the left field 16 , the right-part image 18 , and the duplicated 2D sub-image 14.

3 shows simplified a conversion of an image 12 a source image stream into an image 22 one of a target image stream according to another embodiment. In 3 exist a picture 12 the source image stream 1 from the left part of the picture 16 and the right-hand part of the picture 18 , but in a different stereoscopic format than the target format. A respective picture 22 of the target image stream 12 is generated using the extracted left field 16 and the extracted right-part image 18 , but no duplicated 2D sub-image. Incidentally, the conversion corresponds to the conversion according to 1 and 2 ,

4 schematically shows a converter for performing a conversion method as described above or as follows. The converter includes a picture stream input 40 for accepting a source image stream 10 , a detection device 42 for recognizing a stereoscopic format of image data of the source image stream 10 , a conversion device 44 for converting the source image stream into a destination image stream 20 based on the recognized stereoscopic format of the images of the source image stream 10 , and a picture stream output 46 to output the target image stream 20 in a given stereoscopic format. After the recognition of the stereoscopic format of the 3d image data of the source image stream 10 based on one or more initial images 12 the source image stream 10 was performed, will take more pictures 12 the source image stream 10 and optionally also optionally cached initial images 12 the source image stream 10 in generated images 22 of the target image stream 20 converted, according to information 48 about the recognized format.

5 schematically shows a method for recognizing a perimeter of an image area of 3D image data within the image data of an image 12 the source image stream 10 and recognizing a stereoscopic format of the 3D image data of the source image stream 10 , In one step S10 a hypothesis is assumed about the image area of the 3D image data, that is, about a 3D image data area within the image data, which 3D image data area corresponds to a representation of 3D image contents. This corresponds to an assumption about the size and location of an area, the left field 16 and the right-hand part of the picture 18 corresponds to, in the image data of the image 12 the source image stream 10 , For example, it is initially assumed that the amount of 3D image data corresponds to the entire circumference the image data of the image 12 as in the example of the 3 is illustrated.

In step S12 a method of recognizing a stereoscopic encoding format of left image data and right image data within the assumed 3D image data area based on image contents of the image data is performed; Examples of such a method will be explained below. Optionally, in step S12 Initially, a comparison of an image size of an image corresponding to the assumed 3D image data area of the source image stream with image sizes assigned to different stereoscopic formats takes place, wherein in each case it is checked whether a comparison condition for a stereoscopic format assigned to a respective image size is fulfilled. In this way, for example, as explained in the description above, encoding formats such as frame-packing, side-by-side (full) and / or top-and-bottom (full) can be detected. Said method based on image contents of the image data is then in step S12 executed when none of the respective comparison conditions is met in the comparison.

In step S14 is decided based on the result of the recognition of the stereoscopic encoding format, whether a stereoscopic format of 3D image data of the source image stream 10 was detected. If so, the stereoscopic format was recognized. information 48 The recognized format and the extent of the 3D image data area are output and will subsequently be used for the conversion. If no, in step S16 determined whether another hypothesis on the image area of the 3D image data is to be checked.

If not, it is determined that there is no stereoscopic format of image data of the source image stream 10 was detected. The image data of the pictures 12 of the source image stream are then interpreted as 2D image data and duplicated in images 22 of the target image stream 20 converted in the specified stereoscopic target format.

• - Detektieren von horizontalen Kanten 50 und vertikalen Kanten 52 in Bildinhalten der Bilddaten eines Bildes 12, wobei die Bilddaten des Bildes 12 als 2D-Bilddaten interpretiert werden;
• - Auswählen detektierter horizontaler Kanten 50 und vertikaler Kanten 52, die zusammen einen Rand eines rechteckigen Bildbereichs definieren, wobei die Bilddaten des Bildes 12 als 2D-Bilddaten interpretiert werden.

If yes (in step S16 ), the procedure with step S10 where a new hypothesis is assumed about the image area of the 3D image data. For this purpose, for example, the following method is performed, which is schematically in 6 is illustrated:

• - Detecting horizontal edges 50 and vertical edges 52 in image contents of the image data of an image 12 where the image data of the image 12 interpreted as 2D image data;
• Selecting detected horizontal edges 50 and vertical edges 52 , which together define an edge of a rectangular image area, wherein the image data of the image 12 be interpreted as 2D image data.

In 6 The rectangular image area corresponds to the left partial image 16 and the right partial image 18. The edges 50 . 52 For example, based on contrast values and / or color values of the image contents of the image 12 be detected by means of appropriate feature detectors. The edges 50 . 52 For example, they may correspond to an inner frame of a window of a user interface.

The procedure then goes to the step S12 continued. Optionally, further hypotheses about the image area of the 3D image data can be successively checked, for example, if several rectangular areas within an image 12 available.

The hypotheses about the image area of the 3D image data can also be checked simultaneously or at the same time.

Recognizing a stereoscopic format of image data based on image contents of the image data, that is, particularly recognizing a stereoscopic encoding format of left image data and right image data within the supposed 3D image data region based on image contents of the image data in step S12 and, in particular, recognizing an arrangement of left image data and right image data within the assumed 3D image data region can be done, for example, by means of an artificial neural network. The artificial neural network is used, for example, for the classification of the stereoscopic coding format based on image contents of the image data of an input image 12 trained. For example, the artificial neural network checks in parallel a plurality of hypotheses to be checked about the stereoscopic format of image data of the assumed 3D image data area.

An established method for classifying images is to have the classification method learned from an artificial neural network using classified example images. For the present classification method, a deep convolutional neural network (CNN) can be used. As a network topology, it is recommended to alternate a convolutional layer (8 filter cores each 3x3 pixels) 6 times or more in succession with a max pooling layer, followed by a fully connected layer. As an activation function in the hidden layers ReLU, and in the starting layer SoftMax is recommended. To train the network, for example, about 10,000 Stereo image pairs used, which are used in each of the different 3D formats as input.

7 schematically shows an embodiment of recognizing a stereoscopic format of image data of the source image stream 10 using a (much smaller) artificial neural network. The procedure according to 7 has the advantage that at the time of execution of the procedure, an artificial neural network of the size described above need not be called. The method may, for example, be carried out for a respective hypothesis about the image area of the 3D image data, for example in the step S12 of the example of 5 ,

In one step S20 become image contents of image data of an image 12 the source image stream 10 preprocessed by extracting global image features from image contents of the image data of the assumed 3D image data area. Features include, the mean square sum of the differences of (a) horizontally adjacent pixels, (b) vertically adjacent pixels, (c) by half the image width (or half the width of the assumed 3D image data area) pixels offset from each other, ( d) to calculate half the image height (or half the height of the assumed 3D image data area) pixels offset from each other and (e) pixels in each odd image line to pixels in the respective subsequent even image line. These five scalar values leading to an input image 12 are computed as input to an artificial neural network for stereoscopic format recognition (step S22 ). For example, the artificial neural network may be a fully networked artificial neural network with a hidden layer of 50 ReLU neurons and a SoftMax output layer. Information is given 48 about the recognized stereoscopic format or information 49 that no stereoscopic format was detected. The artificial neural network is again trained as described above.

For recognizing a frame interlaced format, the following global image features may also be extracted from image contents of the image data of the assumed 3D image data area and given as input to the artificial neural network: the mean square sum of the differences of (f) by one frame in time offset pixels (at the same position), and (g) temporally offset by two frames (at the same position). In the case of a frame-interlaced format, the change between temporally consecutive frames (ie, 1 frame spacing) is greater than between frames with the spacing 2 (ie distance of 2 frames). So seven scalar values are used.

In an identified as true stereoscopic format, which corresponds to a true recognized arrangement of left fields and right fields in the 3D image data area of the source image stream, can also in an optional step S24 a distinction is made between stereoscopic formats with or without compression of the image content in the direction of the arrangement of the left fields and right fields. This distinction can be made, for example, if the left partial images and right partial images are present at separate positions in an image (in particular, not for line-interlaced partial images, corresponding to a line alternative format, for color anaglyphic partial images, for a frame). Interlaced format or in sub-images to exploit the Pulfrich effect).

The distinction can be made, for example, based on an aspect ratio of the image area of the 3D image data. For example, with an aspect ratio of 128/149 or about 0.87: 1, a frame-packing format may be inferred that does not compress the image content. For example, with a different aspect ratio that is less than 1: 1, a top-bottom (full) format may be inferred that does not compress the image content. For example, with an aspect ratio greater than or equal to 2: 1, a side-by-side (full) format can be deduced that does not compress the image content. Thus, for example, with an aspect ratio of less than 1: 1 or greater than 2: 1, it can be concluded that there is no compression of the image content. Accordingly, for example, with an aspect ratio in the range of 1: 1 to less than 2: 1 (exclusive), it can be concluded that the image content is compressed by a factor of 0.5 in the direction in which the partial images are arranged side by side or one above the other ,

Instead of the step S24 however, it can also detect side-by-side (full), top-and-bottom (full), or frame-packing formats based on the aspect ratio even before recognizing the stereoscopic format of the steps S20 and S22 take place, wherein the recognition of the stereoscopic format in the steps S20 and S22 is performed only if it was not possible to deduce a particular stereoscopic format based on the aspect ratio, for example, if the aspect ratio is within the specified range of 1: 1 to less than 2: 1 (exclusive) (for example, if the aspect ratio 16: 9, that is about 1.78: 1).

The said distinction in step S24 but can also be based on image content of the 3D image data. This can, for example, a Algorithm for recognizing objects or faces are executed in the image content, wherein the image data are respectively interpreted according to a stereoscopic format with compression as well as a stereoscopic format without compression. The distinction can then be made on the basis of a recognition rate, ie based on numbers of detected objects or faces for the respective interpretation of the image data. It is advantageous that then a distinction can be made regardless of the aspect ratio. Recognition algorithms for the recognition of everyday objects in pictures are known per se. To train appropriate Convolutional Neural Networks (CNN), for example, the image database Image-Net is used. "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications," Andrew G. Howard et al., ArXiv: 1704.04861v1 [cs.CV], describes an algorithm for detecting objects in images.

8th schematically shows a method for recognizing a stereoscopic format of image data of the source image stream 10 based on image contents of the image data by means of feature detectors. The method can, for example, in step S12 of the example of 5 be executed.

In one step S30 becomes a hypothesis to be checked about a stereoscopic format of the 3D image data of the source image stream 10 in particular, a hypothesis to be checked about a stereoscopic encoding format of left image data and right image data within the assumed 3D image data region in step S12 , in particular, a hypothesis to be checked about an arrangement of left image data and right image data within the assumed 3D image data region.

In one step S32 paired features in left image contents and right image contents of the 3D image data are detected by means of a feature detector, wherein the assumption of which image data represents the left image contents and which image data represents the right image contents is made according to the current hypothesis , It is a common practice to use so-called local feature detectors for finding corresponding objects in stereo image pairs. These detect well-locatable regions in images and assign each a description in the form of a feature vector, which makes it possible to identify similar regions. In this way it can be detected to individual image regions, if and if so where these regions are in a second image of a stereo pair. Examples of feature detectors usable for this purpose are SIFT ( US Patent 6,711,293 ), SURF, GLOH, FAST, MSER and SKB ( WO 2012/136642 A1 ).

In the case of a valid stereo image pair, corresponding regions or features lie approximately at the same y-coordinate, depending on the disparity of the x-coordinate. By decomposing an input image into potential stereo image pairs depending on the possible 3D format (i.e., according to the particular hypothesis), the 3D format actually present is characterized by two conditions.

A first condition is that a critical minimum set of corresponding regions or features pairs in the y- Coordinate not more than a critical threshold (if there are several corresponding, ie sufficiently similar regions in the other stereo image, to one region in a stereo image, the closest correspondence pair is taken). This similarity criterion for the y-coordinates of the pairwise corresponding features becomes for the current hypothesis in step S34 checks; if not, the process goes to step S38 in which it is determined whether there is yet another hypothesis to check; in this case, the procedure will start from step S30 continued for the next hypothesis to be tested. If the similarity criterion in step S34 is met in step S36 calculates a measure of the scattering of the pairwise differences of the x-coordinates of the pairwise corresponding features, for example the variance in the pairwise difference of the x-coordinates of the corresponding features. In step S38 if there is another hypothesis to check, continue with step S30 for the next hypothesis to be checked.

If in step S38 All hypotheses have been checked in step S40 among all the hypotheses satisfying the first condition, a determination of the hypothesis having a greatest value of the measure for the dispersion of the pairwise differences of the x-coordinates of the pairwise corresponding features.

This is the second condition for the actual 3D format. This can ensure that e.g. a stereoscopic side-by-side image is not interpreted as a stereoscopic line interlaced image. Hypotheses for which the measure of the scattering lies below a lower threshold value or is above an upper threshold value can be excluded from this evaluation.

The hypotheses to be tested on a stereoscopic format of the 3D image data of the source image stream 10 can also be checked simultaneously or at the same time. That is, instead of a selection in step S30 and a return in step S38 become the steps S32 to S36 for the respective hypotheses executed in parallel. It will then be in step S40 selected the best hypothesis in comparison of the hypotheses.

In step S42 it is determined if any of the potential 3D formats meets the first and second conditions. If no, a 2D format is assumed. In step S42 it is then decided that the potential 3D image data does not have a stereoscopic format. If so, the hypothesis in question is recognized as true, and the corresponding stereoscopic format is recognized as the stereoscopic format of the 3D image data of the source image stream.

Moreover, in a true 3D hypothesis hypothesis corresponding to a true hypothesis of left and right fields in the 3D image data of the source image stream, a distinction may be made between 3D formats with or without compression of the image content in the direction of the arrangement of the left partial images and right partial images are taken. This may be on top of the step S24 described ways. Again, recognizing side-by-side (full), top-and-bottom (full), or frame-packing formats based on aspect ratio, but also before recognizing the stereoscopic format of the steps S30 to S42 done as described above.

9 schematically shows a conversion of an image 12 a source image stream into an image 22 a target image stream 20 according to a further embodiment. The input image 22 contains here 3D image data, which are color-anaglyphic encoded, for example as a red-blue anaglyph image 60 , as well as 2D image data of a 2D field 14 , A red-blue anaglyph image does not contain color information. The red channel of the analglyph image 60 becomes as left partial picture 16 taken as a brightness value (gray value) on all R, G, B color channels of the left field 26 of the generated output image 22 spent; accordingly, the blue channel of the analglyph image becomes 60 as right-part image 18 taken as a brightness value (gray value) on all R, G, B color channels of the right field 28 of the generated output image 22 output. The 3D image data are thus output monochromatically. Incidentally, the conversion corresponds to the conversion according to 1 , A red-green anaglyph image can be converted accordingly. In color combinations where original color information may be included in the anaglyph image, such as a red cyan anaglyph image, an approximate reconstruction of color information for the subpictures may also be used 26 . 28 from the two partial images 16 . 18 done in a conventional manner by correspondence analysis. The detection methods described by means of artificial neural network or by feature detection can be easily extended to the recognition of color anaglyphic encoded stereoscopic formats, if, for example, to detect corresponding features, the extracted left and right fields are evaluated monochromatically in the hypothesis concerned, or For example, in the preprocessing of image contents for an artificial neural network, the average squared sum of the differences of (h) red fractions and blue fractions of pixels is additionally calculated as a feature.

10 schematically shows a conversion of an image 12 a source image stream into an image 22 a target image stream 20 according to a further embodiment, which differs from the example of 9 differs in that no conversion of colors takes place. Thus, a red-cyan anaglyph image may allow for colored stereoscopic representations. The output of the left and right fields 16 . 18 takes place here with the in the input image 12 present colors. In a red-cyan Anaglyphenbild but also the method according to the example of 9 be used.

11 schematically shows a conversion of images 12 a source image stream 10 in pictures 22 a target image stream 20 according to a further embodiment. The pictures 12 the source image stream 10 are designed for the use of the Pulfrich effect. This can be recognized by the fact that there is a predominant movement of image contents from right to left or vice versa. As a right field 28 for an output image to be generated 22 becomes, for example, a drawing file 62 an input image 12 taken as the left-part image 26 for the same output image to be generated 22 a drawing file 64 a picture one frame or two pictures behind 12 ' the source image stream 10 or vice versa, depending on the encoding format). The mutually associated 3D image data lie in successive images 12 . 12 ' in front. For example, the predominant motion may be associated with the background so that it can be seen from the direction of motion whether the left or right eye should receive the delayed image. The artificial neural network detection methods or feature detection methods described above can easily be extended to the recognition of stereoscopic formats designed for the use of the Pulfrich effect if, for example, to detect corresponding features, the left and right partial images in the relevant hypothesis follow successive images of the stereoscopic formats Source image stream are taken or, for example, in the preprocessing of image content for an artificial neural network additionally as a feature the above-described global image feature (f) and possibly also (g) is calculated. When to use the Pulfrich effect of appropriate image content decreases the degree of similarity with each additional frame, as opposed to the frame interlaced format.

Moreover, the artificial neural network detection methods or feature detection techniques described above can be easily extended to include the recognition of a frame interlaced format; This has already been described above for an artificial neural network. For example, when feature detectors are used, corresponding features are determined in image contents of a frame f (n), an immediately preceding frame f (n-1), and a frame f (n-2) immediately preceding it. The sign of the horizontal position differences can be used to identify which frame corresponds to left-hand image data and which frame corresponds to right-hand image content. Thus, the hypothesis can be made about the format and the steps as described above S34 and following. In a frame interlaced format, due to the disparity, the horizontal position differences of corresponding features between frame f (n) and frame f (n-1) are significantly larger than between frame f (n) and frame f (n-2). This criterion for the y-coordinates of the features corresponding to three frames may for this hypothesis in addition to the similarity criterion in step S34 be checked. In a format for exploiting the Pulfrich effect, on the other hand, the horizontal position differences of corresponding features continue to increase with a larger time interval, ie are greater between frame f (n) and frame f (n-2) than between frame f (n) and frame f (n-1). This criterion for the y-coordinates of the features corresponding to three frames may for this hypothesis in addition to the similarity criterion in step S34 be checked.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US Pat. No. 6711293 [0082]
• WO 2012/136642 A1 [0082]

Claims (9)

A method of converting a source image stream (10) containing image data in a stereoscopic format into a target image stream (20) having images (22) in a predetermined stereoscopic format, each image (22) being left-aligned. Comprising a left eye partial image (26) and a right eye partial image (28), the method comprising the steps of: Recognizing a stereoscopic format of image data of the source image stream (10) based on image contents of the image data, Taking left-hand sub-images (16) for a left eye and right-field images (18) for a right eye from image data of the source image stream (10), said image data of the source image stream (10) being interpreted according to the recognized stereoscopic format . Generating images (22) of the target image stream (20) in the predetermined stereoscopic format, including the extracted left fields (16) and right fields (18). Method according to Claim 1 wherein detecting a stereoscopic format of image data of the source image stream based on image contents of the image data comprises: checking (S30, S32, S34, S36, S38, S40) of plural hypotheses about a stereoscopic format of image data of the source image stream (10) And recognizing (S42) the stereoscopic format of a hypothesis as the stereoscopic format of the image data if, in the step of checking, the hypothesis has been recognized as true. Method according to Claim 2 wherein examining a plurality of hypotheses comprises: detecting (S32) for a respective hypothesis of hypotheses to be tested, pairwise corresponding features in left image contents, and right image contents of the image data, the image data being the left image contents according to the hypothesis in question and the right-hand picture contents are interpreted representative; Detecting (S34, S36, S40) a hypothesis as true based on pairs of mainly differing positions of detected, pairwise corresponding features in their x-coordinates. Method according to Claim 3 in which, in the step of recognizing a hypothesis, the finding of a hypothesis as true is made if the hypothesis among those hypotheses for which a similarity criterion for the y-coordinates of the pairwise corresponding features is satisfied for the hypothesis in question is of the greatest value a measure of the variance of the pairwise differences of the x-coordinates of the pairwise corresponding features. Method according to one of Claims 1 to 4 in which the recognition of a stereoscopic format of image data of the source image stream is based on image contents of the image data by means of an artificial neural network (S22). Method according to Claim 5 wherein recognizing a stereoscopic format of image data of the source image stream (10) based on image contents of the image data includes preprocessing (S20) image contents of the image data, extracting global image features from image contents of the image data, and extracting the artificial neural networks receives global image features as input signals. The method of any one of the preceding claims, wherein the method comprises: Recognizing (S10, S12, 14, S16) a perimeter of an image area of image data having a stereoscopic format. The method of any one of the preceding claims, wherein the method comprises: Recognizing a perimeter of an image area of 2D image data, and Extracting 2D sub-images (14) from 2D image data of the source image stream (10), wherein generating images (22) of the target image stream (12) in the predetermined stereoscopic format including the extracted left sub-images (16) and right sub-images (18) and including the extracted 2D sub-images (14) be duplicated. Apparatus for converting a source image stream (10) containing image data in a stereoscopic format into a target image stream (20) having images (22) in a predetermined stereoscopic format, each image (22) being left-aligned A right eye partial image (26) for a right eye and a right partial image (28), comprising: recognition means (42) for recognizing a stereoscopic format of image data of the source image stream (10) based on image contents of the image data; and conversion means (44) for converting the source image stream (10) into the target image stream (20), the apparatus for performing the method according to any one of Claims 1 to 8th is set up.

Images