DE102004006246A1 - Areal scenes and objects recording method, involves creating two-dimensional image from sharp image areas of observed n-sequence, by which two-dimensional image and depth chart are obtained - Google Patents

Abstract

The method involves simultaneously recording a set of images and making sharpness analysis of images to find which areas of the images are recorded sharply or not. A deep edge is created by allotting the sharp areas on the image to corresponding depth levels of an object and/or a camera. A two-dimensional (2D) image is created from respective sharp image areas of observed n-sequence, by which the 2D image and a depth chart are obtained. An independent claim is also included for an arrangement for recording areal scenes and objects.

Description

The The invention relates to methods and arrangements for receiving of spatial Scenes or objects, specifically on such procedures and orders containing information about the spatial Deliver appearance.

Of the The prior art knows in this regard Another method in which a single camera in dynamic Way to different depth levels of the scene to be recorded sharp and according to the respectively sharply imaged object portions assigns a depth. The disadvantage here is that the scenery only unmoved Objects may contain, as otherwise unreadable depth information delivered become.

In contrast, is It is an object of the invention to provide a method by means of which spatial scenes and objects in spatial Can be recorded the scenery may change with time, without the recording results to influence significantly (negatively). It's also like that Specify method implementing arrangement.

• a) gleichzeitige Aufnahme einer Anzahl von n Bildern mit n≥2 Objektiven bzw. Kameras, wobei die n≥2 Objektive bzw. Kameras im wesentlichen aus der gleichen Position auf die Szene bzw. den Gegenstand gerichtet sind, jedoch auf paarweise unterschiedliche Tiefenebenen in der Szene bzw. innerhalb des Gegenstandes scharfgestellt sind, wodurch ein n-Tupel von Bildern aufgezeichnet wird,
• b) gleichzeitig mit der Ausführung des Schrittes a) oder zeitlich nach der Ausführung des Schrittes a) folgende Schärfeanalyse der einzelnen Bilder des aufgezeichneten n-Tupels von Bildern zur Festlegung, welche Bildbereiche der einzelnen Bilder des betrachteten n-Tupels von Bildern scharf und welche unscharf aufgenommen wurden,
• c) Erstellen einer Tiefenkarte zu dem betrachteten n-Tupel von Bildern durch Zuordnung der jeweils in einem Bild des n-Tupels scharf aufgezeichneten Bildbereiche zu der entsprechenden Tiefenebene desjenigen Objektives bzw. derjenigen Kamera, vermöge dessen bzw. vermöge derer der betreffende Bildbereich aufgezeichnet wurde,
• d) Erstellen eines 2D-Bildes aus den jeweils scharf aufgenommenen Bildbereichen des betrachteten n-Tupels ODER gleichzeitig mit der Ausführung des Schrittes a) durchgeführte Aufnahme eines gewöhnlichen 2D-Bildes mit einem (n+1). Objektiv bzw. mit einer (n+1). Kamera mit möglichst großer Tiefenschärfe,

wodurch im Ergebnis ein 2D-Bild der aufgenommenen räumlichen Szene bzw. des aufgenommenen räumlichen Gegenstandes sowie eine zugehörige Tiefenkarte entsteht.This object is achieved according to the invention by a method for recording spatial scenes or objects, comprising the following method steps:

• a) simultaneous recording of a number of n images with n≥2 lenses or cameras, wherein the n≥2 lenses or cameras are directed substantially from the same position on the scene or the object, but on pairs of different depths levels in the Are focused within the object, whereby an n-tuple of images is recorded,
• b) simultaneously with the execution of step a) or temporally after the execution of step a) following sharpness analysis of the individual images of the recorded n-tuple of images to determine which image areas of the individual images of the considered n-tuple of images sharp and which blurred were recorded
• c) creation of a depth map to the considered n-tuple of images by assigning the respective image areas sharply recorded in one image of the n-tuple to the corresponding depth plane of the lens or camera by virtue of which the image area in question has been recorded,
• d) taking a 2D image from the respectively sharply recorded image areas of the considered n-tuple OR simultaneously with the execution of step a) taking a usual 2D image with one (n + 1). Lens or one (n + 1). Camera with the greatest possible depth of field,

resulting in a 2D image of the recorded spatial scene or the recorded spatial object and an associated depth map as a result.

at Step c) may possibly be the possible Offset of the images of the observed n-tuple considered among each other which is because of juxtaposing the lenses / cameras arises. Likewise, it is at the creation mentioned in step d) a 2D image from the sharply captured image areas of the considered n-tuple advantageous, the possible offset of the images to consider the n-tuple among each other, which also because of the juxtaposition of the lenses / cameras caused becomes.

Advantageous in step a), the simultaneous recording of n images by means of n≥2 lenses or Cameras performed, which each such depths of field show that no or almost no depth plane of at the same time two different ones Lenses or different cameras is sharply mapped. In order to the depth assignment becomes the respectively sharply imaged objects clearly held.

The performed in step b) focus analysis of the individual images of the recorded n-tuple of images preferred assets one or more actions from the group consisting of edge detection, high-pass filtering, low-pass filtering, bandpass filtering or Fourier transform.

The inventive method can also be extended by a process step e), the includes:

• - Post processing at least one created depth map and / or one created 2D image, preferably in an electronic computing device, by execution at least one of the following actions:
• - antialiasing,
• - soft focus,
• - integrate of color or grayscale gradients, and / or
• - filling of information holes in the 2D image or in the depth map with mean values of surrounding image or depth information.

Furthermore, the depth planes to which the objects / cameras in step a) each sharp len, before the respective implementation of step a) are adapted to the scene or to the object by determined by a fast focussing and subsequent sharpening analysis of the scene to be recorded or the object to be recorded, the foremost and backmost sub-object in the scene or object be divided and said depth levels within the resulting maximum depth difference between foremost and backward object. Thus, the method according to the invention can also be adapted to sceneries which are substantially changeable in their depth extent. In this case, the division of the depth planes, on which the n≥2 objects or cameras focus, can take place linearly, potentially or exponentially spaced within said maximum depth difference.

to Recording of moving images becomes the method according to the invention repeated several times, so that in fast succession, a plurality of such n-tuples of images recorded and created a corresponding number of depth maps becomes, whereby a spatial Moving picture sequence is generated.

One special embodiment provides that in step a) all images of the n-tuple are taken through a main aperture, which follow one or more semipermeable mirrors which the light on the n≥2 Directing lenses or cameras, so that all n images of the n-tuple from essentially recorded in the same recording position, whereby the displacement of the Image information between different images of an n-tuple, which from the juxtaposition of n≥2 Lenses or cameras, is essentially bypassed.

• – Abspeicherung des oder der Paares) bestehend aus 2D-Bild der aufgenommenen Szenerie bzw. des aufgenommenen Gegenstandes sowie jeweils zugehöriger Tiefenkarte auf bzw. in einem Speichermedium, oder
• – Übertragung des oder der Paares) bestehend aus 2D-Bild der aufgenommenen Szenerie bzw. des aufgenommenen Gegenstandes sowie jeweils zugehöriger Tiefenkarte in einem Übertragungskanal.

Furthermore, the method according to the invention can be extended by a step f) following step d) or step e), which includes the following:

• - Storage of the or the pair) consisting of 2D image of the recorded scene or the recorded object and each associated depth map on or in a storage medium, or
• - Transmission of the or the pair) consisting of 2D image of the recorded scene or the recorded object and each associated depth map in a transmission channel.

• – eine Anzahl n≥2 Kameras, welche gleichzeitig ausgelöst werden können, um gleichzeitig ein n-Tupel von Bildern aufzunehmen, wobei die n≥2 Kameras im wesentlichen aus der gleichen Position auf die Szene bzw. den Gegenstand gerichtet sind, jedoch auf paarweise unterschiedliche Tiefenebenen in der Szene bzw. innerhalb des Gegenstandes scharfgestellt sind,
• – Mittel zum Auslesen des n-Tupels von Bildern aus den n Kameras sowie Mittel zur Speicherung des n-Tupels von Bildern,
• – Mittel zur Ausführung einer Schärfeanalyse der einzelnen Bilder des aufgezeichneten n-Tupels von Bildern zur Festlegung, welche Bildbereiche der einzelnen Bilder des betrachteten n-Tupels von Bildern scharf und welche unscharf aufgenommen wurden,
• – Mittel zum Erstellen einer Tiefenkarte zu dem betrachteten n-Tupel von Bildern durch Zuordnung der jeweils in einem Bild des n-Tupels scharf aufgezeichneten Bildbereiche zu der entsprechenden Tiefenebene desjenigen Objektives bzw. derjenigen Kamera, vermöge dessen bzw. vermöge derer der betreffende Bildbereich aufgezeichnet wurde,
• – Mittel zum Erstellen eines 2D-Bildes aus den jeweils scharf aufgenommenen Bildbereichen des betrachteten n-Tupels ODER eine weitere (n+1). Kamera zur Aufnahme eines gewöhnlichen 2D-Bildes.

The invention also relates to the method according to the invention implementing arrangements. Such an arrangement according to the invention comprises at least:

• A number of n≥2 cameras, which can be triggered simultaneously to simultaneously capture an n-tuple of images, with the n≥2 cameras being directed from the same position to the scene or object, but in pairs Depth planes in the scene or within the subject are in focus,
• Means for reading out the n-tuple of images from the n cameras, and means for storing the n-tuple of images,
• Means for performing a sharpness analysis of the individual images of the recorded n-tuple of images to determine which image areas of the individual images of the considered n-tuple of images were sharply and which were taken blurred,
• - Means for creating a depth map to the considered n-tuple of images by assigning each sharply recorded in an image of the n-tuple image areas to the corresponding depth plane of the lens or the camera, by virtue of or by virtue of which or the relevant image area was recorded .
• Means for producing a 2D image from the respective sharply recorded image areas of the considered n-tuple OR another (n + 1). Camera for taking a usual 2D image.

The said means for reading the n-tuple, means for executing a Sharpness analysis, the Means for creating a depth map and possibly the means for Can create a 2D image particularly advantageously formed by a software controlled PC become.

Also the inventive arrangement can be extended by a main screen, the scene or the object is facing. The extension also advantageously includes one or more semipermeable Mirrors which follow the main panel and which light up the n≥2 lenses or direct cameras, so that all Images of the n-tuple through one and the same main aperture and thus essentially taken from the same receiving position become.

The Invention will be explained in more detail with reference to drawings.

It shows

1 - 8th the result of the sharpness analysis for an n-tuple of recorded images at n = 8, as well

9 blurred the resulting depth allocation.

• a) gleichzeitige Aufnahme einer Anzahl von n Bildern mit n=8 Kameras, wobei die n=8 Kameras im wesentlichen aus der gleichen Position auf die Szene bzw. den Gegenstand gerichtet sind, jedoch auf paarweise unterschiedliche Tiefenebenen in der Szene bzw. innerhalb des Gegenstandes scharfgestellt sind, wodurch ein n-Tupel von Bildern aufgezeichnet wird,
• b) zeitlich nach der Ausführung des Schrittes a) folgende Schärfeanalyse der einzelnen Bilder des aufgezeichneten n-Tupels von Bildern zur Festlegung, welche Bildbereiche der einzelnen Bilder des betrachteten n-Tupels von Bildern scharf und welche unscharf aufgenommen wurden,
• c) Erstellen einer Tiefenkarte zu dem betrachteten n-Tupel von Bildern durch Zuordnung der jeweils in einem Bild des n-Tupels scharf aufgezeichneten Bildbereiche zu der entsprechenden Tiefenebene derjenigen Kamera, vermöge dessen bzw. vermöge derer der betreffende Bildbereich aufgezeichnet wurde,
• d) gleichzeitig mit der Ausführung des Schrittes a) durchgeführte Aufnahme eines gewöhnlichen 2D-Bildes mit einer 9. Kamera mit großer Tiefenschärfe,

wodurch im Ergebnis ein 2D-Bild der aufgenommenen räumlichen Szene bzw. des aufgenommenen räumlichen Gegenstandes sowie eine zugehörige Tiefenkarte entsteht.The test scene for the drawings is a chessboard scene simulated with 3D-Studio Max. In the exemplary embodiment of the drawings for recording spatial scenes or objects, the following method steps were carried out:

• a) simultaneous recording of a number of n images with n = 8 cameras, wherein the n = 8 cameras are directed essentially from the same position on the scene or the object, but on pairs of different depth levels in the scene or within the object are focused, whereby an n-tuple of images is recorded,
• b) temporally after the execution of step a) the following sharpness analysis of the individual images of the recorded n-tuple of images to determine which image areas of the individual images of the considered n-tuple of images were sharply and which were taken blurred,
• c) creating a depth map to the considered n-tuple of images by assigning each of the image areas sharply recorded in one image of the n-tuple to the corresponding depth plane of the camera by virtue of which or the image area in question has been recorded,
• d) taking a common 2D image simultaneously with the execution of step a) with a 9th camera with a large depth of field,

resulting in a 2D image of the recorded spatial scene or the recorded spatial object and an associated depth map as a result.

There the eight two-dimensional images of the n-tuple and the (9th) 2D image are for reasons the redundancy not shown, i. the results from the steps a) and d) are not shown in the drawings.

So there are the 1 - 8th the result of step b), ie the sharpness analysis for the n-tuple of recorded images at n = 8 again. The image sharpness analysis has been carried out here by way of example with a high-pass filter. In each case, sharply drawn edges correspond to sharply displayed image sections in the n-tuple, while the blurred edges correspond to blurred image areas in the respective image of the n-tuple.

The 9 shows schematically the depth assignment obtained from the data of the sharpness analysis according to step c). The depth map was created by assigning each of the image areas sharply recorded in one image of the n-tuple (which were determined by the sharpness analysis in step b) to the corresponding depth plane of the camera by virtue of which the image area in question was recorded were. Thus, with n = 8 cameras exactly 8 different depth steps can be assigned, which is in 9 symbolized by 8 different shades of gray.

Out the 2D image (taken here by the ninth camera, not in the drawings shown) and the depth map, for example, the recorded 3D scene reconstructed in different views and on one 3D screen, for example of the type "X3D-19 Display". Thus, the obtained 2D image with the associated depth map a sufficiently spatial image the recorded scene or the object.

The Advantages of the invention are versatile. In particular, it is possible, too from time-varying and temporally in depth varying scenes or objects one spatial To make a recording. The implementation will not be highly complex Optics needed.

Claims (12)

A method for recording spatial scenes or objects, comprising the following method steps: a) simultaneous recording of a number of n images with n≥2 lenses or cameras, wherein the n≥2 lenses or cameras essentially from the same position on the Are focused on pairwise different depth planes in the scene or within the object, whereby an n-tuple of images is recorded, b) simultaneously with the execution of step a) or after the execution of the Step a) following sharpening analysis of the individual images of the recorded n-tuple of images to determine which image areas of the individual images of the considered n-tuple of images were sharp and which were blurred, c) creating a depth map to the considered n-tuple of images by assigning each sharply recorded in an image of the n-tuple image areas to the corresponding depth plane of that lens or that camera, by virtue of whose or by means of which the relevant image area has been recorded, d) creating a 2D image from the respectively sharply recorded image areas of the considered n-tuple OR simultaneously with the execution of step a) performed recording an ordinary 2D image with one (n + 1). Lens or one (n + 1). Camera with the greatest possible depth of field, resulting in a 2D image of the recorded spatial scene or the recorded spatial object and an associated depth map is created. Method according to claim 1, characterized in that that in Step a) the simultaneous recording of n images by n≥2 lenses or cameras performed is, which each have such depths of focus that no or almost no depth plane of at the same time two different ones Lenses or different cameras is sharply mapped. Method according to one of the preceding claims, characterized characterized in that performed in step b) focus analysis of the individual images of the recorded n-tuple of images by virtue of a or multiple actions from the group consisting of edge detection, High-pass filtering, low-pass filtering, bandpass filtering or Fourier transform is executed. Method according to one of the preceding claims, extended a process step e), which includes: post processing at least one created depth map and / or one created 2D image, preferably in an electronic computing device, by execution at least one of the following actions: - antialiasing, - soft focus, - integrate of color or grayscale gradients, and / or - filling of information holes in the 2D image or in the depth map with mean values around Image or depth information. Method according to one of the preceding claims, characterized characterized in that Depth planes pointed to by the objects / cameras in step a) respectively focus, before the respective implementation of step a) to the Scene or adapted to the object by a fast focussing and subsequent sharpness analysis of the recorded Scene or the object to be recorded, the foremost and rearmost Part object can be determined in the scene or in the object and said depth levels within the resulting maximum Depth difference between foremost and rearmost object divided become. Method according to claim 5, characterized in that that the Division of the depth levels to which the n≥2 objects or cameras focus within said maximum depth difference linear, potential or exponential spaced apart. Method according to one of the preceding claims, characterized characterized in that Procedure is repeated several times, so that in rapid succession a variety of such n-tuples of images taken and a corresponding one Number of depth maps is created, creating a spatial Moving picture sequence is generated. Method according to one of the preceding claims, characterized characterized in that Step a) pass all images of the n-tuple through a main aperture be included, which one or more semi-transparent mirror follow that direct the light to the n≥2 lenses or cameras, so that all n images of the n-tuple from essentially the same pickup position be absorbed, thereby reducing the image information between different images of an n-tuple, which from the juxtaposition the n≥2 lenses or cameras, is essentially bypassed. Method according to one of the preceding claims, extended by a step f) following step d) or step e), includes: - storage of the couple) consisting of a 2D image of the recorded scene or of the recorded object and respectively associated depth map on or in a storage medium, or - Transfer of the couple (s) consisting of a 2D image of the recorded scene or the recorded scene Object and each associated Depth card in a transmission channel. Arrangement for recording spatial scenes for carrying out the method according to any one of claims 1-7, comprising at least: - n≥2 cameras which can be triggered simultaneously to simultaneously record an n-tuple of images, wherein the n≥ 2 cameras are directed substantially from the same position on the scene or the object, but are focused on pairwise different depth levels in the scene or within the object, - means for reading the n-tuple of images from the n cameras and means for storing the n-tuple of images, means for performing a sharpness analysis of the individual images of the recorded n-tuple of images to determine which image areas of the individual images of the n-tuple of images were sharp and which were blurred, means for Creating a depth map to the considered n-tuple of images by assigning each in one Image of the n-tuple sharply recorded image areas to the corresponding depth plane of the lens or camera by virtue of which or by virtue of which the relevant image area has been recorded, Means for producing a 2D image from the respective sharply recorded image areas of the considered n-tuple OR another (n + 1). Camera for taking a usual 2D image. Arrangement according to claim 10, characterized that the Means for reading the n-tuple, the means of execution a sharpness analysis, the Means for creating a depth map and possibly the means for Creating a 2D image through a software controlled PC be formed. Arrangement according to one of claims 10 or 11, extended by a main panel facing the scene or object also expanded around one or more semitransparent mirrors, which follow the main panel and the light on the n≥2 lenses or direct cameras, so that all Images of the n-tuple through one and the same main aperture and thus be taken up substantially from the same recording position.