DE102004007608A1 - Variable focus video camera records image sequences with different focus settings and processes data to create single image with larger depth of field - Google Patents

Abstract

A variable focus video camera records image sequences with different focussing and uses the focussing information to calculate new images from several images in the sequence.

Description

State of technology

Autofocus functions are the state of the art in optics and usually allow motorized the automatic finding of an optimal focusing parameter in one or several measured pixels, in particular the center of the image, to gain an image that is at least the main object of the image sharp draws.

Some Methods for this purpose are described in [DE0069515833T2], [DE0069132196T2], [DE0069127112T2], [DE0069124167T2] or [DE0069120455T2] and references therein Find.

there become objects and surfaces with positive as well as negative distance to the focused object the optical laws of lens-based imaging are increasingly blurred drawn.

Around the blur of objects outside the to reduce the adjusted focus distance are further Various, mostly optical methods have been invented, the one achieve greater depth of focus are supposed to increase the sharp-drawing area.

As well Mathematical methods exist to subsequently calculate blurred image areas "sharply".

All However, known optical methods can for reasons the physics of optical imaging only a limited range a scene actually draw sharply. Mathematical methods, however, have their limitations there, where information is missing due to the blur and therefore need to be extrapolated or interpreted.

Task of invention

Of the Invention has for its object to develop a method with the depth of field information of objects, image areas or pixels can be determined so that of several pictures with different focus the respectively sharpness-optimized Extracts objects, image areas or pixels and from them new image with optimized, extended or modified depth of focus won can be.

The so gained information can do so used to get out of a sequence of images of the same Scene to extract those image components of individual images and combine, for example, which simulate an infinite depth of field, by selecting those parts of the individual images whose sharpness is optimal.

Further Is it possible, adjust the depth range as desired or alienated and modified depth of field realize which with optical methods alone not represent would. This includes e.g. inverse depth of field with big blur at a certain distance and greater sharpness with increasing distance to it, visual effects as artistic Alienation, distance analysis, separation and exemption of pictorial components and other effects resulting from the depth of field control result.

description the invention

These The object is achieved by equipment for the digital recording of image data with the possibility of the focusing plane to vary.

Especially These are digital photo and video cameras that are both direct Recording image data in digital form, the technology for motor-driven change the focusing already included by default, as well as the Electronics for controlling the optics and recording components, and the Evaluation of the data, or about the transfer of the data to e.g. external Programs on a computer to enable the compute-intensive evaluation.

It However, other possibilities are conceivable, such as. analog recordings that are digitized later the computational extraction from any video material, neural Evaluation networks, as well as analog techniques with comparable function.

suitable equipment or method in the context of this invention allow the recording or Extraction of image sequences of different sharpening values, as well as their evaluation in the way that from the dataset to objects, Image areas or pixels Focusing or sharpening values can be won a simultaneous or subsequent analysis or extraction of objects, image areas or pixels are accessible, as well as their recombination to images, objects, image areas or pixel sets.

By a suitable recording device For example, image sequences in e.g. digital form won, respectively the same scene or specific objects with different focus record.

By simultaneous or subsequent evaluation, objects, image areas or image points one or more values associated with, for example, a Fokussiergüte, a focus adjustment (eg position of the focusing mechanism in a camera) or an image of the sequence in which a desired focus is approximated, which may be optimally or specifically deviating from it.

The Data obtained in this way allows objects, image areas or pixels with the desired focus in the image sequence to detect this e.g. to a new picture with predetermined sharpness properties too combine or focus-dependent evaluations or manipulations perform.

One Finding objects, image areas or pixels specific sharpness in this sense can be done by e.g. for each or any selected one Pixels (e.g., every 2nd, 4th, etc., to speed up the evaluation), Points of an image area or pixels associated with an object (Representatives) the corresponding pixels in the sequence for their sharpness examined and with a sharpness value be correlated.

For this are often the surrounding pixels on contrast, or Strength the high frequency components and, e.g. with the hill-climbing algorithm the (maximum) value is determined.

at Determining an infinite depth of field would make the image with the maximum value the sharpness as a reference for the extraction of the object, image area or pixel is noted, to extract these from just this picture and into a generated one Transfer target image.

For a distance decomposition a picture would for each found sharpness value or intervals of sharpening values Objects, image areas or pixels separated or marked, that a later separation possible becomes. This can e.g. take place by adding an additional to the image sequence Record is created for Objects, image areas or pixels contains a value that a corresponding separation allowed. In the simplest case, this one has Record the same dimension as the images of the image sequence, and contains a sharpening value or information from which this can be obtained.

Out This data can therefore be for Objects, image areas or pixels optimal or purposefully different Focusing and gain objects, image areas or pixels extract with appropriate focus and to new images, Combine objects, image areas or pixels.

Especially In this way images of infinite depth of field can be obtained, by for each Object, any image areas or pixels the corresponding the amount of data is selected, has the optimal sharpening.

As well can extract objects, image areas or pixels and to combine a new image, which different sharpness functions suffice. This may correspond to an extended depth of field for a certain focus value or interval of focus values the image responses of optimal focus values are taken while using increasing deviation from deviant, but less blurred Objects as in optical blur chosen become. It can also be any other selection function from the available focus ranges to objects, image areas or pixels, including reverse focus functions, the e.g. Objects, image areas or pixels of a particular Focusing plane out of focus while with increasing distance sharper Extracted images of these objects, image areas or pixels become.

Further You can use objects, image areas or pixels of the same or similar Select sharpening to extract distances, or to separate. In this way, images in different areas Separate distance, manipulate and modify again combine or link to other graphics.

Especially allows this is the three-dimensional analysis of an image, the use of this information for visual Effects of computer animation, as used for modern movie tricks and the Combination of real images with computer animations to be used as well as artistic Edits, modifications and alienations.

effective Applications of the method of this patent are e.g. the 1-pass and the 2-pass recording.

at the 1-pass recording takes the device a picture sequence with different Focusing on, intended for each or a few pixels of the image dimension that image of Sequence in which this one desired Has focus and takes over the pixel (s) from this image into the target image.

The 1-pass method works with large amounts of data and requires corresponding memory, the evaluation, however, can be particularly simple and non-time critical after recording and if necessary through an external device (especially personal computer) happen.

At the 2-pass procedure using the example of a suitable, extended digital camera focussing is reproducible from e.g. Close to infinity drove the data of the pictures taken to the desired sharpness value analyzed and to each or a selection of points of the target image notes the setting that corresponds to the desired sharpness value. At the following, rerunning the focus change then becomes image areas for the selected objects or pixels taken from the image of the sequence in the target image, their focus setting of the desired focusing function corresponds (for example, each with the utmost sharpness).

The 2-pass procedure required less memory than the 1-pass method, but a higher processing speed in the recording device. The Information of the picture sequence and the different sharpening values get lost, so it's best for given Focus Features, like infinite depth of field, suitable.

Appropriate focus features can as fixed programs are stored in the recording device. Most of time should be used the infinite depth of field, but inverse depth of field to a reference point or modified depth of field, e.g. to simulate certain lens focal lengths would be conceivable as sharpness to to a given reference point and stronger than optically given increasing or very big fuzziness from the distance given by the reference point (emphasis on Foreground).

The 2-pass procedure can be speeded up by changing the Focusing on the analysis in a "way out" and the extraction of the respective ones of interest Pixels in the "return path" of the focusing mechanism happens ("path optimization").

Both Methods can be combined and extended by "intermediate solutions".

As well Is it possible, Record moving pictures (videos) with this technique.

For this have to for each the pictures of the video several pictures of different focussing be recorded in accordance with high speed, for each the images of the target video above, corresponding or comparable Leading results Apply method.

Claims (5)

Image recording devices with variable focus adjustment, characterized in that image sequences recorded with different focus and due to the additionally noted, contained therein or to be determined therefrom focusing information on several images of the sequence new images are calculated. Video recording equipment according to Claim 1, characterized in that it images together with focusing information a downstream evaluation whose purpose is the processing of this Image and focusing information to new images is. Video recording equipment according to Claim 1, characterized in that they image sequences of different Focusing on a downstream evaluation whose purpose is to process the data additionally noted, contained or to be determined from this focussing information new pictures is. Video recording equipment according to Claim 1 to 3, characterized in that they image sequences of different Focusing so evaluate that new picture sequences or videos arise, which modify the focus of different image areas. Video recording equipment according to Claims 1 to 3, characterized in that they are used in motion pictures (videos) for each Single picture capture sequences and internally or downstream Evaluate such that moving images (videos) with modified depth of field arise.