WO1997025690B1

WO1997025690B1 - Increased depth of field for photography

Info

Publication number: WO1997025690B1
Application number: PCT/SE1997/000017
Authority: WO
Filing date: 1997-01-09
Publication date: 1997-09-25

Abstract

The invention comprises a method and means for achieving increased depth of focus at photography. The method creates an improvement of depth of field at use of traditional 'Tilt and Shift' - technique, and increased possibilities to obtain good depth of field with other and quite ordinary cameras. The method is primarily meant for electronic cameras, where the image is created electronically from a sensor in the camera and is based on a multi-photo technique. Several photos are shot with different focused parts of the scene subjects in respective photo, and a basic image is integrated by contributions from the different images. Calculated image transfers based on lens-/sensor settings for respective image, compose a starting point for corrections of images and selection of respective image contributions.

Claims

AMENDED CLAIMS[received by the International Bureau on 15 May 1997 ( 15.05.97) ; original claims 1-14 replaced by amended claims 1-16 (5 pages) ]

1. A method to increase the depth of focus, giving improved shaφness of subjects with such extensions that shaφness regularly cannot be obtained over the whole picture, using at least two images obtained with different camera settings, which selectively include change of focus, change of aperture, in mainly keeping the same position of the lens optical center relative to the subject, and combining information from those images to obtain one new image, including correction of contours, comprising: processing the result of still cameras by computer means, selectively external to the still camera functions, using data transfered from the camera on-line or through storage media, for principally non-synchronous or off-line processes in the steps of; ' a. matching the different photo-images to the position, scale and perspective of a selected basic image in the steps of; selecting a basic image; identifying and localizing border lines and highly contrasting structures in a number of images; calculating the geometrical transfer of a number of images to the basic image, selectively using position shifts, change of scale and perspective tilting; choosing common references for valuating and/or correcting the matching of the images using the positions of identified common structures; selectively adding at least one image of images, photographed with significantly reduced apertures, increasing the depth of focus and images photographed with tilted focus planes, cutting at least another image focus plane; b. building a new created basic image from the matched images in the steps of; using fast variations of light and or colours as functions of area (x,y) based coordinates for determining the best focused images for an analyzed image element; comparising for best focused states between images, using border lines and highly contrasting structures; selecting for the new image, the identified border lines and structures from respective focused image; selectively watching overlaps and loose ends of the structures; selecting transition positions for structures with continous extensions over neighbouring images, where the structures are divided belonging to different images; selecting border lines for the steps of; analysing a number of border line corridors, narrow pixel areas around borders, for differences on each side of the border line between respective images for the steps of; separating between borders in the same focused image and borders between different focused images; correcting the border line corridor, being a transiton border between different images, reducing the spill over from the defocused corridor side to the focused corridor side; defining transition positions at border lines, where the border line is the border between different focused images; analysing the areas between border lines for indications of image elements with better focus from other images for the step of; defining transition positions between structures belonging to different neighbouring images;

AMENDED SHEET {ARTICLE 19) 14 selectively correcting border line corridors and other transition positions between images by using information from added images, photographed with significantly reduced apertures or with tilted focus planes; selecting additional transition positions for the same images through areas containing slow variations of light and colour and containing no defined structures from other images; building the new created basic image through the transition positions between different images keeping continuity of light level;

Claim 2. A method according to claim 1, further comprising: analysing the speed of variation in light or colour, selectively using derivatives of, relative derivatives of, differences of, and combinations or quotients thereof and of the strenght of light or colours.

Claim 3. A method according to claim 1, further comprising: using the methods of reduced aperture or tilted image planes for correction of defocused spill over across borders between different focused images in the step of; selecting borders, where those methods give a focused image on both sides of the border; selecting this both-side focused image for the border presentation, or one-side focused images of the border corridor, after calculating and reducing the spill over to respective one-side focused image.

Claim 4. A method according to claim 1, further comprising: building the new basic image successively in the steps of; starting at a selected focused image element of a selected image; increasing the examined area continously, while comparing with neighbouring images; making transition of images, when element grows shaφer at another image; increasing the new basic image area with the area from the present image; selectively starting on the front image and successively working towards the background.

Claim 5. A method according to claim 1, further comprising: building the new basic image, selecting strategies in the steps of; counting density of image border lines; selecting low density strategies for large low density areas; selecting high density strategies for small high density areas; building low density strategies on the steps of; connecting selected border lines of he selected image with affiliated border lines by defining transition positions across areas in between; transfering the closed image areas to the new basic image;

Claim 6. A method according to claim 1, further comprising: including to the computer means a man machine interface, MMI, for interactions with an operator; 15 reducing the complexity of the image processing by an operator selectively performing steps of; defining on the computer screen, which parts from which image to be transfered to the new basic image, reducing the processing to mainly image matching and final corrections of the pans in position and border defocusiπg; selecting details from different images for the image matching; selectively defining strategies for different areas for the new image building;

16 - 4TA method according to one of the claims 1 - 3, comprising: determining the defocused width of borders, edges etc. and subsequently correcting the shaφness also for other image elements, selectively by also estimating the image positions of the shaφ focused elements. 5TA method according to one of the claims I - 4, comprising; using consecutive images, calculating the image plane distance of an image element and its real focused extension at that plane, using that for image elements that are defocused in images at both side of the theoretically shaφ image plane. -6. A method according to one of the claims 1 - 5, comprising; inter- or extrapolating geometrical structures, lines, rectangles, circles etc. from neighbouring image(s) to achive focusing in defocused spaces in between, and selectively expanding the method to include storing of geometrical structures in a library for comparisons with obtained image elements e g by use of correlation, and when good agreements, the geometrical structures are used for corrections, the geometric structures possibly also including signs, letters or symbols.

/ J "77 A method according to one of the claims 1 - 6, comprising; correcting at least one of position, scale, perspective for different images, performing position comparisons between corresponding image elements in at least two images, and selectively using successive comparisons between different images, achieving several images related to each other. tl . A method according to one of the claims 1 - 7, utilizing the sensor resolution better, by photographing parts of a product with larger focal distance, the product part covering a larger part of the sensor area, comprising; photographing other parts of the product with the same or different magnification, where also image distances related to the focal distance might vary, and that different images are transfered to a basic image, where the whole image is covered and then with a corresponding higher resolution, where the capacity of the computer is utilized, which can give much higher resolution than that of the sensor , covering the whole final image, and selectively integrating different images, shot while keeping the position of the lens optical center relative to the subject, e g performing that, by turning the camera with the optical lens center as the turning point, and simplifying the image matching by overlapping images, where well defined elements are included and used at comparisons between images for possible corrections, and photographing one or more images with reduced focal distances, where essential parts of the total image are included in the same photo, and then might contribute with general overview information about the basic image.

' *- *f. Means for performing the tasks according to one of the claims 1 - 8, comprising control settings for the cameras, calculation unit for storing of image data with related settings and software programs for processing of several images and for calculations of transfer of images through change of scales of size, perspectives and positions, comprising; control settings for lens and sensor, relating respective position including angles to a geometry in the camera; and programs with algorithms, which use settings for respective image photo to transfer the images to a selected basic image, which also might be defined by settings, but that must not necessarily be a really photographed image; and a program unit, which possibly with the help of an operator, selects which image elements to be used from respective transformed image, and integrating those to a basic image. / /-TlT Means according to any of the claims 9 - 10, aiding the photographer / operator with controls of the different photo focal plane settings, comprising; means for defining positions of points in the image, means for focusing those points and register the focus in the calculation unit, means in the calculation unit for calculating subsequent corresponding values for lens and / or sensor settings, means for control of the settings for photographing, and selectively adding means for storing basic data for respective control in the calculation unit, being e g a pocket calculator or a PC, for use at calculations and calibrations of values for control of the control means.

/ ^ H^Means according to claims 9 - 11, comprising: means for the photographer to mark points on the computer screen image, and computer programs calculating and controlling the camera settings automatically by electronic controls.

' ⁱ Jβ A method and means according to any of the claims 1 - 12, where photography film are used instead of electronic sensors, comprising; photographing in the corresponding way, and subsequently scanning the film photos to obtain electronically stored images, which are subsequently processed in the corresponding way.

/( XT A method according to any of the claims 1 - 13, comprising; adding photo with reduced focal distance, selectively by zoom or change of lens, where image transformations are simplified by keeping the position of the lens optical centers relative the subject, giving a smaller image, not fully utilizing the sensor resolution, while depth of field has a potential to be increased, although by remagnification of the image also the defocused areas are magnified, the defocused elements obtaining improved focus and the image being used for adding information and to help matching other images.