WO2011120588A1

WO2011120588A1 - Image enhancement

Info

Publication number: WO2011120588A1
Application number: PCT/EP2010/054447
Authority: WO
Inventors: Stephen Philip Cheatle
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2010-04-01
Filing date: 2010-04-01
Publication date: 2011-10-06

Abstract

A method for enhancing an acquired digital image, comprising using a processor to generate saliency data representing regions of interest in the image, selecting a plurality of tile mask positions within an output image, and generating output image data at respective ones of the tile mask positions by: using the saliency data at corresponding ones of those positions within the acquired digital image to generate respective blending parameters for each position, and at a given tile mask position, using the corresponding blending parameters to modify pixels in the output image within an area defined by a tile mask by combining the pixels in the output image with those from one or more other images.

Description

IMAGE ENHANCEMENT

BACKGROUND

[0001] Photographs can lack impact because of problems with their background due to poor composition for example. The background can have too much empty space, or it can contain areas known as "distractions" which can attract a viewer's attention away from an intended "subject" of the photograph. Many image enhancement and stylisation effects are available with photo-editing tools such as the filters available in Adobe Photoshop for example. These effects apply to an entire image unless the user explicitly masks certain areas and blends between processed and original areas using layers. These techniques enable many sophisticated manipulations and enhancements to be performed, but they require highly skilled users to apply them.

[0002] Enhancements which are controlled by the saliency of the image are also known in the art. One example is automatic image cropping (such as in commonly assigned US Patent Application Serial No. 12/491067 for example) in which image saliency is used to determine which parts of an image to retain in a crop. Another example is where the degree of local image enhancement such as sharpening or blurring varies with the saliency estimation at each location in the image (such as in commonly assigned International Patent Application No. PCT/US2009/045367 for example).

BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Various features and advantages of the present disclosure will be apparent from the detailed description which follows, taken in conjunction with the accompanying drawings, which together illustrate, by way of example only, features of the present disclosure, and wherein:

[0004] Figure 1 a is a functional block diagram depicting an architecture of a computing apparatus;

[0005] Figure 1 b is a block diagram of an image enhancement process according to an embodiment;

[0006] Figure 2 is a block diagram of an overview of an image enhancement process of an embodiment;

l [0007] Figure 3 is a block diagram of a process for generating a soft saliency matte according to an embodiment; and

[0008] Figure 4 is a block diagram of a process for image enhancement according to an embodiment.

DETAILED DESCRIPTION

[0009] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0010] It will also be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first item could be termed a second item, and, similarly, a second item could be termed a first item, without departing from the scope of the present invention.

[001 1] The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the description of the invention and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. [0012] Figure 1 a is a functional block diagram depicting an architecture of a computing apparatus 101 suitable for use in the process of enhancing digital images according to certain embodiments of the invention. The apparatus comprises a data processor 102, which can include one or more single-core or multi-core processors of any of a number of computer processors, such as processors from Intel, AMD, and Cyrix for example. As referred to herein, a computer processor may be a general-purpose processor, such as a central processing unit (CPU) or any other multi-purpose processor or microprocessor. The processor 102 comprises one or more arithmetic logic units (not shown) operable to perform operations such as arithmetic and logical operations of the processor 102.

[0013] Commands and data from the processor 102 are communicated over a communication bus or through point-to-point links (not shown) with other components in the apparatus 101. More specifically, the processor 102 communicates with a main memory 103 where software can be resident during runtime. A secondary memory (not shown) can be used with apparatus 101. The secondary memory can be, for example, a computer-readable medium that may be used to store software programs, applications, or modules that implement embodiments of the invention, or parts thereof. The main memory 103 and secondary memory (and optionally a removable storage unit 1 14) each includes, for example, a hard disk drive 1 10 and/or a removable storage drive such as 104, which is a storage device connected to the apparatus 101 via a peripherals bus (such as a PCI bus for example) and representing a floppy diskette drive, a magnetic tape drive, a compact disk drive, etc., or a non-volatile memory where a copy of the software is stored. In one example, the secondary memory also includes ROM (read only memory), EPROM (erasable, programmable ROM), EEPROM (electrically erasable, programmable ROM), or any other electronic, optical, magnetic, or other storage or transmission device capable of providing a processor or processing unit with computer- readable instructions. Apparatus 101 can optionally comprise a display 1 12 connected via the peripherals bus (such as a PCI bus), as well as user interfaces comprising one or more input devices, such as a keyboard, a mouse, a stylus, and the like. A network interface 1 1 1 can be provided for communicating with other computer systems via a network. [0014] Embodiments of the present invention can be implemented by a dedicated hardware module, such as an ASIC, in one or more firmware or software modules, or in a combination of the same. A firmware embodiment would typically comprise instructions, stored in non-volatile storage, which are loaded into the CPU 102 one or more instructions at a time. A software embodiment would typically comprise one or more application programs that is/are loaded from secondary memory into main memory 103, when the programs are executed.

[0015] Figure 1 b is a block diagram of an overview of an image enhancement process according to an embodiment. An acquired digital image 121 has an iterative image enhancement process 125 applied to it in order to provide an output image 123. More specifically, a tile 131 for the output image 123 is selected. A tile position within the output image 123 deriving from the tile is provided 129 (for example, the position can be provided at the centre of the tile, or at some other position inside or outside of the tile area). At the desired tile position, saliency data 133 generated from image 121 and representing salient regions of the image 121 is used in order to provide one or more blend parameters 135. The blend parameter(s) define the way in which tile 131 is used at position 129 to blend image data from image 121 to image 123. Image enhancement method 127 takes the tile 131 and the or each blend parameter 135 as input. Process 125 repeats for a set of tiles at a set of tile positions. That is to say, the process is iterative in nature, such that successive positions for a tile are selected, and wherein - at each tile position - the area of an output image defined by the tile is updated by blending image data with the current pixels of the output image area defined by the tile. The blend is controlled by parameters whose value is determined using a value derived from saliency data at the position corresponding to that of the tile position selected in.

[0016] Figure 2 is a block diagram of an image enhancement process according to an embodiment. A data store 201 comprises a hard disk 1 10, removable drive 104, or any other suitable data storage device such as a flash memory card or the internal storage of a digital camera for example. The data store holds image data representing at least one digital image to be processed according to an embodiment. Such image data is provided at step 203 in the form of an acquired digital image - that is to say, data representing a digital image which has been captured by a digital stills or video camera for example. The digital image is processed using processor 102 at step 205 in order to generate a saliency map for the image. The generation of the saliency map will be described below with reference to figure 3.

[0017] The saliency map for image 203 is a topographically arranged map that represents visual saliency of the visual scene corresponding to image 203. According to an embodiment, a grayscale saliency map is used, in which different degrees of calculated saliency for an image are according respective different luminance values, ranging from 0 to 255 for example such that salient areas are relatively brighter, with non-salient areas being relatively darker, although this is not intended to be limiting and other alternatives are possible as will be appreciated by those skilled in the art.

[0018] Using raw saliency data representing the saliency map of 205, a soft saliency matte is generated to provide the soft saliency matte 209. The process of figure 3 takes the raw saliency data as input, and outputs soft saliency data representing a soft saliency matte 209. The soft saliency matte is a saliency-based matte in which luminance values for salient portions have a gradual transition from fully transparent to fully opaque. In such a matte, the edges of a salient shape from image 203 reduce gradually to transparency before the edge of the input image is reached. This prevents any hard edges being seen when the resulting matte is applied as will be explained below in more detail. According to an embodiment, the primary use of the matte is to determine at least one tile blending parameter for iterative blending of portions of the output canvas area (tiles). Optionally, the matte can be used as a conventional alpha map, to superimpose the salient areas of the entire source image into the output image at some stage during the process of constructing the output image. The matte is therefore used as part of the process to combine one or more image elements to provide an output image.

[0019] At step 213 an output image canvas is created/defined. The canvas is a construct used to define a region for an output image. The region can be the same size as the input digital image 203, or can be larger or smaller as required, with the same or a different aspect ratio. The process now enters an iterative loop in which successive positions for a tile are selected (215). At each tile position, the area of the output image defined by the tile is updated by blending image data with the current pixels of the output image area defined by the tile (217). The blend is controlled by parameters whose value is determined from the value of the soft saliency map at the position corresponding to that of the tile position selected in 215.

[0020] A tile position within the canvas region is selected at 215. According to an embodiment, a tile is a grayscale mask and can be a shape, image, character (numeric or alphanumeric for example), or any other type of mask which can be used when blending the input image onto the output canvas to form an output image 225 using the matte generated at 209. Selecting a tile position can be done in a number of ways. For example, by choosing random locations; using a pre-defined or calculated set of points such as a regularly spaced grid; or using random perturbations from a pre-defined set of points. Placement locations may also be determined or modified on the basis of features extracted from the acquired source image and/or the soft saliency matte. For example, tiles may only be placed in relatively empty areas of the source image.

[0021] According to an embodiment, a threshold number of tile positions within an output canvas can be provided. The threshold number can be predefined, or can be calculated by determining an optimal number of tiles required to enhance a given image (such that only a minimum number of tiles is used to develop the salient portions of a source image onto an output canvas for example). At step 219 the number of tile positions selected is compared to the threshold. If the number is greater than or equal to the threshold, then the process ends with the output image as generated (225).

[0022] At each selected tile position a blend operation is performed (step 217). The blend operation takes as input a tile 223, at least one blending parameter 221 , and a subset of image data representing a portion of the input digital image 203 at the tile position and within the mask area defined by the shape and properties of the tile in question. Blending parameters 221 utilize information from the saliency matte generated with reference to Fig. 3 as indicated by the dotted line joining the functional blocks 21 1 and 221.

[0023] Pixels from the original image (203), are blended with corresponding pixels in the output image. The output image pixels are replaced with the blended values. In each iteration of the loop, only those pixels defined by the selected tile location are blended. The tile shape, scale and rotation is dependent on the soft saliency matte value at the selected tile position, such that these tile parameters can vary for a given output image depending on the saliency matte value at a given position. In addition the blend transparency of the entire tile is dependent on the control value obtained from the soft saliency matte. It may be desirable to use a single tile shape with varying scale and/or rotation at different positions for example. Alternatively, a set of tile shapes can be used, where the selection of the shape at a given position can be random, or can be dependent on the saliency value. For example, respective tile shapes in a set can correspond to predetermined ranges of saliency values, with selection of a shape occurring if the value at a selected tile position falls within the range for that shape.

[0024] Following the blend operation, if the number of selected tile positions is below the threshold, the next tile position is selected (215) and blending continues, otherwise the output image is provided (225).

[0025] According to an embodiment, a tile is a grayscale mask defining a shape. Grayscale values indicate the transparency of the mask at each pixel. For example in the range 0 to 255 where 0 is transparent and 255 is fully opaque.

[0026] In an exemplary case, each iteration of the tile blending process consists of: a. Selecting a position in the output image on which to position the tile; b. Geometrically adjusting the tile (eg by scaling and/or rotation); c. Modifying each pixel in the area of the output image that is overlapped by the tile.

The modification is as follows:

where O_n represents the current value of a pixel to be modified in the output image, O_n+i represents the modified value of the output image pixel after this blend iteration, S represents a pixel value in the source image at the source image location corresponding to the location of pixel O in the output image, and a represents an alpha blend value in the range 0 to 1.

[0027] In this example the alpha blend value can be defined by a combination of the values from the tile mask and the saliency matte value at the point corresponding to the position of the tile in the output image: a = (T / 255) * (M / 255) where T represents the corresponding pixel value in the tile mask, and M represents the value of the soft saliency matte at the position corresponding to the position where the tile has been placed in the output image.

[0028] In this example tiles which are positioned in areas with weak values in the soft saliency matte will be blended with high transparency, whereas tiles positioned in areas with strong values in the soft saliency matte will be blended with high opacity. The blending is thus controlled by the saliency matte parameter, M.

[0029] This basic process can be extended in many ways. In (b) the geometric adjustment can be controlled by M. One possibility is to set the scale factor so that the tile is large for high values of M and small for low values of M. Another possibility, (shown in 41 1 below), is to rotate the tile between 0 and 90 degrees where 0 degrees applies to low values of M and 90 degrees is used for high values of M.

[0030] The output image can be initially filled with a suitable colour, colour gradient, or alternative image, on which the tiles are superimposed.

[0031] Reference will now be made to figure 3, in conjunction with figure 2 and the process outlined above. Figure 3 is a block diagram of a process for generating a soft saliency matte for an input acquired digital image according to an embodiment. A soft, smooth, non-rectangular matte is generated around the salient areas of the image. The shape of the matte is determined by the shape of the salient areas of the image, and deliberately includes part of the background surrounding the salient objects. It will be appreciated that many techniques for determining the salient regions of an image can be used. For example, salient regions of an image under consideration could be defined manually, or automatically generated regions of interest could be manually adjusted to suit a user's taste for example. According to an embodiment, a saliency map for an acquired image 203 can be generated using the method described in the Applicant's copending US Patent Application, Serial No.: 12/491067, the contents of which are incorporated herein in their entirety by reference.

[0032] Briefly, a saliency map is generated by first applying a face detection engine to the image data in order to detect any faces within the image. Each face that is detected is used to estimate and create a border of the head and shoulders of the person. The borders of the people identified and located can then be grouped into one people box segment, which provides an image portion comprising the head (including the face) and shoulders of the person(s) within the image. Such people are likely to be subjects of the image, and will therefore represent salient portions

[0033] In one example embodiment, numerous image colors are also identified within a digital image. A much smaller number of broad color value clusters are selected for use in the analysis, and each pixel of the image is assigned to a representative color cluster based on its own color value. Adjacent pixels of the same color cluster are then grouped into regional segments and classified as subject, background or distraction. The classifications are based on relative unusualness of the color, region size and location in the image. Classified regions form a saliency map and subject regions are grouped into subject boxes. The subject regions within each subject box which do not touch the edge of the image are referred to as the "core" of the subject box. Each subject box is scored by the size of its core. The classified segment regions allow a much more complete and detailed search of all areas of a digital image for aesthetic criteria significance. Following determination of the salient regions of an image, a crop rectangle can be generated which can be a minimum crop rectangle, such that predominantly salient portions of the image are contained within the crop rectangle, plus some background, with minimum distraction included for example.

[0034] For example, a minimum crop rectangle, (MCR), can be created from the core of the subject box with the highest score. This is expanded to include the people box and the central 15% of the image area. To cope with unusual images, if the MCR < 20% of the image area, or if the area of subject regions in the MCR < 10% of the image area, then the MCR can be expanded to include the central 25% of the image area.

[0035] Referring now to figure 3, at step 301 saliency map data representing the saliency map of the acquired digital image as generated above is provided. A grayscale image, represented by data G, is formed at step 303 from the saliency map by setting the luminance value of all pixels in the map to 0 except the following, which are set to a value of 255: a) pixels in the image belonging to regions which are both classified as "subject" and which are wholly contained in a minimum crop rectangle; b) pixels in the image which are in a rectangle formed by expanding the box defining a detected face which is contained in the minimum crop rectangle (note that the expansion approximates the head and shoulders of the person).

[0036] Other alternatives for generating grayscale image G are possible. For example, if a saliency map has been generated by a process that produces the saliency representation as a grayscale image, it may be possible to simply threshold this image. Another alternative would be for a user to manually draw a rough boundary around the salient areas.

[0037] At step 305, image data G is transformed into a new image, represented by data GS, such that the pixels from G are represented at a reduced scale, and positioned at the centre of a larger image area. This causes a more dramatic smoothing effect when the image is smoothed as described below. Furthermore, the additional margin allows effects at the edge of the original image to be handled smoothly. At 303 the image shape in GS is softened by repeating steps 305 through 31 1 a predetermined number of times. For example, the softening process of 303 can be performed 3 times on the image GS in order to provide an output softened image. More specifically, at step 305, the image is blurred. The image can be blurred using a Gaussian blur filter for example in which the blur is provided using a standard Gaussian function. Other alternatives are possible as will be appreciated by those skilled in the art. At step 307, the blurred image is thresholded so that pixels with a luminance value below a lower predetermined threshold value, (such as 10 for example), have their luminance value set to 0 whilst others with a luminance value above the lower threshold are set to 255. This has the effect of expanding the original shape.

[0038] At step 309, the thresholded image is again blurred, with the resultant image thresholded again so that pixels with a luminance value below a higher threshold, (such as 230 for example), are set to 0 and others are set to 255. This contracts the original shape. It will be appreciated by those skilled in the art that alternative lower threshold and higher threshold values are possible depending on the nature of the soft matte required. The repeated expansion and contraction of the image in 303 results in a shape which expands smoothly around the original pixels in GS. At 313, the thresholded shape resulting from the final iteration of step 303 is smoothed by performing a final blur to generate a smooth shape image, S.

[0039] It is possible that the smooth shape image, S has non zero pixels which extend beyond the area occupied by the original image. If this image was simply cropped and re-sized to generate the output matte image, this would result in a harsh edge where non-zero pixels had been cropped. To overcome this problem an avoidance matte 316, A, is created by forming an image which is the same size as S, containing 0 (black) in the area occupied by the original image in S and 255 (white) in a surrounding border of predetermined width (such as border of width in the range 10-100 pixels for example). This image is then blurred, thresholded and re-blurred to create an expanded white border region which fades into the central black area of A. The matte A is subtracted from S at 314 to ensure that S fades out gradually to the edge of the original image area (note that negative values are clipped to 0). The portion of the resultant image, S' which corresponds to the original image is scaled up at 315 to the original image size to create the full resolution matte in the form of output image data 317.

[0040] Other alternatives for creating a soft saliency map are possible. For example, saliency detection methods which create a grayscale saliency map as output may be able to use this directly, or after some refinement such as bluring and noise reduction using standard image processing techniques.

[0041] Reference will now be made to figure 4, which is a block diagram of a process for image enhancement according to an embodiment. An acquired digital image is shown at 402, with corresponding image data representing the image 402 at 401. As described above, the digital image data 401 is processed in order to provide data 403 representing a saliency map 404 for the image 402. As explained, the saliency map can be generated using the method as described. Alternatively for example, the method proposed by Itti, Koch and Niebur in "A Model of Saliency-based Visual Attention for Rapid Scene Analysis", IEEE Transactions on PAMI, 20(11 ): 1254-1259, 1998 can be used. Alternatives methods may transform the original image directly into a suitable soft saliency matte, such that the intermediary step is not required. There is therefore no requirement that the process of generating a soft saliency matte should first create a binary saliency map (205, 404) and then soften it (209, 406). [0042] The data 403 representing the saliency map 404 is processed as explained above with reference to figure 3 in order to provide data 405 representing a soft-saliency matte 406 for image 402. As can be seen from figure 4, the matte 406 comprises an expanded area which comprises the determined salient regions of image 402 as well as some background. It will also be appreciated that the process as described with reference to figure 3 has provided a matte in which the extremities of the determined salient region gradually 'fade' - that is to say, the luminance value in 406 associated with the salient portions reduces towards the extremities of the area covering the salient portions plus background. This can provide a visually appealing result in an output image 41 1 such that regions of interest do not suddenly appear, but rather, can be 'faded in'. Alternatively, of course, it may be desirable to have a 'harder' (more sudden) introduction of salient regions.

[0043] According to an embodiment, a blend module 407 uses at least one tile (408) of predetermined size and shape along with a set of one or more blend parameters in order to generate an output image 41 1. The choice of tile could be based on the soft saliency matte value at the tile position, or it could be a random choice. The tile represents a matte shape, for example a rectangle or star. The tile is positioned at multiple positions on the source image. At each position the tile shape may be geometrically transformed, for example by scaling and/or rotation. The pixels in the source image which are identified by the tile matte, at its current size and location, are blended into the output image. The final output image is built up iteratively from a selection of multiple tile positions, the blend from each tile position modifying the corresponding portion of the output image which it overlays. Each corresponding portion of the output image may be termed an update region as it is an area in which image data is updated.

[0044] The blend module 407 also takes soft-saliency matte data 405 at the desired tile (408) position 409 as input. For each iteration, the soft-saliency matte value at the current tile position can be used to control at least one of the parameters controlling the selection of tile shape; the parameters of the geometric tile transform; and the relative opacity of the blend of the source image, any auxiliary image, and the current pixel in the output image. [0045] Blend parameters can, for example, comprise one or more of the following, which is a non-exhaustive list of parameters which can be set depending on the saliency matte value at the tile position:

1. Choice of a tile mask image (a grey scale image in which each pixel defines an opacity value which determines how the tile is rendered);

2. Tile size, relative to the source image;

3. Tile rotation;

4. Tile opacity - a global value which scales pixel opacity values specified in the tile mask image;

5. Tile displacement in the output image relative to the source image tile location;

6. Tile distortion;

7. Tile shape border properties such as whether a border should be shown, how opaque it is, how large it is etc.

[0046] The saliency value used to control these parameters may be extracted from the saliency matte using various possible sampling techniques. According to an embodiment, the extraction can be done by using the value at the centre point of the tile, or the average value of all the saliency pixels occupied by the tile at the selected position.

[0047] In the example shown in figure 4, output image 41 1 uses a rectangular tile where the size and a rotational value of the tile vary depending on the saliency value of the matte, and hence the underlying image 402. As a result, the more salient parts of image 402 as provided in the saliency map 404 and the saliency-based matte 406 are shown, whereas parts of image 402 which are classified as non-salient regions are not shown to the same extent (although the regular grid of rectangles in these regions still allows some such portions of image 402 to be visible, the blend parameters at these tile positions mean that the size of the tiles, their rotation, and their opacity is strictly controlled so that much less of the image is visible at these positions than is the case at the more salient areas).

[0048] With reference to figure 4, it can be seen that the rectangles in image 41 1 rotate gradually depending on the saliency value at the tile position in question, as well as becoming larger in size. In the example shown, the rectangles rotate through 90 degrees from areas of minimum saliency (zero luminance value in 406) to areas of maximum saliency (luminance value of 255 in 406). The relative size the rectangles can be controlled in a similar way depending on saliency value at the selected tile position. It will be appreciated that numerous variations are possible, and that the above is only intended to illustrate the general theme for a specific set of parameters and for one tile shape.

[0049] According to an embodiment, an additional step can be added in which the matted version of the source image is blended in its entirety. This can ensure that all the source image pixels in the salient area are blended into the output.

[0050] According to an embodiment, some tiles can be centered on faces in the source image. Further, these face tile locations could be rendered in a non-standard way, for example, being rendered as the last tile to cover the face, with high opacity, to ensure that the face features were not distorted by visible tile boundaries for example.

[0051] According to an embodiment the pixels which are blended into a tile location on the output image may come from an auxiliary image instead of the source image. For example, an auxiliary image may be used which corresponds to the shape of the tile mask, such as a logo or other simple graphic. In this embodiment the saliency matte value could be used to make tiles positioned in highly salient areas fully transparent, and tiles positioned in non-salient areas fully opaque. This embodiment would typically be combined with the embodiment that uses the soft saliency matte to blend the original image into the output image.

[0052] A further alternative embodiment is to use pixels to blend into the output image which are themselves a blend between the auxiliary and source images. In this case the blend factor controlling the relative proportions of source and auxiliary image strength would be determined by the soft saliency matte value. For example, this could be used to ensure that pixels blended into high saliency areas of the output image are predominantly those from the original image, whereas those used in low saliency areas are predominantly those of the auxiliary image.

[0053] According to an embodiment, alternative tile mask shapes or auxiliary images may be selected from a small set at random such that an output image can have a number of tile shapes/sizes which have been used. Alternatively, a tile shape/size, or set of tile shapes/sizes can be chosen manually, or chosen automatically taking into account the subject-matter of the source image, or the value of the soft saliency matte. For example, if no faces are detected in a source image it may be more appropriate to use certain tiles and/or backgrounds compared to the situation in which one or more faces are detected.

Claims

CLAIMS What is claimed is:

1. A method for enhancing an acquired digital image, comprising: using a processor to generate saliency data representing regions of interest in the image; selecting a plurality of tile mask positions within an output image, and generating output image data at respective ones of the tile mask positions by: using the saliency data at corresponding ones of those positions within the acquired digital image to generate respective blending parameters for each position; and at a given tile mask position, using the corresponding blending parameters to modify pixels in the output image within an area defined by a tile mask by combining the pixels in the output image with those from one or more other images.

2. A method as claimed in claim 1 , wherein the or each other images comprises at least the acquired digital image.

3. A method as claimed in claim 1 , wherein combining pixels in the output image further comprises: selecting pixels in the corresponding location of the or each other image; and modifying pixels in the output image by combining corresponding pixels from the or each other image with those from the output image

4. A method as claimed in claim 1 , wherein the saliency data is in the form of a soft matte which provides a measure of the relative saliency of regions of the acquired digital image such that regions of interest at a given pixel location have a different luminance value from those in low interest regions.

5. A method as claimed in claim 4, wherein the matte has feathered edges such that a measure of relative saliency transitions gradually in the area outside of a region of interest.

6. A method as claimed in claim 1 , wherein the value of respective blending parameters varies according a measure of the relative saliency of the input digital image at the corresponding selected tile mask position.

7. A method as claimed in claim 1 , wherein the value of a blending parameter defines the way in which a pixel is modified in the output image.

8. A method as claimed in claim 1 , wherein at least one of the blending parameters defines an opacity parameter used to control the proportions by which the color and/or luminance values of the current output image pixel are blended with those of a pixel obtained from the one or more other images to produce the modified output image pixel.

9. A method as claimed in claim 1 , where modifying pixels in the output image includes modifying the geometry of a tile mask by adjusting a scale, distortion and/or relative rotation in order to provide an alternative mask area defined by the tile.

10. A method as claimed in claim 9, wherein parameters for geometric adjustments are determined from the saliency value at the tile position.

1 1 . A method as claimed in claim 1 , wherein one or more opaque tile masks are centered over regions in the output image corresponding to the position of detected faces in the acquired digital image in order that said faces appear undistorted in the output image.

12. A method as claimed in claim 1 1 , in which the opaque tile masks are processed at a final iteration in order that their effects are not overlaid by the subsequent selection of positions.

13. A method as claimed in claim 1 , wherein the acquired digital image is blended into the output image using the saliency data to ensure that salient parts of the acquired digital image are displayed in their entirety in the output image.

14. A method for processing a first digital image in order to provide an enhanced output digital image, the method comprising: determining at least one salient portion of the first digital image representing a region of interest within that image; blending a plurality of sections of a digital image into respective update regions of the output digital image using a set of blending parameters for the sections; wherein the set of blending parameters for the sections is determined using a soft saliency matte generated using the values for the saliency of the first digital image at the sections.

15. A method as claimed in claim 14, wherein blending a plurality of sections of a digital image comprises using the first digital image as the digital image.

16. A method as claimed in claim 14, further comprising: selecting a further image; and modifying pixels in the output image by combining pixels from the further image with those in the output image.

17. A method as claimed in claim 14, wherein the update regions are defined by tile areas, the geometry of which are dependent on the relative value for the saliency of the first digital image at the position in the first digital image corresponding to the position of a tile area for the section in question.

18. A method as claimed in claim 14, wherein update regions which are positioned in areas with corresponding lower saliency values are blended with higher transparency, whereas input regions positioned in areas with higher values are blended with higher opacity.