GB2405763A - Selection of colours when editing colours of graphic images - Google Patents

Abstract

The Colour Control Bar is a user interface element which provides an intuitive and flexible way to display and select groups of colours in an image, and, in the context of a mask generation tool, to assign those colours to different sections of the mask. In conjunction with a classification and segmentation method, it allows a more accurate mask to be generated than is possible by purely automatic means, by letting the user focus at a finer level of detail than elsewhere on problematic areas or groups of colours and manipulate them independently of the automatic segmentation. A user is presented with a colour palette restricted to colours present in the unedited image to allow selection of a colour an identified pixel is to take.

Description

i.. :e 2405763

SELECTION OF COLOURS WHEN EDITING GRAPHIC IMAGES

This invention relates to means enabling the rapid and accurate selection of colours when editing graphic images.

It is commonplace to edit digital artwork by selecting one or more given pixels and changing the colour assigned to each pixel. This is timeconsuming and laborious and, particularly where substantial files of high definition artwork are required to be processed, is labour-intensive and accordingly rarely justified. The only way in which such images can be in practice handled is by breaking them up (so-called segmentation) into groups of pixels of different properties.

A variety of so-called segmentation methods are known for handling the data set which corresponds to a digital image.

Particular attention is drawn to the segmentation method disclosed in published application WO 03/052696. That e e e ee.

e e e e e e e e e e e e e e e e e e e e e e e e e e e e e. e e publication discloses an automatic method of segmenting images using certain mathematical techniques and this enables considerably improved image processing to occur.

Many image processing operations act not on all the pixels in an image, but on a subset of those pixels.

Customarily, programmes have been provided which allow the selection of a set of pixels from an image based on their colour. For this purpose, a drop down or pop-up menu of colour possibilities is often presented on screen to the user and this may be enlarged to look at a particular subset of colours before selecting which colours to use as the basis of selecting pixels in the image on which to perform further operations.

With the very substantial numbers of colour combinations available using modern software, this is an inconvenient approach. Despite the flexibility provided, it very often happens that the colours selected do not accurately define the desired group of pixels, causing the selection to be of a lower quality than that desired by the user.

In contrast to this approach, according to the present invention, the range of colours by means of which a particular group of pixels may be selected is limited to those colours which have already been determined to be present in the relevant portion of the image being processed. Thus, a considerably narrower palette of material is provided from which to choose, and the user is enabled much more securely to choose colours which match the desired group of pixels.

The technique is of particular value where the selection of particular groups of pixels is required when those groups are identified e.g. by the segmentation method a a a sea ea a a e a a a a a a a a e a a a as a a described in the specification referred to above as being all of a similar colour, but where a finer level of selection is required than is given by the segmentation. By working with a restricted selection of colours, the segmentation can be refined by selecting only certain colours within each segment, and the limitations of the segmentation are thus overcome.

One particular area of value in applying the present invention relates to the steps needed in order to achieve a satisfactory cut-out of the foreground of one image from its original background and then to overlay seamlessly that foreground on to a new background. In carrying out such a procedure a choice must usually be made between labour-intensive manual techniques, where the mask is generated by hand using painting tools, or more automated techniques, where most of the work is done semiautomatically on the basis, for example, of colour (e.g. "Blue-screen" methods). The most advanced automatic methods include the classification of the colours in an image into sets (colour classes) which are contiguous in colour space at a certain resolution, and the segmentation of the image into contiguous regions of pixels whose colours are members of the same set. One such method is described in WO 03/052696.

A problem with these methods arises in the difficulty of distinguishing between pixels which are of a given colour because the area of image they represent contains only one material, and those which are a very similar colour, but in which that colour has been generated by the mixing of the colours of two materials which are overlapping or defocussed in that pixel.

Examples of situations in which such difficulties can occur include:: c . a t - 4 (1) the colours on the two sides of a border between the selected part of an object and its context are too similar, as often happens in deeply shaded regions.

(2) an edge is soft or blurred, as with coat of a furry animal or the edges of most shadows.

(3) the edges of the object are complex, creating promontories and small separated regions, for example leaves against the sky.

(4) the part to be cut out is very thin, as in the case of individual hairs or grasses.

A good example is that of hairs on a background of blue sky (Figure 1). The variation of blue colour in the sky may well be such that darker blue pixels of pure sky are a very similar colour to pixels whose colour mixing proportion is e.g. 90% sky, 10% hair. The desired result is that the mixed hair/sky pixel is given the appropriate 'mixed' status, so that it can be processed for example in the way suggested in WO 03/052696, and

that the pure sky pixel is given the 'background'

status. However, if these pixels have such similar colours that they are members of the same colour class, and are nearby in the image, they may well be in the same region of the image, and therefore indistinguishable to colour-class and region based selection tools.

A solution to this problem, in accordance with the present invention, is to allow the user to intervene, and refine the status of the pixels in a given selection by examining their colour in fine detail, thus distinguishing between colours which have been . . . . . . . . . - . . . . - 5 automatically grouped into the same colour class. The invention may be implemented by displaying to the user a "Colour Control Bar" which is adapted to display only those colours present in the selected part of the image.

This enables the pixel selection process to be rendered simple, flexible and intuitive.

The colour control bar may make use of display techniques which are familiar in many areas of computing practice, viz the presentation on screen of a tree structure to display data. This is commonplace in much computer application software. Essentially, it provides for the display of a hierarchy of objects, such as folders (which may contain other folders and files) and files on a storage device, and allows the display of the contents of each element to be switched on and off (expanded and contracted). In this way large quantities of data can be rapidly explored, analyzed and modified.

Applying this tree approach in the present case, each colour in a digital image is represented by a set of band values, which in association with the meaning of the bands signify the visible colour being represented.

Common sets of bands to use are: Red, Green and Blue (corresponding to the usual emissive or additive primary colours used in computer and other displays); Cyan, Yellow, Magenta and Key (Black) (corresponding to the usual reflective or subtractive primary colours used in printing); Y (luminance), Cr (red chrominance) and Cb (blue chrominance) (often used for the transmission or compression of colour video data).

The Colour Control Bar used when carrying out the method of the present invention presents the user with a tree- structured display of data comprising the colour values . . .e a*- ë . . . . - 6 - of all the pixels present in the current selection. Each level represents one of the dimensions (or bands) of the colour space in which the image is being analyzed, e.g. H(ue), S(aturation), L(ightness). Elements at each level display in some useful way the nature of all the elements below them, e.g. a "Hue" element might be coloured with the fully saturated colour of that hue, or a "Lightness" element be a grey colour somewhere between black (lightness = 0) to white (lightness = 1).

In one way of putting the invention into effect, the Colour Control Bar consists of the following elements, which are presented to the user in a window on the screen (this is illustrated by way of Example in Figure 2 of the accompanying drawings).

A number of columns of colour buttons (C in Figure 2).

This number is between 1 and the total number of bands in each colour, depending on whether or not colours have been expanded to show lower bands. Each colour button represents a colour or set of colours that are a subset of the set being examined. These are arranged so that in the first column there is one button for each value of the first band present in the set. Associated with each colour button is a group of values in the next band; a switch beside each button toggles display of these associated values on and off. Clicking on a colour button switches the status of the each pixel which it represents (i.e. which has a colour equal to one of the colours below that button in the tree, or for bottom level buttons a colour equal to the colour of the button) between that currently assigned and Interesting'. Any pixel given 'Interesting' status is highlighted in the image.

Buttons to select the order in which the bands are . . c c ce. c. . .. ..

displayed and sorted (B in Figure 2). The columns of colour buttons are displayed in the order dictated by these order_selecting buttons, and the colours will thereby be grouped in a tree structure, sorting first by the leftmost displayed band and so on.

Reassignment buttons (A in Figure 2). When pressed, these assign the colours selected as 'interesting' using the colour buttons to a given mask status in the image,

such as Object, Background or Edge.

The operation of the Colour Control Bar is as follows: 1: Select a region of the image for examination.

2: Open the Colour Control Bar if it is not already displayed.

3: If desired, adjust the order in which the bands are displayed and sorted.

4: By clicking on the appropriate colour buttons, expanding the display to reach specific colour values if necessary, and considering the pixels highlighted within the image, select a subset of the available colours which correspond to a set of pixels in the image whose status you wish to reassign.

5: Use the reassignment buttons to reassign those colours to a status other than that which was originally selected, where they occur in the current selection.

6: Optionally, repeat 3-5 to reassign further sets of colours, until all the colours have the desired status.

7: Close the Colour Control Bar if it is not required A: .. eee en: - 8 - immediately for subsequent operations.

Figure 3 shows the result of using a colour classification tool to select the sky as the background of the image in Figure 1. Some of the mixed hair/sky pixels have been wrongly assigned background status, and the colour classification tool is being used to select

the sky as background, which leads to false

classification of some edge pixels.

To find the pixels within this selection which should be classified as edge pixels, the Colour Control Bar is opened and the most appropriate ordering for the colour bands is selected, in this case, Saturation, Lightness, Hue. Then each colour button can be clicked to highlight the pixels it represents. If finer detail is required, colour sets lower down in the band structure can be accessed by expanding the display (see Figure 2). When all pixels of interest have been selected, their status can be reassigned to 'Edge' (Figure 4)which shows the Updated Colour Control Bar and image after reassignment of mixed pixels. - 9 -

Claims (6)

1. A method of image processing of a digital image consisting of a large number of pixels, wherein the colour of one or more selected pixels in the image can be changed from a first colour to a second colour selected from a palette of colours, wherein the palette of colours presented to a user for selection of a second colour is restricted to a palette of colours already present in pixels of the unedited image.

2. A method according to Claim 1 wherein the pixels of the original image are automatically analysed and the image segmented into a plurality of continuous regions of pixels whose colours are members of the same set.

3. A method according to Claim 1 or 2 wherein, in order to display the restricted palette of colours, an area of the image is selected and the colours associated with individual pixels in that area of the image are displayed as a series of differently coloured areas on a colour control bar from which the desired second colour may be selected and applied to selected pixels within the selected area.

4. A method according to any of Claims 1 to 3 wherein the individual colours are analysed into the three factors, hue, saturation and lightness.

5. A method according to Claim 4 wherein the order of hue, saturation and lightness in a tree structured colour control bar can be selected by the user.

6. A method of selecting colours when editing graphic images substantially as hereinbefore described.