METHOD AND ARRANGEMENT FOR COLOUR CORRECTION OF DIGITAL IMAGES
TECHNICAL FIELD
The invention relates to a method and an arrangement for colour correcting digital images, wherein a reference card placed in one such image is used for creating a colour profile that is applied to subsequent images for correction of grey level and colour.
BACKGROUND ART
Photographs taken using a digital camera will often result in images that require some kind of colour correction. The colours in a digital image can be inaccurate for a number of reasons, such as the colour temperature of the ambient lighting, the white-balance settings in the camera, the use of a flash and the quality of the CCD-element in the camera.
Colours may be corrected using an International Color Consortium (ICC) colour profile. Manufacturers of digital cameras will provide digital cameras with an internal ICC profile to correct known colour deviations caused by the camera itself. It is also possible to make personalised ICC profiles using an imaging program, and apply the profile to your images. Such profiles gives improved colour accuracy, shadow and highlight detail, and contrast.
However, to achieve a profile that gives the image an exact colour accuracy, it is necessary include some kind of reference in at least one image. This can be done by using a standard reference card containing a number of exactly defined colours. When the digital image is opened in an imaging program, the reference colours on the card can be compared to the reference colour stored in the imaging program. By comparing grey levels and a number of reference colours it is possible to create a colour profile for correcting colour accuracy. This profile can then be used to correct each individual image in a batch of images taken at the same time as the reference image.
Reference cards containing a colour chart are widely used by professional photographers for calibrating camera equipment and for correcting images with respect to a desired ambient colour temperature or for ensuring that the photographed subject is portrayed with accurate colours. Examples of known standard reference cards are for example GretagMacbeth Color Checker ™ and Fotowand ™ FT8 Scanner Reference.
A problem with known reference cards are that they are often too expensive or too large to be practical for non-professional users. In addition, the software required to process and colour correct digital images are often very complex and/or to expensive for non-professional users. Examples of known widely available software are Photoshop ™, GretagMacbeth ColorPicker™, GretagMacbeth ProfileMaker™ and Sinar CaptureShop™. Making a colour profile using the above cards and imaging programs is often both slow and painstaking, and requires the user to learn how the imaging program works before a satisfactory result can be achieved.
A known method for colour correction is disclosed in US 5 852 675. In this document, a colour chart is placed on a subject, photographed and digitally processed by picking individual colours from the image, comparing them to the corresponding , known reference colours and computing a correcting value that is applied to the original image. A problem with this method is that the reference colours must be fed into the computer either manually, using a pointing device, or by means of a separate scanner.
Hence, there exits a need for an inexpensive, simple and easy to use method and arrangement for colour correcting digital images and creating colour profiles.
DISCLOSURE OF INVENTION
The invention relates to a method and an arrangement for colour correcting digital images, wherein a reference card placed in one such image is used for creating a colour profile that is applied to subsequent images for correction of grey level and colour. This is achieved by means of a method according to claim 1 and an arrangement according to claim X, and their respective dependent claims.
According to a preferred embodiment of the invention, the method involves a computer controlled method for calibrating and correcting digital images acquired by a digital camera, wherein at least one of a batch of images contains a reference card containing at least three primary colours and at least two grey levels. The card will be described in more detail below.
The method comprising the steps of;
- displaying an image containing the reference card on a display, - indicating the position of the reference card on the display, by marking at least one reference point on the card on the display,
- locating the reference card in said image by means of said at least one reference point and one further reference marker on the card
- generating a first set of calibration data based on a comparison between an actual grey level on said card and a known reference grey level
- generating a second set of calibration data based on a comparison between a number of colours placed in a predetermined pattern on said card and a set of known reference colours
- creating a final set of calibration data based on the first and second sets of calibration data and storing said final set of data in a data storage means as a colour profile,
- selecting at least one colour profile stored in said data storage means and applying it to at least one image in said batch of images.
The position of the reference card is indicated using at least one manually indicated position and an automatically detected, distinct reference marker on said reference card. The automatically detected reference marker can be an area of a predetermined colour located at a predetermined position in a colour chart, a printed distinct shape or logotype, or a marker combining shape and colour. The manually indicated positions are used to determine the position of the reference card in the image, while the automatically detected reference marker is used to identify its orientation.
If the position of the card is indicated using one or two manually indicated positions, then these will have to be predetermined positions on the card. When determining the location of the card the computer has no information about the distance to the card. In order to give the computer information about the relative size of the card in the image, one or two predetermined positions and an automatically detected reference marker recognised by the software running on the computer is the minimum information required. Also, in this case the card must be placed substantially parallel to the focal plane of the camera
According to a further embodiment, a method that is more error tolerant, and therefore more user friendly, uses three or four manually indicated positions on the reference card. By using more than two positions it is possible to determine not only the location of the card in the image, but also its relative size and angle in relative to the focal plane of the camera. If a number of predetermined, manually indicated positions are used then this can be sufficient to determine both location and orientation of the card.
Alternatively, any three or four positions may be indicated to determine the location of the card, while an automatically detected reference marker is used to determine the orientation, as described above.
The design of the reference card, such as shape, number of and relative placing of colours in a pattern or a grid of a colour chart, and relative
positioning of the markers for manual indication and automatic detection can be varied freely within the scope of the invention.
Once the location and orientation of the reference card in the image has been determined, then the software is able to capture data representing each colour in the grid of the colour chart. The data from each colour in said grid in the image can be compared to corresponding reference colours. A first set of calibration data is generated using multiple areas on the reference card having the same grey level to verify that the card itself is correctly exposed. To achieve this a number of standard grey areas should be distributed around the periphery of the colour chart, preferably in or near the edges or corners thereof.
Subsequently data captured from the entire colour chart is used to generate a second set of calibration data, allowing a colour correction to be performed based on the exposure properties of the digital camera. The software may also allow the user to modify the second set of calibration data dependent on, for instance, the type of subject in the image and to allow weighting of colours in the colour chart depending on image subject. For portraits it is possible to weight the colours of a group of skin tones in the colour chart, while a landscape can be weighted using a number of primary colours such as blue and green.
The first and second sets of calibration data are processed in order to create a final set of calibration data and storing said final set of data in a data storage means as an ICC colour profile. By using the format of the widely used ICC standard, the original image and its associated colour profile can be processed by virtually any make of imaging software. More than one such profile can be created and stored for each reference image, so that profiles containing both standard, weighted, or otherwise modified colour profiles can be stored in a library of profiles for later application the one or more images.
If the processed image is one of a batch of images taken at the same time, the software can subsequently be instructed to apply the resulting colour
profile to all images in said batch. However, no changes are made to the data file original image, as the colour profile is stored as a separate data file in the same or a parallel folder in the memory of the computer.
After creating and storing said colour profile, a user can select the created colour profile and/or any other colour profile from the storage means. The profile applied to the images can be selected from a range of profiles having different calibration properties. For instance, after colour correcting one or more images using the created profile, a further, predefined profile can be applied to the corrected image. Ready-made profiles of this kind can be used e.g. to emulate a particular colour temperature, such as daylight (approximately 5,500 K), the colour characteristics of a predetermined type of emulsion based film, such as Kodachrome 64™ or Fuji Velvia™, or the exposure characteristics of a predetermined type or make of camera.
According to the preferred embodiments described above, the reference card used may be provided with a predetermined number of reference positions for manual indication, as well as a reference marker for automatic detection.
However, the software may also include the option that the type of reference card is selected from a range of standard reference cards. The user must then be given instructions relating to which positions and/or markers to use in order to allow the software to find the location and orientation of the selected reference card. The software will be provided with known reference colours for the appropriate colour chart.
The invention further relates to an arrangement for calibrating and correcting digital images, wherein a batch of images includes at least one reference image containing a reference card containing at least three primary colours and at least two grey levels. This arrangement comprises;
- a data storage means for storing digital image data,
- a display means for displaying an image containing a reference card,
- an indicating means, such as a mouse, for manually indicating at least one reference point on said reference card on the display
- a microprocessor means provided with an image recognition means for automatically detecting at least one further reference marker on said reference card, to determine the position of the reference card in the reference image, - the microprocessor means further provided with a grey level comparing means arranged to perform a comparison of grey levels between multiple, identical areas on said grey card and a known reference grey level, to determine that the card is evenly exposed and to generate a first set of calibration data, - the microprocessor means further provided with a colour comparing means arranged to perform a comparison of a predetermined pattern of colours on said card and a set of known reference colours, to generate a second set of calibration data,
- the microprocessor means further provided with a colour profile generating means arranged to perform a calculation based on said first and second sets of calibration data to create a colour profile for the reference image and store said profile in the data storage means,
- the microprocessor means further provided with a colour correcting means arranged to retrieve at least one colour profile from said data storage means and perform a colour correction to one or more of said images.
The microprocessor may further be provided with an image enhancing means arranged to zoom in on the reference card, to facilitate marking of reference points, detection of the further reference marker, and comparison of grey levels and colours.
The software used for colour correcting and creating colour profiles is a so- called plug-in program that is intended for installation on any type of PC or Macintosh ™ computer, independent of type of platform , or version of operative system. It is also designed to handle any standard image format, such as TIFF, JPEG, GIF, etc.
BRIEF DESCRIPTION OF DRAWINGS
In the following text, the invention will be described in detail with reference to the attached figures. These figures are used for illustration only and do not in any way limit the scope of the invention. In the drawings: Figure 1 shows a schematic diagram of the arrangement according to one embodiment of the invention
Figure 2A shows one embodiment of a reference card according to the invention
Figure 2B shows one embodiment of a reference card according to the invention
The above schematic figures are not to scale and do not in any way limit the scope of the invention.
MODES FOR CARRYING OUT THE INVENTION
Figure 1 shows a schematic diagram of one embodiment of the invention, wherein a reference card 1 is been placed in the image to be photographed by a digital camera 3. The card 1 is preferably placed so that it is evenly lit by a light source 2 or the ambient light. In addition, the card should preferably, but not necessarily, be placed flat and substantially parallel to a focal plane F of the camera. The digital camera 3 captures the image including the reference card 1 , which can then be removed before taking one or more subsequent images. Alternatively the card can be included in all images taken at one time, if this is desired for later reference. If the card is removed, it may be put back at a later time, e.g. if the light condition suddenly change.
The images stored in the internal memory of the camera 3 is then transferred to a data storage unit 4, such as a hard drive in a computer. The computer is provided with a display 5 for displaying digital images, a command input device 6 in the form of a keyboard and an indicating means 7 in the form of a mouse. The internal communication between these devices, processing of digital image data and calculations using various algorithms is performed by
a microprocessor 8. In order to process one or more images the display 5 shows a reference image including the reference card 1 , whereby a number of reference points on the card 1 are indicated by the mouse 7. This is simply done by positioning a cursor C on the display 5 and clicking on a number of reference points. These reference point determine the location, including the relative outline, of the reference card 1 in the reference image shown on the display 5. In a first preferred embodiment, four reference points are used. According to a further embodiment, three reference points are used, wherein these three points are used by the software to define a rectangle in order to determine the location of the fourth reference point. Once the reference points have been indicated, the rectangle may need to be moved, twisted or rotated to adjust it to the shape or position of the card. This is achieved in the same way as in most image processing programs, by means of the mouse 7. The reference card 1 will be described in more detail below.
The microprocessor 8 is provided with an image recognition means 9 for automatically detecting at least one further reference marker on said reference card 1 , to determine the orientation of the reference card in the reference image shown on the display 5. Once the location and orientation have been established, the exact positions of all grey levels and colours in the reference card 1 are known to the computer. The reference card in the image can then be compared with a reference card stored in the computer.
Alternatively other types or makes of reference cards may be used. For the user this would involve selecting a particular card from a predetermined range of cards. In case of the software used, this will require it to recognise various pre-programmed features of each card, as other makes of cards have different colour charts and will most probably lack the reference positions and marker of the card according to the invention.
The microprocessor 8 can also be provided with an image enhancing means
10 arranged to zoom in on the reference card, to facilitate marking of reference points, detecting reference markers and comparison of grey levels
and colours. This may be necessary if the card has been placed at a relatively large distance from the camera. In this case the size of the reference card 1 in relation to the total image may make it difficult to indicate the reference points on the card. By enlarging the general area containing the card in one or more fixed steps, or by continuous zooming, both indication and image processing is facilitated.
After establishing the location and orientation of the reference card 1, the microprocessor 8 uses a grey level comparing means 11 to perform an exposure check for the card 1. This is done by comparing a number of identical grey level areas on the card with a known reference grey level stored in the computer. This is performed in order to determine that all areas of the card is evenly exposed. If a discrepancy is detected, this indicates that the card may have been placed partially in the shade or in a non-planar position where it may reflect light from a light source. If this occurs, then the user is warned that the resulting colour correction may give undesired effects.
According to a preferred embodiment, the reference card is provided with at least four standard grey level areas placed in each corner of the colour chart. In this context, the term "standard grey level" denotes a shade of grey reflecting 18% of incident light, which grey level is commonly used to neutralise colours to daylight conditions (5500 K).
If the reference card is correctly exposed the grey level comparing means 11 carries out a comparison of a number of additional grey levels, as well as a black and a white area on the card. In the case of the latter it is desirable to have some, albeit small, contrast in both areas. Once this process is completed, the grey level comparing means 11 determines what corrections are required to neutralise the grey levels on the reference card 1 to daylight conditions. Based on these data the grey level comparing means 11 generates a first set of calibration data.
Subsequently a colour comparing means 12 in the microprocessor 8 is arranged to perform a comparison of a predetermined pattern of colours on said reference card 1 and a set of known reference colours stored in the computer. Based on this comparison colour comparing means 12 determines what corrections are required to adjust the colours on the reference card 1 to the known reference colours stored in the computer. Based on these data the colour comparing means 12 generates a second set of calibration data.
The microprocessor 8 is further provided with a colour profile generating means 13 arranged to perform a calculation based on said first and second sets of calibration data. Based on these data the colour profile generating means 13 creates a colour profile for the reference image, which profile is stored in said data storage means 4. The profile can be stored in a standard ICC-format, in a Photoshop™ format, or in the format of other commonly used imaging programs. The format of the profile can be determined by the user, who may select at least one alternative from a list of file types and versions when saving it on the data storage means 4. The colour profile and the digital image are saved as separate files in the same or in parallel folders, or with the colour profile data encapsulated in the image data. This selection is made by the user. In addition, the profile can be provided with information relating it to a particular image, and vice versa. When such an image is opened, the user will automatically be asked if the related colour profile should be applied. Although no changes are made to the image data of the original image or images at this stage, single image files containing colour corrected images can of course be created at a later stage.
When the colour profile has been created a colour correcting means 14 applies the profile to the displayed reference image. The user can then approve the resulting colour correction and allow the profile to be applied to one or more images captured under the same conditions as the reference image, arranged to retrieve at least one colour profile from said data storage means and perform a colour correction to one or more of said images.
The above steps, from the point where the user indicated the reference points and provided that the reference card is correctly exposed, to the point where the created colour profile is applied, can be performed automatically without intervention from the user. For more advanced users, it is possible to put in check-points at each stage, allowing the user to approve each subsequent correction or to make personalised alterations after each step.
Figure 2A and 2B shows one embodiment of a reference card according to the invention. Figure 2A shows the reference card 1 with a black and white representation of a colour chart, while Figure 2B shows the same card with printed colour definitions. According to an alternative embodiment these colour definitions could be printed on the back of the card for reference.
According to a preferred embodiment the card 1 is rectangular and is to the largest part covered by a colour chart having a predetermined distribution of reference colours. Theoretically, there are no restrictions to the placing of or choice of colours on the card. For practical purposes, however, the card should at least include two grey levels and at least the three primary colours. The card according to one embodiment of the invention is provided with a series of grey shades G ranging from white to black (see Fig. 2B; nos. 1-7), including at least one standard grey level S. The standard grey levels are used to determine whether the card is evenly exposed and are arranged spaced apart from each other. According to a preferred embodiment one such grey level S is placed in each corner of the colour chart. Additional groups of related colours includes a group of skin tones T and a group of primary colours P. The skin tones are used for weighting a colour profile towards skin colours when photographing portraits, while the primary colours are used to weight a colour profile towards blue, green and yellow when photographing landscapes.
The reference card can also be provided with a printed code containing information about the type or make of card, which in this example is a QPcard 201 ©. This facilitates the selection of card type by the user, as
different cards of different makes, sizes and colour combinations can be recognised and processed by the arrangement according to the invention. According to one embodiment the software is arranged to query the user about the type of reference card used, giving a list of possible choices that can be processed be the software. The colours indicated in Figure 2B, showing a number of standard reference colours and colour codes are given by way of example only and do not limit the scope of the invention.
In order to determine the location of the reference card in the image, the card is provided with a number of reference points R or markers. As shown in Figure 2A these reference points R^ are preferably, but not necessarily, located at each corner of the colour chart. The reference points Ri can be printed as substantially L-shaped markers.
An additional marker R2 used to determine the orientation of the card can be placed at any suitable position. According to one embodiment, the marker R2 is a distinct area of a predetermined colour located at a predetermined position in a colour chart, e.g. a black square. According to a further embodiment, the marker is a printed distinct shape or logotype, or a marker combining shape and colour.
In addition the card can be provided with a cross hair pointer X, that can be used to aid focussing, and a scale M in suitable units, for making comparative or relative measurements. The units can be selected depending on the size of the photographed subject and may include millimetres, centimetres and/or inches. The scale is preferably placed along one or both of the longer of the side edges of the card.
As stated above, Figure 2B describes one example of possible colours to be used for the card, which colour are indicated using standard reference codes or similar codes. However, this is just one example of a possible combination of colours, as the colour scheme may be varied freely within the scope of the claims.
To facilitate handling of the card, it may be provided with an adhesive layer over at least a part of its rear surface. This feature allows a number of cards to be packaged in a stack when in storage, and makes it possible to releasably attach individual cards to a suitable surface when in use.
The card 1 is intended to be placed within a predetermined distance from the camera 3 and preferably substantially parallel to the focal plane F thereof. The image recognition means 9 can handle an image placed at an angle, but preferably this angle should not exceed 45° to the focal plane Even with image enhancing zoom function, allowing enlargement of the part of the image containing the card, sufficient pixels containing each area of grey level and colour must be available for analysis if the colour correction is to be reliable. Depending on the size of the card, the resolution and zoom setting of the camera and the type of subject to be captured, a maximum distance to the camera may have to be observed to enable image processing.
During subsequent processing of the captured images, the software used by the arrangement may detect a number of potential problems or errors using the reference points Ri and the colour chart. Examples of such problems are that the reference card is not evenly exposed, due to a shadow falling over the card, that it is placed at an angle exceeding a predetermined limit, or that the card has been bent. If such a problem is detected, then the user will be given a warning that the resulting colour correction may not be accurate. The user can then choose to abort the process or to override the warning. In the latter case it is possible to allow manual correction of one or more colours. Such a manual correction may be carried out by the user either before the processing continues or once the colour correction has been applied and found unsatisfactory.