SYSTEM AND MEANS FOR MATCHING COLOUR ON AN OUTPUT PATH TO A VIEWING PATH
This invention relates to the provision of a device dependent colour space translation method for use in the visual manipulation of colour images that move between different viewing and output environments such as monitors and film recorders.
BACKGROUND
Digital postproduction and digitally created visual content are being increasingly used in the film and commercial industries.
Any single frame of a film shot on celluloid can be digitised, and once digitised, a computer can be used to manipulate the image in any manner desired by the filmmaker. Increasingly, images are being captured directly in digital form making it even easier to digitally manipulate the captured images.
Changes can be easily made while an image is in the digital domain. Examples of such changes include the correction of a colour imbalance caused at the time the film was shot, insertion or removal of artefacts and objects in a scene, the addition of special effects and generally the image can be enhanced.
Once the changes are made, the digital image is typically transformed back into celluloid by using a film recorder. Such a machine is expensive to purchase and maintain and those that exist are in constant demand. The film recorder projects the digital image of a frame in a way that creates a celluloid negative which is chemically processed in a processing laboratory. The celluloid
negative is used to create a celluloid positive that is also chemically processed in a processing laboratory ready for projection onto a screen.
The film recorder and processing laboratory can however be in separate locations. This can result in delays in addition to those created by having to transfer the celluloid images by conventional means. It can take three days to create a film containing the digital changes made, and only then can the actual results of the changes be assessed, because it is only when the image is on celluloid that it can be projected and viewed under its final viewing conditions.
Clearly, it will be appreciated that a number of iterations of the above- described process will add expense and frustration to the creative process of making moving imagery.
The process of translating from film to a digital image display monitor is the same problem in principle as that associated with the matching of colours on different monitors viewing the same image. This problem is even less obvious and some times goes unnoticed till too late, particularly if the monitors happen to be in different geographical locations.
Simply stated there is a need to provide a method of matching the viewed data representative of an image on one system or device with the way in which that same data is viewed on a further system or device. The systems or devices may be the same, as in the case of two monitors. Alternatively, the devices may be very different, as in the case of a celluloid film used to display an image on a picture theatre screen and a on a digital monitor. In any case, ideally, the intermediate viewing system, for example a computer monitor, should faithfully
represent the captured image in a way that is equivalent or very similar to the way it will be finally viewed, for example as an image on a picture screen via celluloid film.
The invention described in this specification is intended to assist in the maintenance of colour fidelity during the process of manipulating images while they exist in the digital domain and for when they are eventually transferred into the analog domain or are displayed on another digital output red, green, blue (rgb) viewing system or device.
BRIEF DESCRIPTION OF THE INVENTION
In a broad aspect of the invention a method for transforming a digital value representative of colour, that is displayed on an original rgb viewing system (P), such that when said transformed value is used by another rgb viewing system (Q) to display said colour the two displayed colours are matched, the method steps comprising: a) determine the colour primaries in a predetermined tri-stimulus colour space for P; b) for a given digital value stimulus for P determine the viewable result of P as an expression of said colour primaries; c) determine the colour primaries in said predetermined tri-stimulus colour space for Q; d) determine the response of Q by combining the viewable result of P and the primaries of Q; and e) determine the stimulus for Q from the response by Q required to display the colour of said given digital value stimulus.
In a further aspect of the method the tri-stimulus colour space is CIE XYZ.
In a yet further aspect the colour primaries are obtained from only the pure maximum colour components as stimulus of P and Q.
In a further aspect of the invention the colour primaries are obtained from a predetermined range of pure colour components as stimulus of P and Q.
In yet a further aspect the stimulus to response determination of step b) is obtained by empirical measurement of P.
For a further aspect of the invention the response to stimulus determination of step e) is obtained by empirical measurement of Q.
Specific embodiments of the invention will now be described in some further detail with reference to and as illustrated in the accompanying figures. These embodiments are illustrative, and not meant to be restrictive of the scope of the invention. Suggestions and descriptions of other embodiments may be included but they may not be illustrated in the accompanying figures or alternatively features of the invention may be shown in the figures but not described in the specification.
BRIEF DESCRIPTION OF THE FIGURES
Fig 1 depicts a functional block diagram of the process used to gather the information required to create a grey scale 'profile' of a particular rgb viewing device (such as a computer monitor);
Fig. 2 depicts a functional block diagram of the process used to gather th information required to create a colour primary 'profile' of a particular rgb viewing device (such as a computer monitor);
Fig. 3 depicts a pictorial representation of the process used to gather the information required to create a profile of a particular other rgb viewing device, using a spectrodensitometer to measure the colour density of differently coloured films which themselves are generated using a range of full colour primaries and a grey scale values;
Fig. 4 depicts a functional block diagram showing an example of the pat the image takes in the described system showing the locations of different stimulus and response of the various rgb viewing devices and systems;
Fig. 5 depicts the steps associated with determining for a given digital value stimulus for the rgb viewing system the viewable result of the system as expression of the colour primaries;
Fig. 6 depicts the steps associated with determining the colour primaries in a predetermined tri-stimulus colour space of another rgb viewing system;
Fig. 7 depicts the steps associated with transforming a digital value representative of colour useable to stimulate another rgb viewing system to display the colour; and
Fig. 8 depicts a subdivision search process on the viewing profile curve to determine what stimulus needs to be provided to the viewing profile to receive the required amount of each primary.
DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
The following embodiment is used to illustrate use of the invention when the intermediate or viewing system is a computer monitor on which digital images are manipulated. Those same images will eventually be displayed on another viewing system involving the projection of those images through celluloid film on to a picture screen.
What has not been previously available to postproduction houses is a method and means for ensuring that the colours displayed and created on the computer screen are the same as those to be projected on to the big screen in a picture theatre.
The first steps in providing such a method and means is to record the characteristics of the viewing devices used in the process of displaying the image. In this embodiment the device upon which the images are viewed and amendments are made is a computer monitor. A celluloid film is the means by which the final images are viewed by projection on to a screen so it is the film itself that is characterised by the measurements.
Figures 1 and 2 depict pictorial representations of suggested steps for determining the brightness and colour actually displayed on a particular monitor
for which known red, green, blue (r, g, b) representations of colour have been input.
A step of the process is to set up the monitor in such a way that it will provide the best response possible for viewing the colours produced by the screen. This involves guiding the user to set the black point, the brightness and contrast of the monitor and finally the white point of the monitor.
Once these parameters are within acceptable ranges the process of the invention involves the creation of a mapping of grey scale (r, g, b) values input to the monitor against the brightness produced by that particular monitor for the grey scale value being input.
In a practical embodiment, selected grey scale values are generated and the monitor's brightness output is measured and expressed in an internationally recognized colour space metric. No monitor will produce a perfect grey colour representation and the actual amounts of red, green and blue output that become visible on the monitor's screen corresponding to a known grey scale input needs to be detected and measured. The measurement of those values is done with a colourmetric sensor device.
In this embodiment, the tri-stimulus colour space within which the colour components measured are expressed is the internationally recognised CIE value XYZ format. This however, is not the only metric that can be used for this process, alternatives include but are not limited to CIE xyY, CIE xyz and any defined tri-stimulus colour space. It is of some advantage that each component in the CIE XYZ format is one of a tri-stimulus system that in combination expresses
any colour in positive amounts of each stimulus. This is a useful characteristic but not an essential feature of the method of the invention.
By way of explanation only, the CIE YXZ format evolved in 1931 from the efforts of the Commission Internationale de TEclairage (CIE) to allow for the expression of colour by three new primaries X, Y and Z. This new format involved only positive coordinates as negative weights were inconvenient to calculate (before computers and such like) and they would replace r, g and b for the purpose of computations. It is important to note that these new primaries are not themselves visible. The matching curves for them are obtained only by transformation from the rgb curves. No part of the CIE XYZ colour matching curves is negative, which means any colour can be referenced by some positive combination of x, y and z. Y was chosen to represent the luminance (the intensity of the light) and can be quantified in the units of candela/ square meters. The non-luminance part of light is referred to as the chrominance. The format is based on the principal that any light source is made up of a spectrum of monochromatic lights, and thus it is possible to compute the XYZ colour coordinates of any spectrum P() by summing (actually integrating) the contribution of all of the monochromatic elements:
X=k P() d Y=k P() d Z=k P() d
The colour matching curves between x,y,z and X,Y,Z are normalised to a unit power, so k is a scale constant to be set according to the source.
For the purpose of determining the viewing profile of the computer monitor, it has been found adequate to input to the computer monitor, sixteen (16) grey scale values (e.g. evenly spaced between white (r=0,g=0,y=0) and black (r=255,g=255,y=255). The resultant visual output of the monitor is measured with a colourmetric sensor device that outputs a CIE value, as is shown pictorially in Fig.l.
To perform the measurement, the colourmetric sensor device is held against the screen of the monitor while a particular grey scale value is input and the sensor output is recorded.
It has been found that several measurements for each grey scale value should be recorded and then an averaged. Furthermore, it has been found that after storing all the respective averaged measured values it is preferable to convert the XYZ values to xyz. While in the xyz format it is useful to apply a smoothing function to the low-end xy values that compensates for a slight aberration in the chromaticity (xy) readings of the device at low illumination. Averaging the xy results over time is one viable smoothing function. These values are then converted back into XYZ space before continuing the steps involved in the process. This correction step is preferable only, as the majority of values are unaffected by the aberration. If an XRITE DTP92 was used to take readings from the computer monitor screen, it has been determined that the response at the low end of the brightness scale is not totally accurate so a practical approach is to take multiple readings and average them.
Interpolating grey scale values in-between the 16 predetermined values, although not measured, can in a graphical representation, be achieved using a
best-fit curve algorithm. Alternatively, an explicit formula may be determined from the results or a mathematical transform can be created by techniques known in the art.
This process provides the brightness (luminance) characteristics of a particular monitor. If the output device were a film stock its luminance characteristic would be measured in rgb density.
Fig. 2 depicts a pictorial representation of a step used for determining the primary colours of the particular monitor. It should be noted that it matters not in what order the brightness or colour of the monitor for particular inputs is measured.
In practical terms it has been found adequate to generate the digital values of the pure maximum colour components red, blue and green {rι(r 255,-,-);g2(-,g 255,-);b3(-,-,b 255)}, and input them in turn to the monitor and measure off the screen with a colourmetric device, the corresponding CIE values {r(X ,Y1,Zι);g(X2,Y2,Z2);b(X3,Y3,Z3)}. The pure primary colour components are produced by supplying a first full stimulus to a single channel and no stimulus to the other two channels of the RGB viewing device such as a computer monitor. In a normalised system this equates to a stimuli of (1,0,0) for red, (0,1,0) for green and (0,0,1) for blue. The sensor output is measured in CIE XYZ and stored as an XYZ triple.
This step of the process characterises how the computer monitor visually represents the primary colours. Monitors do not as a rule faithfully reproduce
the colours input to them. A monitor can also reduce or increase the saturation of the colour it is supposed to be displaying on screen.
The results of these two measurements steps and are then stored in a file in a preferable format as follows.
Example Monitor Profile.
CMSMONITOR 2 date:972003691
1.000000,0.000000,0.000000,23.000000,12.400000,1.280000 0.000000, 1.000000, 0.000000, 14.360000, 29.600000, 5.300000 0.000000,0.000000,1.000000,6.800000,2.800000,34.940000 0.000000,0.000000,0.000000,0.080000,0.000000,0.000000 0.066667,0.066667,0.066667,0.100000,0.100000,0.040000 0.133333,0.133333,0.133333,0.400000,0.360000,0.300000 0.200000, 0.200000, 0.200000, 0.900000, 0.900000, 0.800000 0.266667,0.266667,0.266667,1.900000,1.900000,1.700000 0.333333,0.333333,0.333333,3.200000,3.300000,2.900000 0.400000, 0.400000, 0.400000, 5.000000, 5.100000, 4.600000 0.466667,0.466667,0.466667,7.200000,7.400000,6.760000 0.533333,0.533333,0.533333,10.000000,10.200000,9.380000 0.600000,0.600000,0.600000,13.280000,13.500000,12.460000 0.666667, 0.666667, 0.666667, 17.000000, 17.400000, 16.020000 0.733333,0.733333,0.733333,21.300000,21.800000,20.140000 0.800000, 0.800000, 0.800000, 26.180000, 26.700000, 24.740000 0.866667, 0.866667, 0.866667, 31.480000, 32.100000, 29.900000 0.933333,0.933333,0.933333,37.440000,38.160000,35.520000 1.000000,1.000000,1.000000,43.960000,44.760000,41.700000
The monitor profile stores pairs of triples in the format
Red_stimuli, Green_stimuli, Blue_stimuli, X_response, Y_response, Z_response
In this format the three primaries (1,0,0,X,Y,Z), (0,1,0,X,Y,Z), (0,0,1,X,Y,Z) are provided first followed by any number of other pairs of triples. This is not an
difficult or obtuse format therefore a skilled operator can manually check for reasonableness in the values collected by the equipment and stored for use.
Although not depicted in a figure it is also possible to input predetermined ramped colour values to the computer monitor, say evenly spaced values of red, then evenly spaced values of green and then evenly spaced values of blue, in (r, g, b). The values may be, using red for example, rl (50,0,0); r2(100,0,0); r3(150,0,0);r4(200,0,0); r5(250,0,0). Similar values for each ramped colour can be input, and a measurement with the colourmetric sensor device is recorded in CIE values.
The result of this additional process is empirical values for the colours.
A set of these values can be used to interpolate the output CIE value for any digital colour input. This creates a device specific colour space, which in this embodiment, is the CIE standard. However, any predetermined colour definition scheme can be used to define the colour space used in the process of the invention.
In this embodiment, to determine the characteristics of the final viewing path that will occur and to which the colours displayed on the computer monitor are to be matched for a particular digital value stimulus, a celluloid film path is described and depicted.
Figure 3 depicts a pictorial representation of the process involved in determining the colour profile of a particular device's image viewing process. In
this embodiment the particular device end result is a celluloid film frame produced by a film recorder. Negative/ positive film stock is typical of the stock used to do the final recording of a cinematographic film for projection on to a picture theatre screen. To a large degree it matters not what the processing path is that produces the output film stock as all of the subtle and not so subtle colour altering steps in its creation are embedded in the final stock. It is the final stock that is measured by the process steps described herein. Film is commonly used when translating the digital output of a post production process back into a form ready for display on a picture theatre screen.
Thus the result of the process described and the subject of the embodiment is clearly device dependent.
Referring to Fig. 3 a standard light source is shone through the processed film stock, the light source typically being part of a spectrodensitometer apparatus that is used to measure the density of the colour in processed film.
Preferably, multiple digital representations of frames having equal amounts red, green and blue stimulus are produced by the processing path and the film stock output transformed into a film "chip", as they are referred to. Then a series of frames having stimulus in only one of the colours red, green and blue, are created and provided to a film recorder for eventual transformation into further film "chips".
Each chip represents values that are stored as lObit logarithmic colour components. The format in which the values are stored is called CINEON (a trademark of the Eastman Kodak Company).
Preferably, it has been found sufficient, to create 40 evenly spaced ramped colour "chips" and evenly spaced primary colour "chips". Each "chip" is recorded to negative film with a digital film recorder, for example, a KODAK Lightning (also a trademark of the Eastman Kodak Company) and then a positive film "chip" is processed.
A specrodensitometer measures the density of the various colours represented by the "chips". However, other types of "chips" may be used to assist the accuracy of the determination and characteristics of the film stock.
The output of the spectrodensitometer is provided as a set of three positive density readings. A positive density reading is also referred to as a "status A reading".
A mathematical transformation to describe a curve of best fit can be created so that any CINEON (r, g, b) value can be represented as a density. A curve of best fit can be calculated using the preferred range of chips.
A spectrodensitometer also measures device dependant colour saturation and in this embodiment it is preferable that the measurement is provided in CIE XYZ values. Such readings are required for the three primaries to return three values of CIE XYZ type.
These values are then stored in a preferred file format with a structure like the following:
o o o o o o o o o o σ o o o σ
390000.00
The data entries are of the format:
<cineon code value> : <red density>, <green density>, <blue density> The first five entries are as follows. Red XYZ Green XYZ Blue XYZ Black Density White Density
Once the data comprising the viewing path characteristics (Fig's 1 and 2) and the output viewing path characteristics (Fig.3) are available, it is possible to use that information in a method for achieving the aims of the invention.
The example being, as described previously, in the filed of postproduction digital image enhancement.
The filmmaker has created a film comprising thousands of frames of celluloid film and digital postproduction on each frame requires that each frame be scanned to create a digital representation thereof.
Current arrangements for postproduction digital enhancement merely display the scanned image on a monitor screen. This will unavoidably include colour and intensity variation characteristics of the monitor itself and not reflect how the image will be eventually be viewed. This approach necessarily affects the usefulness of any digital postproduction processing undertaken using that monitor.
If the postproduction process is to properly reflect the desired effects, all the rgb colour values for every pixel of each frame needs to be represented on the postproduction computer monitor in a manner which equates to the way the film will eventually be seen by the cinema patron. In doing so it is imperative to take into consideration not only the final output path but also the characteristics of the monitor itself.
It is also conceivable that the method disclosed in this specification might be used to normalise the viewing of a digital image on more than one computer monitor. The result of which will be that each monitor will display the colour and intensity of the digital image in the same way regardless of the differences in monitor characteristics. This facility can be useful within postproduction houses as this ensures consistent and repeatable colour representation in-house where collaboration towards a common goal is imperative.
Alternatively, it is possible to arrange the invention to allow the image projected via a celluloid film to match the image colour created on a computer screen.
Thus, the input and output path used is merely a matter of choice and not limited by equipment or the imagination.
It is also conceivable that a digital file may originate from many different sources some of which reflect the trend to collect image files in digital format in the first place. Examples of this include, a computer based renderer that creates files based on instructions from an artist, a file that is captured from a video
device such as a Betacam or a file that is created by an artist in a digital painting software package (eg. PhotoShop - trademark of Adobe Systems Incorporated).
Once these digital files exist, the invention can be used to view, during the postproduction processes, the colours as they are intended to be viewed when displayed using the intended output device such as an other monitor or a celluloid film projected onto a picture theatre screen. The output colour space is clearly not limited to the input medium or any intermediate postproduction medium but clearly intended to reflect the final output viewing medium.
To achieve any of these aims it is preferable for the following process steps to be followed. The outcome of such an arrangement is pictorially represented in Fig. 4 while Fig's 5 to 8 represents most of the steps involved.
Transforming a desired/ output (r, g, b) triple into a stimulus for the viewing path involves the following preferable steps for the embodiment described herein.
The first step in determining the transformation required is to convert the information stored about the two profiles (view and output) into curves that can be used typically in a look up table (LUT) format that is ideal for use in the computer environment. The processes described herein are all capable of being implemented in software. However when and if suitable it would be possible to make one or more of the steps occur (including the use of a LUT), as the result of an interaction with a hardware device such a computer hardware device dedicated to the performance of such a step. Such an approach is possible and sometimes preferable since the speed at which the hardware device can operate can exceed the equivalent operation performed in software.
If the device being modelled is a computer monitor then the preferred process is as follows:
Loading the computer monitor profile involves reading in the data as it is stored and creating 3 curves, one for each colour (channel) used by the monitor.
Looping through the stored data and firstly storing the values of the three primaries as phosphors creates three curves. Phosphors are the XYZ readings corresponding to (1,0,0,X,Y,Z), (0,1,0,X,Y,Z), (0,0,1,X,Y,Z).
These phosphors are then stored in a matrix representative of the primary XYZ vales, as also depicted in Fig. 5.
[ rX,rY,rZ ] [ gX,gY,gZ] [ bX,bY,bZ]
This matrix is inverted to obtain a decomposition matrix as is also depicted in Fig. 5. Then for each entry of grey scale wedge values the red, green and blue components are stored in the appropriate curve by taking the red, green and blue components of the XYZ reading the decomposition matrix.
Essentially for each digital value stimulus provided to the particular monitor, the resultant 'colour' is stored as an expression of the amount of each phosphor that is recorded.
Thus if a particular colour is to be viewed on that monitor then the curves provide the stimulus input to the monitor to replicate that colour, as depicted in
the stimulus to normalised response curve in Fig. 5. In our embodiment if the film stock output value (the response of the other rgb viewing system) needs to be the pure maximum colour component of red then the relevant stimulus values can be determined from the graphs described. The graph or in other embodiments a LUT or mathematical expression will denote the relevant amounts of red, green and blue components to be input as stimulus to the monitor.
If the other and in this embodiment a final viewing system or device is a film or density based system then the preferred process is as follows.
The principle for creating an information store of information relating to a film profile is similar to that for a monitor but rather than XYZ information, rgb density information is obtained from a densitometer used to provide empirical information about the film stock. First, store the primaries as the XYZ values that they are stored in. Black and white is used for scaling the values to fit into the viewable range. Fig. 6 pictorially represents this step in the process.
Each of the rgb density curves is then stored as 1/ power (density). This converts the density (non-linear with higher values meaning less colour) into a normalised range (0->l with higher values meaning more colour).
Once the information from the two profiles being used is obtained, it is possible with the processes described to give the user extra control by allowing them to modify variables that can be added to the process to complete the simulation. More specifically it is possible to add 'code values' to the underlying calculations at the conversion stage so that the system mimics the application of
'film printer lights' which function by adding cineon code value increments to an underlying log image.
Obviously the user needs to be able to select the viewing and output profiles that are to be used and other normal interface controls are provided where necessary.
There are two approaches that can be taken for determining the transformation that can be used.
The 2d look-up-table (LUT) approach is a fast approximation of a transformation useful for viewing data, as it will appear on chosen output path. It is not a true representation but because it can be implemented directly into the gamma maps of a monitor it is very fast and hence more readily useable. Thus when the term matched is used in this specification and claims it is used in the practical sense and not the mathematically precise sense.
This approach requires building look-up-tables for each of the channels in the monitor. So given two profiles and any of the user settings described above look-up-tables for red, green and blue can be created.
First, determine what 'white' is on the output profile by looking up the colour (1,1,1) on the curves on Fig. 6.
Looking up a colour on a profile requires adding the XYZ value of the red phosphor, scaled by the 'luminance' component stored in the red curve for that stimulus value, to the XYZ value of green, also scaled, and the XYZ value of blue,
also scaled. This takes advantage of the fact that X white = Xred + X green +Xblue etc.
A scaling factor is then determined so as to ensure the output paths ability to produce light is not exceeded but still maintain a maximum possible range.
A primaries matrix is populated from the viewing profile primaries and a decomposition matrix is created by inverting the primary matrix.
For each value in the look-up-table that is being created it is necessary to determine what colour it is that is required to be matched to, by looking up the colour (value, value,value) in the output profile and then multiplying the decomposition matrix by this colour. This procedure provides the 'amount' of each primary on the viewing path that is required. These steps are pictorially represented in Fig. 7.
A subdivision search, refer to Fig. 8, is then performed on the viewing profile curve to determine what stimulus needs to be provided to the viewing profile to receive the required amount of each primary. The results of this search are stored in the look-up-table.
Another possible approach in contrast to the 2-dimensional LUT approach creates a large computational load but which can create a 3-dimensional mapping of colour values and uses geometric transformation in XYZ space. This accounts for changing primaries and more accurately tracks saturation differences across the range.
A 3-dimensional space represents, not unlike the tri-stimulus space of the previous embodiment, the XYZ values wherein the red, green and blue primaries exist. By taking measurements of each primary at varying luminance an arc shaped plot represents the gamut of the colour space being measured.
This allows the particular system, whether that be the rgb viewing system or the other rgb viewing system, to be defined as a space in XYZ space and then each colour in that space can be defined as a product of the red, green and blue components.
The transform for determining the stimulus for the viewing system is calculated by firstly determining if the required stimulus for the rgb viewing system exists in the viewing colour space. If it does, a simple transform is used. However, if it is not then the geometrically closest value in the space is used. From that value it is possible to determine what red, green and blue components are required within the viewing space to produce the required colour on the other rgb viewing system.
It will be appreciated by those skilled in the art, that the invention is not restricted in its use to the particular application described and neither is the present invention restricted in its preferred embodiment with regard to the particular elements and/ or features described or depicted herein. It will be appreciated that various modifications can be made without departing from the principles of the invention, therefore, the invention should be understood to include all such modifications within its scope.